Skip to content

vector search fails for some queries and silently degrades to BM25-only (storage error: vector search failed) #38

Description

@citron07r

Summary

On an onnx-jina-coreml backend, vector search failed for some queries but not others against the same index. Vera degrades to BM25-only results and continues, so the only signal is a WARN line.

WARN vera_core::retrieval::hybrid: vector search failed, falling back to BM25-only results
  error=storage error: vector search failed

Reproduction

Index used: a clone of this repository at v1.0.0, 375 files, 4790 chunks, embeddings jinaai/jina-embeddings-v5-text-nano-retrieval (fp16), backend onnx-jina-coreml.

Counting only the WARN line, one run per query:

query vector search failed
fusion no
hybrid fusion no
hybrid fusion work no
how does hybrid fusion work yes
reranking no
cross-encoder reranking of search candidates yes

It is deterministic per query across repeated runs, not flaky. Longer, more natural-language queries are the ones that fail, which is the shape of query the tool is designed for.

Notes

  • Reproduces identically on the released v1.0.0 binary and on a local build, so it is not related to any local change.
  • The error surfaces at vector_store.search(...) in crates/vera-core/src/retrieval/vector.rs:69, wrapped as VectorSearchError::StorageError with context "vector search failed". The underlying store error is not included in the logged message, so I could not get further without patching.
  • RUST_LOG=debug did not add detail beyond the same WARN.

Impact

Silent quality loss. The command exits 0 and returns plausible results, so a user has no reason to suspect the vector half of hybrid retrieval did not run. Combined with reranking being unavailable on this backend (see #33), a CoreML user can be served pure BM25 output while believing they are getting the full pipeline.

Suggestion

Propagate the underlying store error into the warning so the cause is diagnosable from the log. A hybrid search that loses a whole retrieval arm may also deserve louder treatment than a WARN.

Environment

Apple M4, macOS 26.6.1, ONNX Runtime 1.24.4, vera 1.0.0, backend onnx-jina-coreml.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions