Skip to content

fix(local-models): ensure_ort_runtime silently ignores its library argument after first call #168

Description

@citron07r

A11 [P3] ensure_ort_runtime silently ignores its library argument after first call

  • Files: crates/vera-core/src/local_models/ort.rs:20-33; callers local_provider.rs:305,356,366; hand-rolled workaround retrieval/local_reranker.rs:27-56
  • Category: functional / order-dependent initialization ("a check and the thing it validated can silently diverge")

Summary

The once-lock stores only Result<(), String> - not which library was loaded -
so every later call with a different resolved path returns Ok(()) without
loading it. The reranker works around this by hand via
should_acquire_ort_library(requested_ep, ort_runtime_initialized()), and its
own comment admits a mismatch would be "silent rather than loud".

Concrete symptoms today:

  • Duplicate download: on Apple Silicon, CPU and CoreML resolve to the
    identical archive (cuda.rs: both get gpu_suffix = "") but land in different
    dirs (lib/ vs lib/coreml/). A CoreML cold start via the reranker downloads
    the same dylib twice.
  • Order-dependent probes: in vera doctor --probe, the reranker probe calls
    ensure_ort_runtime(Some(~/.vera/lib/libonnxruntime.dylib)) - a file that does
    not exist on a CoreML-only install. It passes solely because an earlier stage
    happened to initialize the OnceLock from the CoreML path first. Reorder the
    checks and the probe fails misleadingly with "Run vera setup".

Forward risk: any future path initializing ORT from the CPU-only dylib before a
CUDA/CoreML session is built gets silent CPU fallback while the UI reports GPU
mode - the #33/#90 class again.

Suggested fix

Store the initialized library path in the OnceLock; on a later call with a
different resolved path, error loudly or warn naming both paths. Collapse the
macOS CPU/CoreML duplicate so one download serves both.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions