Skip to content

2.18.6

Latest
Latest

Choose a tag to compare

View all tags
@KennethEnevoldsen KennethEnevoldsen released this 22 Jul 13:28
2.18.6
fa36ee7

2.18.6 (2026-07-22)

Fix

  • fix: type mismatch between some whisper models and mel features (#4990) (acbf4ca)

Unknown

  • [MOEB] Register whisper-large-v3-turbo (#4989) (f095887)

  • Cite the Fusion Embedding arXiv paper in both ModelMetas (#4987) (0ae18d6)

  • task: add VSC2022 video-to-video copy-detection retrieval (v2v) (#4985)

  • task: add VSC2022 video-to-video copy-detection retrieval (v2v)

  • chore: add descriptive statistics for VSC2022Retrieval

  • lint

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (80f6e3b)

  • task: add MomentSeeker composed video retrieval (it2v, vt2v) (#4967)

  • task: add MomentSeeker composed video retrieval (it2v, vt2v)

Adapts MomentSeeker long-video moment retrieval (CC-BY-NC-SA-4.0) into
corpus retrieval: 30-second non-overlapping 360p chunks of the source
videos form the corpus, with relevance from overlap against annotated
answer intervals. Two directions: text+image queries and text+video
queries, 400 each. Self-contained: adds the it2v TaskCategory code.
Closes #4944.

  • chore: add descriptive statistics

  • style: apply ruff format to moment_seeker.py

  • task: use map_at_5 as main_score for MomentSeeker (matches the paper) (9082aa7)

  • Add new version for LCO-3B (#4971)

Add new version for LCO-3B model. This version release on May 2025 and has shown better performance compared to it's predecessors. (4f1a28c)

  • model: Update fusion-embedding-2 revision to v0.2 (9451b840f0d1) (#4983)

Update fusion-embedding-2 revision to v0.2 (9451b840f0d1)

The v0.2 release fine-tunes on the expanded AudioCaps 2.0 pool; non-audio
paths are unchanged (bitwise-verified on the released artifact). Weights and
card at the new revision; the v0.1 pin remains valid in history. (7967939)

  • Fix JinaV4Wrapper crashes and missing PairClassification prompt mapping (#4981)

Three related bugs surfaced while running PairClassification tasks with
jina-embeddings-v4:

  • get_text_embeddings/get_image_embeddings called task_type.startswith()
    without a None guard. get_prompt_name legitimately returns None when a
    task isn't in the model's prompt map, which the surrounding code already
    handles elsewhere (jina_task_name = model_prompts.get(task_type) if
    task_type else None) -- these two call sites just missed the same guard.

  • jina_embeddings_v4's model_prompts was missing a PairClassification entry
    that every other jina registration in this file already has (v3 maps it
    to "classification"; v4 has no classification adapter, so text-matching
    is the correct equivalent -- the same adapter v4 already uses for STS).
    Without it, PairClassification tasks silently fell back to the retrieval
    adapter instead of text-matching.

  • JinaV4Wrapper.encode() returned raw GPU tensors (or lists of GPU tensors)
    without converting to numpy, unlike JinaWrapper (v3)'s equivalent method
    which already does this. Downstream evaluators that call np.asarray() on
    the result (e.g. PairClassificationEvaluator) crashed on GPU tensors. (5f517f5)

  • task: add VimSketch query-by-vocal-imitation retrieval (a2a) (#4963)

  • task: add VimSketch query-by-vocal-imitation retrieval (a2a)

Adds the first audio-to-audio retrieval task: 2,168 vocal-imitation
queries (max 4 per reference class) against 544 reference sounds from
the VimSketch dataset (Vocal Imitation Set + VocalSketch, CC-BY-4.0,
Zenodo 2596911). Ground truth is the instance-level imitation-to-
reference mapping encoded in the dataset. Closes #2242.

  • task: add VimSketch imitation clustering (AudioClustering)

Clusters the 2,168 vocal imitations by imitated reference class (542
classes), derived from the retrieval qrels of the same dataset. Data
lives in a clustering config of the existing dukesun99/VimSketch repo.

  • chore: add descriptive statistics

  • Update citation

Update citation

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • fix: canonicalize VimSketch bibtex formatting for citation test

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (9d15297)

  • task: add CLD composed audio retrieval (at2a) (#4965)

  • task: add CLD composed audio retrieval (at2a)

Adds the first composed audio-text-to-audio retrieval task, built from
the CLD subset of ADIFF (ICLR 2025, MIT annotations): a source Clotho
v2.1 evaluation clip plus a language description of how the target
differs, retrieving the target among 1,045 clips. 2,000 queries sampled
with a fixed seed from the 5,225 evaluation pairs. Data repackaged in
standard corpus/queries/qrels layout. Related to #4943 (AudioDiffCaps
itself cannot be rebuilt publicly: its repo ships no s2 JAMS).

  • Update mteb/tasks/retrieval/eng/cld_at2a_retrieval.py

update metadata

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • chore: add descriptive statistics

  • task: add CLDA2TRetrieval, audio-to-text difference retrieval (a2t)

  • chore: add descriptive statistics for CLDA2TRetrieval

  • format

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (3e4ec52)

  • task: add SPEECH-COCO spoken-caption image retrieval (a2i, i2a) (#4964)

  • task: add SPEECH-COCO spoken-caption image retrieval (a2i, i2a)

Adds audio-image cross-modal retrieval in both directions using
SPEECH-COCO (arXiv 1707.08435, CC-BY-4.0): spoken captions synthesized
with eight TTS voices over MS-COCO images. Data is a deterministic
downsample of the mteb/SpeechCoco validation split originally processed
in PR #3070 (2,048 images, 1,000 queries per direction), repackaged in
standard corpus/queries/qrels layout. Requires the a2i/i2a TaskCategory
extension. Closes #2298.

  • chore: add descriptive statistics (e8f0b21)

  • model: add DINOv3 ViT family (6 checkpoints) (#4976)

Adds ModelMeta entries for the facebook/dinov3-vit* web-image
(LVD-1689M) checkpoints, reusing the existing DINOModel wrapper.
ConvNeXt and satellite (SAT-493M) variants are left out for now:
ConvNeXt outputs spatial feature maps that the CLS-pooling wrapper
does not handle. Repos are gated behind the DINOv3 license.

Closes #3031 (e490c78)

  • dataset: add cleaned STSBenchmark v2 (#4876)

  • dataset: add cleaned STSBenchmark v2

  • reupload

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (7f64db9)

  • Add voiceclap-large-v2 (#4972) (b1d6882)

  • task: add SoundingEarth audio-image retrieval (a2i, i2a) (#4966)

  • task: add SoundingEarth audio-image retrieval (a2i, i2a)

Adds cross-modal retrieval between co-located field recordings and
aerial images from SoundingEarth (CC-BY-4.0 on Zenodo): 2,048 locations
sampled with a fixed seed, 1,000 queries per direction, instance-level
ground truth. Audio is sourced per recording from Internet Archive via
the authors' pipeline. Self-contained: adds the a2i/i2a TaskCategory
codes. Closes #4942.

  • chore: add descriptive statistics (81958bb)
Assets 6
Loading