Skip to content

[NIDX-03 workstream] Replace the per-segment series index #14003

Description

@hanahmily

[NIDX-03 workstream] Replace the per-segment series index

Parent: #13990
Blocked by: #14007

Tracking parent — do not apply Backlog. Split this workstream just in time after NIDX-02 closes; downstream RED contracts must be based on the writer and reader seams actually merged on main.

End-state boundary

banyand/internal/storage.newSeriesIndex selects native sidx for Measure, Stream, and Trace. The completed workstream owns field-set-aware insert/update/delete, projection, exact/prefix/wildcard identity, field dictionaries, composed filters, deterministic series/index/time sort, index-mode Measure, statistics, snapshots, cache reset, and mixed-version external segment receive.

Required behavior

  • Fixed product fixtures return independently declared series IDs, fields, timestamps, and versions.
  • Native and pinned legacy nodes exchange raw segments in both directions without replay or conversion.
  • Interrupted, invalid, and duplicate external transfers never become partially visible.
  • Restart selects a complete committed generation.
  • CRC32 fields remain layout-only and are never calculated or validated.

Just-in-time decomposition gate

After #14007 closes, use its measured boundaries and run reports to file ordered live-caller leaves. Each leaf adds one observable series capability and activates it through a named Measure/Stream/Trace path; no sidx core, codec, query helper, or receiver may merge solely for future use. Apply Backlog to one unblocked leaf only.

Completion criteria

All future leaves merge; the complete newSeriesIndex role and replication path are native; compatibility, crash, race, and product suites pass; and this workstream closes before NIDX-04 decomposition.

Lexical non-regression

Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive bluge tokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutral legacy oracle or compatibility writer labels plus an immutable revision or content hash rather than adding a retired module name.

This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.

Design

BDB-NIDX-SPEC-001 revision 0.2 — NIDX-03 behavior

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions