What breaks
Two low-severity refinements the #1362 security review surfaced, both fail-safe and
strictly better than what they replace, so out of #1362's scope but worth tracking.
1. A stuck teardown reservation blocks a name forever. #1362 reserves a
collection name in collection_teardowns until the durable cleanup drops its table
and releases it. If the drop permanently fails (Prefect retries exhausted, or the
commit-to-dispatch gap loses the run entirely), the row lingers and
CollectionAccessService.claim refuses that name indefinitely. There is no reaper.
This is the safe direction - a blocked name beats an adoptable populated table, and
a lost dispatch now strands a blocking tombstone rather than an adoptable table -
but a name should not be unusable for good because one worker run failed.
Fix: an age-based sweep (a Prefect flow, like the other *-sweep deployments) that,
for a tombstone older than some threshold, re-attempts the drop and clears the row;
and/or an admin CLI to release one by hand.
2. The default-KB clear has a same-tenant write-vs-drop window. #1362 fixed
#1361's P1 so DELETE /rag/collections/{name} on a default base now drops (clears)
its table while keeping the row. A concurrent ingest into that default during the
commit-to-drop window would _ensure_collection-adopt the about-to-be-dropped table
and lose those chunks to the drop. It is same-tenant and transient (re-ingesting
recovers) - writes don't pass through claim, so the tombstone deliberately does not
block them - and strictly better than the stale-searchable-chunks bug it replaces.
Worth closing if the default-clear path stays.
Where
backend/app/repositories/collection_teardown.py, app/worker/tasks/teardown_tasks.py (sweep)backend/app/services/knowledge_base.py::delete_for_rag_collection (default-clear window)
How you'd know it's fixed
A collection name whose drop failed becomes claimable again within the sweep window
rather than never; and a default collection cleared while an ingest races it does not
silently drop the freshly-ingested chunks.
Found reviewing #1362 (the teardown tombstone). Both are follow-ups to the RAG
teardown / delete governance cluster in #168; siblings of #1347/#1349/#1355/#1359/#1362.
What breaks
Two low-severity refinements the #1362 security review surfaced, both fail-safe and
strictly better than what they replace, so out of #1362's scope but worth tracking.
1. A stuck teardown reservation blocks a name forever. #1362 reserves a
collection name in
collection_teardownsuntil the durable cleanup drops its tableand releases it. If the drop permanently fails (Prefect retries exhausted, or the
commit-to-dispatch gap loses the run entirely), the row lingers and
CollectionAccessService.claimrefuses that name indefinitely. There is no reaper.This is the safe direction - a blocked name beats an adoptable populated table, and
a lost dispatch now strands a blocking tombstone rather than an adoptable table -
but a name should not be unusable for good because one worker run failed.
Fix: an age-based sweep (a Prefect flow, like the other
*-sweepdeployments) that,for a tombstone older than some threshold, re-attempts the drop and clears the row;
and/or an admin CLI to release one by hand.
2. The default-KB clear has a same-tenant write-vs-drop window. #1362 fixed
#1361's P1 so
DELETE /rag/collections/{name}on a default base now drops (clears)its table while keeping the row. A concurrent ingest into that default during the
commit-to-drop window would
_ensure_collection-adopt the about-to-be-dropped tableand lose those chunks to the drop. It is same-tenant and transient (re-ingesting
recovers) - writes don't pass through
claim, so the tombstone deliberately does notblock them - and strictly better than the stale-searchable-chunks bug it replaces.
Worth closing if the default-clear path stays.
Where
backend/app/repositories/collection_teardown.py,app/worker/tasks/teardown_tasks.py(sweep)backend/app/services/knowledge_base.py::delete_for_rag_collection(default-clear window)How you'd know it's fixed
A collection name whose drop failed becomes claimable again within the sweep window
rather than never; and a default collection cleared while an ingest races it does not
silently drop the freshly-ingested chunks.
Found reviewing #1362 (the teardown tombstone). Both are follow-ups to the RAG
teardown / delete governance cluster in #168; siblings of #1347/#1349/#1355/#1359/#1362.