Skip to content

Empty index HnswDocumentIndex causes error next time script runs #1446

@wizrds

Description

@wizrds

When creating an empty HnswDocumentIndex, if there were no files indexed (.index called) then the next time the object is instantiated with the same work_dir it will fail.

Python version: 3.11.2
Docarray version: 0.30.0

Reproducible Example:

from docarray import BaseDoc
from docarray.typing import NdArray
from docarray.index import HnswDocumentIndex

class Text(BaseDoc):
    embedding: NdArray[128]
    text: str


index = HnswDocumentIndex[Text](work_dir="./index")

Output:

Traceback (most recent call last):
  File "/home/tim/Documents/git/_/docarray_example.py", line 11, in <module>
    index = HnswDocumentIndex[Text](work_dir="./index")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tim/Documents/git/_/.venv/lib/python3.11/site-packages/docarray/index/backends/hnswlib.py", line 116, in __init__
    self._hnsw_indices[col_name] = self._load_index(col_name, col)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tim/Documents/git/_/.venv/lib/python3.11/site-packages/docarray/index/backends/hnswlib.py", line 386, in _load_index
    index.load_index(self._hnsw_locations[col_name])
RuntimeError: Cannot open file

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions