Skip to content

feat: add nested access for document array#956

Merged
JoanFM merged 26 commits into
feat-rewrite-v2from
feat-nested-access
Dec 23, 2022
Merged

feat: add nested access for document array#956
JoanFM merged 26 commits into
feat-rewrite-v2from
feat-nested-access

Conversation

@anna-charlotte

@anna-charlotte anna-charlotte commented Dec 19, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Signed-off-by: anna-charlotte charlotte.gerhaher@jina.ai

Nested access in V2 is possible by calling the nested attributes you want to access on a DocumentArray, e.g. da.image.title. This does not work for nested DocumentArrays though:

class Author(Document):
    name: str


class Book(Document):
    authors: Optional[DocumentArray[Author]]
    content: str


da = DocumentArray[Book](
    [
        Book(
            authors=DocumentArray[Author]([Author(name='Jenny'), Author(name='Ben')]),
            content=f'hello{i}'
        ) for i in range(10)
    ]
)

authors = da.authors.name  # does not work since da.authors is a list of DocumentArrays and does not have

# instead do this:
authors = da.traverse_flat(access_path='authors.name')  # returns List of strings, with len = 20

traverse_flat takes the access path and returns a list of those attributes, flattened on the first level.
da.traverse_flat(access_path='authors.name') therefore returns a list of 20 strings instead of list of ten lists of 2 names each.

  • add traverse_flat + test
  • check and update documentation, if required. See guide

@anna-charlotte anna-charlotte linked an issue Dec 20, 2022 that may be closed by this pull request
Investigate nested access from DocumentArray ("access paths") #957
Closed
@anna-charlotte anna-charlotte mentioned this pull request Dec 20, 2022
Closed
47 tasks
anna-charlotte added 9 commits December 20, 2022 15:00
test: add test for traverse flat
47f0837 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
test: update test for traverse flat
434ab27 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
chore: change ruff line length from 88 to 100
e4f8486 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: add traverse mixin with traverse flat
6cbacb4 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
chore: revert pyproject toml ruff
1f00d9f 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
docs: fix docs in traverse mixin
9ed50c9 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
c651fc8 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
a2a18b6 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
fd0f954 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
anna-charlotte added 9 commits December 20, 2022 15:14
fix: add allmixins
6a52969 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
8c7e84a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
bf0e617 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
fe3fd8f 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
refactor: move traverse flat to abstract array
85c69bf 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: traverse flat for stack mode
e0d9f1a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
docs: add docs for stacked da
130dea2 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: traversal flat for stacked and unstacked in abstract array
201a42d 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: move traverse_flat to array and stacked array instead of abstra…
af74fb7 
…ct array

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
anna-charlotte added 4 commits December 22, 2022 18:00
fix: remove size for instance check
cdb49f5 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: change instance check
c4038cb 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
ee2a0e3 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: torchtensor and ndarray instead of only tensor
aab4c6a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
JoanFM
JoanFM requested changes Dec 23, 2022
View reviewed changes
Comment thread docarray/array/abstract_array.py Outdated
fix: add type hint and len check in flatten
710b927 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
JoanFM
JoanFM reviewed Dec 23, 2022
View reviewed changes
Comment thread docarray/array/array.py Outdated
anna-charlotte added 3 commits December 23, 2022 11:42
refactor: rename flatten to flatten one level
fe8bf3c 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
test: add test for flatten one level
baa1c31 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: revert change in array get item
459c4dd 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte anna-charlotte marked this pull request as ready for review December 23, 2022 10:47
@github-actions

github-actions Bot commented Dec 23, 2022

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-feat-nested-access--jina-docs.netlify.app 🎉

@JoanFM

JoanFM commented Dec 23, 2022

Copy link
Copy Markdown
Member

In the v1, we decided to hide traverse flat behind the [@] accessors, are we expecting to expose this now?

@anna-charlotte

anna-charlotte commented Dec 23, 2022

Copy link
Copy Markdown
Contributor Author

@Joan We haven't made the final decision for this yet. For now, we decided on exposing it, but want to reevaluate this when Sami and Johannes are back.

@JoanFM JoanFM merged commit 6e66c31 into feat-rewrite-v2 Dec 23, 2022
@JoanFM JoanFM deleted the feat-nested-access branch December 23, 2022 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JoanFM JoanFM JoanFM approved these changes

Assignees

No one assigned

Labels

area/core area/testing component/array size/m

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Investigate nested access from DocumentArray ("access paths")

2 participants

@anna-charlotte @JoanFM