Skip to content

feat: torch tensor type#800

Merged
samsja merged 13 commits into
feat-rewrite-v2from
feat-torch-tensor
Nov 17, 2022
Merged

feat: torch tensor type#800
samsja merged 13 commits into
feat-rewrite-v2from
feat-torch-tensor

Conversation

@JohannesMessner

Copy link
Copy Markdown
Member

Goals:
Add type TorchTensor as wrapper around torch.Tensor.

closes #783

…tte-v2

# Conflicts:
#	docarray/document/mixins/proto.py
#	docarray/typing/ndarray.py
#	docarray/typing/tensor/__init__.py
#	docarray/typing/tensor/tensor.py
#	docarray/typing/url/image_url.py
#	tests/integrations/typing/tensor.py
#	tests/units/document/proto/test_proto_based_object.py
#	tests/units/typing/test_image_url.py
@JohannesMessner JohannesMessner changed the title Feat torch tensor feat: torch tensor type Nov 16, 2022
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
@JohannesMessner JohannesMessner linked an issue Nov 16, 2022 that may be closed by this pull request

@samsja samsja left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comment

Comment thread docarray/typing/tensor/torch_tensor.py Outdated
JohannesMessner and others added 2 commits November 16, 2022 11:39
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
@samsja samsja merged commit f317f20 into feat-rewrite-v2 Nov 17, 2022
@samsja samsja deleted the feat-torch-tensor branch November 17, 2022 07:51
samsja added a commit that referenced this pull request Nov 17, 2022
* feat: add tensor type for ndarray

* fix: fix mypy typing

* feat: torch tensor type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: protobuf for pytorch type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* ci: install all extras in the ci

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* refactor: make nice looking

* docs: update docarray/typing/tensor/torch_tensor.py

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>

* refactor: code style

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: black and mypy

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: suppress mypy import error

* ci: fix ci install

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
samsja added a commit that referenced this pull request Nov 17, 2022
* feat: add tensor type for ndarray

* fix: fix mypy typing

* feat: torch tensor type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: protobuf for pytorch type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* ci: install all extras in the ci

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* refactor: make nice looking

* docs: update docarray/typing/tensor/torch_tensor.py

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>

* refactor: code style

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: black and mypy

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: suppress mypy import error

* ci: fix ci install

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
JohannesMessner added a commit that referenced this pull request Nov 23, 2022
* feat: allow da bulk access to return da for document

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy type pb

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add link to the mypy issue

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: remove useless list type hint

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: torch tensor type (#800)

* feat: add tensor type for ndarray

* fix: fix mypy typing

* feat: torch tensor type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: protobuf for pytorch type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* ci: install all extras in the ci

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* refactor: make nice looking

* docs: update docarray/typing/tensor/torch_tensor.py

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>

* refactor: code style

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: black and mypy

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: suppress mypy import error

* ci: fix ci install

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>

* feat: add fastapi to dependency

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat(wip): add fake method to dump tensor to json

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat(wip): add fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add json dump for type and document

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add json compatible with orjson

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* refactor: clean tests

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: remove duplicate

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: better json schema for tensor

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix fast api test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* refactor: move to json test to integration

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: json laod from tensor type now working

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add tensor to fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add new fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add more testing for text uri about json

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix default orson not returning

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: apply johannes suggestion on docstring

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: does not perform copy anymore on torch tensor unwrap

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add johannes suggestion

Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>

* fix: fix ruff line lenght

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy pb

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>
Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
dongxiang123 pushed a commit that referenced this pull request Nov 23, 2022
* feat: allow da bulk access to return da for document

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy type pb

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add link to the mypy issue

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: remove useless list type hint

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: torch tensor type (#800)

* feat: add tensor type for ndarray

* fix: fix mypy typing

* feat: torch tensor type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: protobuf for pytorch type

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* ci: install all extras in the ci

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* refactor: make nice looking

* docs: update docarray/typing/tensor/torch_tensor.py

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>

* refactor: code style

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: black and mypy

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

* fix: suppress mypy import error

* ci: fix ci install

Signed-off-by: Johannes Messner <messnerjo@gmail.com>

Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>

* feat: add fastapi to dependency

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat(wip): add fake method to dump tensor to json

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat(wip): add fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add json dump for type and document

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add json compatible with orjson

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* refactor: clean tests

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: remove duplicate

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: better json schema for tensor

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix fast api test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* refactor: move to json test to integration

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: json laod from tensor type now working

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add tensor to fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add new fastapi test

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add more testing for text uri about json

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix default orson not returning

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: apply johannes suggestion on docstring

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: does not perform copy anymore on torch tensor unwrap

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: add johannes suggestion

Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>

* fix: fix ruff line lenght

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* fix: fix mypy pb

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>
Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Type: PyTorch tensor

2 participants