feat(storage/dataflux): add dataflux interface#10748
Merged
tritone merged 46 commits intogoogleapis:mainfrom Sep 19, 2024
akansha1812:main
Merged
feat(storage/dataflux): add dataflux interface#10748tritone merged 46 commits intogoogleapis:mainfrom akansha1812:main
tritone merged 46 commits intogoogleapis:mainfrom
akansha1812:main
Conversation
MattIrv
approved these changes
Aug 22, 2024
tritone
reviewed
Aug 26, 2024
Contributor
tritone
left a comment
There was a problem hiding this comment.
Some initial comments. Overall I'd like to see an MVP that works end-to-end (maybe just sequential list?) before we merge, as well as an example_test.go file and basic integration test. That will help validate that this interface is workable before we release anything. Obviously we can make breaking changes in the subpackage since we have the preview tag, but it's not ideal.
tritone
reviewed
Sep 11, 2024
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 673380763 Source-Link: https://togithub.com/googleapis/googleapis/commit/8ebfd76bd91ba97b86491de9161c9e5e6884a68a Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/c7314d1edf63123ef0efbe9bc4d996391025fb44 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzczMTRkMWVkZjYzMTIzZWYwZWZiZTliYzRkOTk2MzkxMDI1ZmI0NCJ9 BEGIN_NESTED_COMMIT feat(bigtable/admin): Add support for Cloud Bigtable Row Affinity in App Profiles PiperOrigin-RevId: 673093969 Source-Link: https://togithub.com/googleapis/googleapis/commit/cbf696d38a963c5ab333f85fc9a910b5698ad415 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/a2f7ec1191813304b3bd0097caa33956bdb3b637 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYTJmN2VjMTE5MTgxMzMwNGIzYmQwMDk3Y2FhMzM5NTZiZGIzYjYzNyJ9 END_NESTED_COMMIT BEGIN_NESTED_COMMIT feat(aiplatform): add Pinecone and Vector Search integration for Vertex RAG PiperOrigin-RevId: 673087899 Source-Link: https://togithub.com/googleapis/googleapis/commit/afb6b3599d50103e022e9c22c5057bf94be9dcf8 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/00a4515ab465e98d56627075675209631ee51f39 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMDBhNDUxNWFiNDY1ZTk4ZDU2NjI3MDc1Njc1MjA5NjMxZWU1MWYzOSJ9 END_NESTED_COMMIT
* feat(firestore): Adding distance threshold and result field * refactor(firestore): Renaming method names * refactor(firestore): Move threshold and result field to options. Rename FindNearestOptions * refactor(firestore): Rename to FindNearestOptions * refactor(firestore): Refactoring code
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 673380763 Source-Link: https://togithub.com/googleapis/googleapis/commit/8ebfd76bd91ba97b86491de9161c9e5e6884a68a Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/c7314d1edf63123ef0efbe9bc4d996391025fb44 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzczMTRkMWVkZjYzMTIzZWYwZWZiZTliYzRkOTk2MzkxMDI1ZmI0NCJ9 BEGIN_NESTED_COMMIT feat(bigtable/admin): Add support for Cloud Bigtable Row Affinity in App Profiles PiperOrigin-RevId: 673093969 Source-Link: https://togithub.com/googleapis/googleapis/commit/cbf696d38a963c5ab333f85fc9a910b5698ad415 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/a2f7ec1191813304b3bd0097caa33956bdb3b637 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYTJmN2VjMTE5MTgxMzMwNGIzYmQwMDk3Y2FhMzM5NTZiZGIzYjYzNyJ9 END_NESTED_COMMIT BEGIN_NESTED_COMMIT feat(aiplatform): add Pinecone and Vector Search integration for Vertex RAG PiperOrigin-RevId: 673087899 Source-Link: https://togithub.com/googleapis/googleapis/commit/afb6b3599d50103e022e9c22c5057bf94be9dcf8 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/00a4515ab465e98d56627075675209631ee51f39 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMDBhNDUxNWFiNDY1ZTk4ZDU2NjI3MDc1Njc1MjA5NjMxZWU1MWYzOSJ9 END_NESTED_COMMIT
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 673380763 Source-Link: https://togithub.com/googleapis/googleapis/commit/8ebfd76bd91ba97b86491de9161c9e5e6884a68a Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/c7314d1edf63123ef0efbe9bc4d996391025fb44 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzczMTRkMWVkZjYzMTIzZWYwZWZiZTliYzRkOTk2MzkxMDI1ZmI0NCJ9 BEGIN_NESTED_COMMIT feat(bigtable/admin): Add support for Cloud Bigtable Row Affinity in App Profiles PiperOrigin-RevId: 673093969 Source-Link: https://togithub.com/googleapis/googleapis/commit/cbf696d38a963c5ab333f85fc9a910b5698ad415 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/a2f7ec1191813304b3bd0097caa33956bdb3b637 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYTJmN2VjMTE5MTgxMzMwNGIzYmQwMDk3Y2FhMzM5NTZiZGIzYjYzNyJ9 END_NESTED_COMMIT BEGIN_NESTED_COMMIT feat(aiplatform): add Pinecone and Vector Search integration for Vertex RAG PiperOrigin-RevId: 673087899 Source-Link: https://togithub.com/googleapis/googleapis/commit/afb6b3599d50103e022e9c22c5057bf94be9dcf8 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/00a4515ab465e98d56627075675209631ee51f39 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMDBhNDUxNWFiNDY1ZTk4ZDU2NjI3MDc1Njc1MjA5NjMxZWU1MWYzOSJ9 END_NESTED_COMMIT
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 673380763 Source-Link: https://togithub.com/googleapis/googleapis/commit/8ebfd76bd91ba97b86491de9161c9e5e6884a68a Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/c7314d1edf63123ef0efbe9bc4d996391025fb44 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzczMTRkMWVkZjYzMTIzZWYwZWZiZTliYzRkOTk2MzkxMDI1ZmI0NCJ9 BEGIN_NESTED_COMMIT feat(bigtable/admin): Add support for Cloud Bigtable Row Affinity in App Profiles PiperOrigin-RevId: 673093969 Source-Link: https://togithub.com/googleapis/googleapis/commit/cbf696d38a963c5ab333f85fc9a910b5698ad415 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/a2f7ec1191813304b3bd0097caa33956bdb3b637 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYTJmN2VjMTE5MTgxMzMwNGIzYmQwMDk3Y2FhMzM5NTZiZGIzYjYzNyJ9 END_NESTED_COMMIT BEGIN_NESTED_COMMIT feat(aiplatform): add Pinecone and Vector Search integration for Vertex RAG PiperOrigin-RevId: 673087899 Source-Link: https://togithub.com/googleapis/googleapis/commit/afb6b3599d50103e022e9c22c5057bf94be9dcf8 Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/00a4515ab465e98d56627075675209631ee51f39 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMDBhNDUxNWFiNDY1ZTk4ZDU2NjI3MDc1Njc1MjA5NjMxZWU1MWYzOSJ9 END_NESTED_COMMIT
tritone
reviewed
Sep 18, 2024
tritone
approved these changes
Sep 18, 2024
Contributor
tritone
left a comment
There was a problem hiding this comment.
Two more small comments, otherwise LGTM -- I will approve now and let you fix.
tritone
reviewed
Sep 18, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: add dataflux interface and helper functions to storage/dataflux.
Dataflux fast-listing will be used to quickly list objects in a bucket in parallel.
Fixes #10731