fix: Fix issue with stream upload batch size upload limit #2290
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GarrettWu
approved these changes
Nov 26, 2025
| offsets_col: str, | ||
| ) -> bq_data.BigqueryDataSource: | ||
| """Load managed data into bigquery""" | ||
| MAX_BYTES = 10000000 # streaming api has 10MB limit |
Contributor
There was a problem hiding this comment.
Seems the consts can be moved to the top of the file, and if different safety margin there, you can give them different names, SAFTY_MARGIN_STREAM, SAFTY_MARGIN_WRITE, would be easier to read and compare.
Merged
GarrettWu
added a commit
that referenced
this pull request
Dec 3, 2025
PR created by the Librarian CLI to initialize a release. Merging this PR will auto trigger a release. Librarian Version: v0.7.0 Language Image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:c8612d3fffb3f6a32353b2d1abd16b61e87811866f7ec9d65b59b02eb452a620 <details><summary>bigframes: 2.30.0</summary> ## [2.30.0](v2.29.0...v2.30.0) (2025-12-03) ### Features * Support mixed scalar-analytic expressions (#2239) ([20ab469](20ab469d)) * Allow drop_duplicates over unordered dataframe (#2303) ([52665fa](52665fa5)) * Preserve source names better for more readable sql (#2243) ([64995d6](64995d65)) * use end user credentials for `bigframes.bigquery.ai` functions when `connection_id` is not present (#2272) ([7c062a6](7c062a68)) * pivot_table supports fill_value arg (#2257) ([8f490e6](8f490e68)) * Support builtins funcs for df.agg (#2256) ([956a5b0](956a5b00)) * add bigquery.json_keys (#2286) ([b487cf1](b487cf1f)) * Add agg/aggregate methods to windows (#2288) ([c4cb39d](c4cb39dc)) * Add bigframes.pandas.crosstab (#2231) ([c62e553](c62e5535)) * Implement single-column sorting for interactive table widget (#2255) ([d1ecc61](d1ecc61b)) ### Bug Fixes * Pass credentials properly for read api instantiation (#2280) ([3e3fe25](3e3fe259)) * Update max_instances default to reflect actual value (#2302) ([4489687](4489687e)) * Improve Anywidget pagination and display for unknown row counts (#2258) ([508deae](508deae5)) * Fix issue with stream upload batch size upload limit (#2290) ([6cdf64b](6cdf64b0)) * calling info() on empty dataframes no longer leads to errors (#2267) ([95a83f7](95a83f77)) * do not warn with DefaultIndexWarning in partial ordering mode (#2230) ([cc2dbae](cc2dbae6)) ### Documentation * update docs and tests for Gemini 2.5 models (#2279) ([08c0c0c](08c0c0c8)) * Add Google Analytics configuration to conf.py (#2301) ([0b266da](0b266da1)) * fix LogisticRegression docs rendering (#2295) ([32e5313](32e53134)) * update API reference to new `dataframes.bigquery.dev` location (#2293) ([da06439](da064397)) * use autosummary to split documentation pages (#2251) ([f7fd2d2](f7fd2d20)) </details>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕