Skip to content

S3 sync rework tests#5789

Merged
abhizer merged 4 commits intomainfrom
s3-sync-rework-tests
Mar 12, 2026
Merged

S3 sync rework tests#5789
abhizer merged 4 commits intomainfrom
s3-sync-rework-tests

Conversation

@abhizer
Copy link
Contributor

@abhizer abhizer commented Mar 10, 2026

Describe Manual Test Plan

Tested locally using the sync tests and minio.

Checklist

  • Unit tests added/updated
  • Integration tests added/updated
  • Documentation updated
  • Changelog updated

@abhizer abhizer requested a review from mythical-fred March 10, 2026 13:07
Copy link

@mythical-fred mythical-fred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concrete bug (serde(default) missing) and a pre-existing variable swap worth fixing while you're in this code.

@abhizer abhizer force-pushed the s3-sync-rework-tests branch 2 times, most recently from bc672cb to 38225c3 Compare March 10, 2026 17:00
Copy link

@mythical-fred mythical-fred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both fixes look good — #[serde(default)] is on read_bucket and the retention_min_count/retention_min_age swap is corrected. LGTM.

@abhizer abhizer requested a review from blp March 11, 2026 14:40
Copy link
Member

@blp blp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

abhizer added 4 commits March 12, 2026 09:37
Adds the `read_bucket` config to the sync config that is used as a one
time, read only source for the pipeline to start from. This lets one
pipeline start from the state of another pipeline, which saves the time
required to backfill the new pipeline.

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
Configures python docs to automatically use the python package version.

+ clippy fixes

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
@abhizer abhizer force-pushed the s3-sync-rework-tests branch from 38225c3 to 3f4ee71 Compare March 12, 2026 09:45
@abhizer abhizer enabled auto-merge March 12, 2026 09:45
@abhizer abhizer added this pull request to the merge queue Mar 12, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 12, 2026
@abhizer abhizer added this pull request to the merge queue Mar 12, 2026
Merged via the queue into main with commit 9218265 Mar 12, 2026
1 check passed
@abhizer abhizer deleted the s3-sync-rework-tests branch March 12, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants