Tags: uber/submitqueue
Tags
Adding Phabricator ChangeProvider (#241) ## Why? In order to get diff information for Phabricator ChangeIDs, we must implement a Phabricator ChangeProvider. ## What? This change creates a Phabricator ChangeProvider, similar to the existing GitHub ChangeProvider. Like the GitHub ChangeProvider, it: - Retrieves diff information in batches (currently set at 10 diffs per batch) - Converts responses into a `[]ChangeInfo` ## Test Plan I created a temporary unit test (removed before committing) that pointed the Phabricator ChangeProvider at Uber's internal Phabricator instance, and confirmed that we could retrieve diff information by passing a bearer token, similar to the way we initialize the GitHub ChangeProvider in the example server. All unit tests pass, with 99% coverage in the new `phabricator` directory.
feat(stovepipe): add Ingest RPC to gateway for trunk commit verificat… …ion (#262) ## Summary ### Why? Stovepipe verifies commits that are already on trunk (post-merge/post-submit build and test), but its gateway only exposed Ping. It needs an intake RPC, analogous to SubmitQueue's Land, where a client hands in a trunk commit and gets back a tracking id while verification runs asynchronously. ### What? Adds the `Ingest` RPC to the Stovepipe gateway proto contract (proto-only; controller and server wiring follow separately): - `IngestRequest{ queue, change }` and `IngestResponse{ spid }`, where `spid` is the "stovepipe id" handle for tracking the request lifecycle (mirroring Land's `sqid`). - `change` reuses the shared `uber.base.change.Change` wire type from the parent PR; for git, callers pass a `git://<remote>/<repo>/<ref>/<commit_sha>` commit URI. Unlike Land there is no merge strategy, since the commit is already on trunk. ## Test Plan - ✅ `make proto` - ✅ `./tool/bazel build //...` - ✅ `make test` ## Stack 1. #258 1. #261 1. @ #262 Co-authored-by: Cursor <cursoragent@cursor.com>
refactor(proto): add shared api/base proto contracts (Change, Strateg… …y) (#261) ## Summary ### Why? Each service's gateway proto redefines its own `Change` message (and SubmitQueue its own `Strategy` enum), duplicating wire contracts that are meant to stay identical across domains. This is the proto-level gap mirroring what `platform/base/{change,mergestrategy}` already solved for the Go entities, and Stovepipe's upcoming gateway API would otherwise add a third copy of `Change`. ### What? Introduces an `api/base/` proto tree as the shared-wire-contract analog of `platform/base/`: - Adds `api/base/change` (`uber.base.change.Change`) and `api/base/mergestrategy` (`uber.base.mergestrategy.Strategy`), each a message-only contract that domains import instead of redefining. - Teaches the hermetic codegen rule (`tool/proto/proto_codegen.bzl`) to resolve cross-package proto `import`s (exec-root proto_path + imported sources as action inputs) and to skip the gRPC/YARPC generators for service-less contracts via a new `gen_services` attribute. - Migrates the SubmitQueue gateway proto to import the shared `Change`/`Strategy` and drops its local definitions; updates Go consumers (land controller + unit/integration/e2e tests) to the shared `protopb` packages. - Makefile now creates the `protopb/` output dir and marks copied stubs writable, so net-new proto packages generate cleanly. Known cosmetic gazelle warning: resolving the new cross-proto import prints "multiple rules may be imported" for the `# keep` rules_go go_proto_library alias. It does not change generated BUILD files, the build, or the committed-files path that consumers resolve via the root `# gazelle:resolve go` directives. ## Test Plan - ✅ `make proto` (regenerates stubs, including the message-only base protos) - ✅ `./tool/bazel build //...` (213 targets) - ✅ `make test` (54 unit tests pass) - ✅ `make fmt` ## Issues ## Stack 1. @ #261 1. #259 1. #260 Co-authored-by: Cursor <cursoragent@cursor.com>
ci(rebase-stack): own branch deletion (require auto-delete off) (#263) ## Summary ### Why? The "Rebase Stacked PRs" job was silently no-opping: with the repo's "Automatically delete head branches" setting on, GitHub deleted the merged head branch on merge, which auto-retargeted every child PR's base from the merged head branch to the merged base (e.g. `main`) *before* the job ran. Its child lookup (`gh pr list --base <merged-head>`) then found nothing, so it rebased nothing and still went green. Run #257 (and #256 before it) did exactly this, leaving #258/#259/#260 with doubled diffs that had to be rebased by hand. The repo setting is now off, making this workflow the sole owner of head-branch cleanup. But that exposed a second gap: the job only deleted the merged branch at the end of its own run. When a child rebase conflicts, the run intentionally keeps the merged branch (the child still bases on it) and never fires again — so after the author manually rebases and retargets the child, nothing ever deletes the orphaned merged branch. ### What? - Document the hard requirement that "Automatically delete head branches" stays **off**, with the retarget-race rationale, in the workflow header. - Replace the outcome-gated single-branch delete with an invariant-based sweep, `cleanup_orphaned_merged_branches`, that runs on every invocation: for each recently merged PR whose head branch still exists, delete it **iff** no open PR references it as a base or head. This: - deletes the just-merged branch on a clean run (children retargeted away), - **keeps** it when an immediate child rebase failed (that child still bases on it — deleting it would retarget the child to `main` and recreate the broken diff), - reaps branches stranded by an earlier conflicted run on the next merge, once their children were manually fixed. - Branch existence is snapshotted in one `git ls-remote`; the merged base (e.g. `main`) is never touched. Conflicted runs also emit a `::warning::` for visibility while still exiting non-fatally, and no-stack merges log a clear line. ## Test Plan - ✅ `yaml.safe_load` parses the workflow. - ✅ `bash -n` on the extracted `run:` script. - ✅ Simulated the sweep loop with mock data (dedup, skip-gone, skip-base, consider-delete all behave correctly). - Post-merge: the next Rebase Stack run should log child rebases (or "No open child PRs … nothing to rebase"), then a sweep that deletes the merged branch when nothing depends on it. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
refactor(proto): move protos to top-level api/ tree by domain/service (… …#258) ## Summary Stacked on top of `refactor` (#257). Moves all proto wire contracts into a single top-level `api/` tree organized by `domain/service`, and lays the groundwork for upcoming messagequeue payload contracts. - Relocate each service's `proto/` and `protopb/` dirs to `api/{domain}/{service}/` (history-preserving renames). New importpath: `github.com/uber/submitqueue/api/{domain}/{service}/protopb`. - Update `go_package`, all Bazel importpaths, root `gazelle:resolve` hints, consumer `deps`, and all Go imports. - Generalize the `tool/proto` codegen rule (`srcs` list) and the `make proto` copy loop to support **multiple `.proto` files per service package**, so messagequeue contracts can later live alongside each service's RPC contract under the same `api/{domain}/{service}/proto/` dir. - Update docs (`CLAUDE.md` project layout / import paths / proto-gen workflow, `example/README.md`). The proto `package` declarations are intentionally left unchanged (only `go_package` moved) to avoid churning wire-level type names. ## Test plan - [x] `make build` - [x] `make test` (54/54 pass) - [x] `make check-gazelle` - [x] `make lint` - [x] `make check-tidy` - [x] `make proto` produces zero drift vs committed stubs Made with [Cursor](https://cursor.com) ## Stack 1. @ #258 1. #261 1. #262 Co-authored-by: Cursor <cursoragent@cursor.com>
refactor: consolidate shared code under platform/ (#257) ## Summary Move the cross-domain shared trees out of the repo root into a single `platform/` umbrella. Domain-scoped trees (`submitqueue/`, `stovepipe/`, `runway/`) keep their own `core/`, `entity/`, `extension/`. | From | To | |------|----| | `core/{errs,metrics,consumer}` | `platform/{errs,metrics,consumer}` | | `core/httpclient` | `platform/http` | | `entity/*` (`change`, `mergestrategy`, `messagequeue`) | `platform/base/*` | | `extension/*` (`counter`, `messagequeue`) | `platform/extension/*` | Details: - `platform/http`: package renamed `httpclient` -> `http`; `net/http` aliased as `nethttp` inside the package; callers that also import `net/http` use a `phttp` alias. - `platform/base`: root doc package renamed `entity` -> `base`; `change` / `mergestrategy` / `messagequeue` preserved as subpackages. - Rewrites all Go import paths and Bazel labels; updates `Makefile` schema/admin paths and `go:generate` roots. - Docs refreshed to the `platform/` layout (`CLAUDE.md`, package READMEs, RFCs), documenting current state only. > Stacked on #256 (hermetic Go SDK). Review/merge that first. ## Test plan - [x] `make gazelle` clean (BUILD files in sync) - [x] `make build` — 201 targets build (hermetic) - [x] `make test` — 54/54 unit tests pass Made with [Cursor](https://cursor.com) ## Test Plan ## Issues ## Stack 1. @ #257 1. #258 1. #259 1. #260 --------- Co-authored-by: Cursor <cursoragent@cursor.com>
build(bazel): make Go SDK fully hermetic (#256) ## Summary - Drop `go_sdk.host()` from `MODULE.bazel` so Bazel toolchain resolution no longer instantiates the host Go SDK, which fails with "Could not detect host go version" when `go` is not on `PATH`. - Keep `go_sdk.from_file(go_mod = "//:go.mod")` (downloads the pinned `1.24.5` SDK for host + common RBE platforms) plus an explicit `linux/amd64` SDK for the `build-*-linux` Docker cross-compile targets. - Remove the redundant version-pinned host `download` already covered by `from_file`. With this, `make build` / `make test` / `make gazelle` work in a clean shell with no local Go install. ## Test plan - [x] `make build` succeeds with `go` removed from `PATH` (hermetic) - [x] `make gazelle` succeeds with `go` removed from `PATH` - [x] `make test` — all unit tests pass
feat(runway): add merge wire contract and topic keys (#244) ## Summary ### Why? Both the merge-conflict check and the merge itself are moving out of SubmitQueue into an asynchronous round-trip with a separate service, runway. A merge-conflict check is a dry run of a merge, so **one contract serves both**. Runway owns the cross-service queues, their topic keys, and the wire contract — SubmitQueue cannot read runway's storage and vice versa, so the payloads carry full data, not entity IDs. This change adds only those runway-owned definitions; runway's gateway/orchestrator/controllers are out of scope. ### What? Adds a new `runway/` domain folder holding contract-only definitions: `runway/entity` — `MergeRequest` (client-owned `ID`, `QueueName`, ordered `[]MergeStep`), `MergeStep` (`StepID`, `[]change.Change`, `mergestrategy.MergeStrategy`), `StepResult` (`StepID`, VCS-neutral `OutputIDs`, `Reason`), and `MergeResult` (`ID`, `Success`, `Reason`, `[]StepResult`), with JSON `ToBytes`/`FromBytes`. The ordered step list encodes base-layering so one request expresses both "candidate vs target branch" (one step) and "candidate + in-flight vs target" (N steps). It imports only the shared `entity/change` and `entity/mergestrategy` — never `submitqueue/entity`. `runway/core/topickey` — two queue pairs over the one contract: the dry-run check (`merge-conflict-checker` / `merge-conflict-checker-signal`) and the committing merge (`merger` / `merger-signal`). The queue a request arrives on encodes whether runway commits the result and reports the revisions it produced (`StepResult.OutputIDs` — a git commit SHA, hg changeset, svn revision, …; empty for a dry-run check). ## Test Plan ✅ `bazel test //runway/...` (round-trip serialization tests pass) ✅ `bazel build //...` ## Issues ## Stack 1. @ #244 1. #245 1. #247 Co-authored-by: Cursor <cursoragent@cursor.com>
refactor(entity): promote RequestLandStrategy to shared entity/merges… …trategy (#243) ## Summary ### Why? The land-merge strategy enum lived in `submitqueue/entity` as `RequestLandStrategy`, but the upcoming runway merge-conflict contract needs the same type, and runway must not import `submitqueue/entity`. Mirroring the `Change` promotion in #240, the strategy belongs in the shared top-level `entity/` so both domains reference one type with no per-domain enum or mapping. ### What? Moves `RequestLandStrategy` and its constants out of `submitqueue/entity/request.go` into a new top-level `entity/mergestrategy` package as `MergeStrategy` (`MergeStrategyUnknown/Rebase/SquashRebase/Merge`). `Request.LandStrategy` now has type `mergestrategy.MergeStrategy`. All references across the gateway and orchestrator controllers, entities, and tests are updated to the shared type. Pure mechanical refactor — no behavior change. ## Test Plan ✅ `bazel build //...` ✅ `bazel test //... --test_tag_filters=-integration,-e2e` (51 tests pass) ✅ `make gazelle` clean ## Issues ## Stack 1. @ #243 1. #244 1. #245 1. #247
refactor(entity): promote RequestLandStrategy to shared entity/merges… …trategy (#243) ## Summary ### Why? The land-merge strategy enum lived in `submitqueue/entity` as `RequestLandStrategy`, but the upcoming runway merge-conflict contract needs the same type, and runway must not import `submitqueue/entity`. Mirroring the `Change` promotion in #240, the strategy belongs in the shared top-level `entity/` so both domains reference one type with no per-domain enum or mapping. ### What? Moves `RequestLandStrategy` and its constants out of `submitqueue/entity/request.go` into a new top-level `entity/mergestrategy` package as `MergeStrategy` (`MergeStrategyUnknown/Rebase/SquashRebase/Merge`). `Request.LandStrategy` now has type `mergestrategy.MergeStrategy`. All references across the gateway and orchestrator controllers, entities, and tests are updated to the shared type. Pure mechanical refactor — no behavior change. ## Test Plan ✅ `bazel build //...` ✅ `bazel test //... --test_tag_filters=-integration,-e2e` (51 tests pass) ✅ `make gazelle` clean ## Issues ## Stack 1. @ #243 1. #244 1. #245 1. #247
PreviousNext