feat(api/runway): add external merge queue contract#260
Open
behinddwalls wants to merge 1 commit into
Open
Conversation
This was referenced Jun 17, 2026
behinddwalls
added a commit
that referenced
this pull request
Jun 17, 2026
## Summary
Move the cross-domain shared trees out of the repo root into a single
`platform/` umbrella. Domain-scoped trees (`submitqueue/`, `stovepipe/`,
`runway/`) keep their own `core/`, `entity/`, `extension/`.
| From | To |
|------|----|
| `core/{errs,metrics,consumer}` | `platform/{errs,metrics,consumer}` |
| `core/httpclient` | `platform/http` |
| `entity/*` (`change`, `mergestrategy`, `messagequeue`) |
`platform/base/*` |
| `extension/*` (`counter`, `messagequeue`) | `platform/extension/*` |
Details:
- `platform/http`: package renamed `httpclient` -> `http`; `net/http`
aliased as `nethttp` inside the package; callers that also import
`net/http` use a `phttp` alias.
- `platform/base`: root doc package renamed `entity` -> `base`; `change`
/ `mergestrategy` / `messagequeue` preserved as subpackages.
- Rewrites all Go import paths and Bazel labels; updates `Makefile`
schema/admin paths and `go:generate` roots.
- Docs refreshed to the `platform/` layout (`CLAUDE.md`, package
READMEs, RFCs), documenting current state only.
> Stacked on #256 (hermetic Go SDK). Review/merge that first.
## Test plan
- [x] `make gazelle` clean (BUILD files in sync)
- [x] `make build` — 201 targets build (hermetic)
- [x] `make test` — 54/54 unit tests pass
Made with [Cursor](https://cursor.com)
## Test Plan
## Issues
## Stack
1. @ #257
1. #258
1. #259
1. #260
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
3d59d02 to
ce264a6
Compare
35e8531 to
cee7fb5
Compare
ce264a6 to
6c53dd4
Compare
cee7fb5 to
5c1798d
Compare
behinddwalls
added a commit
that referenced
this pull request
Jun 17, 2026
## Summary ### Why? The "Rebase Stacked PRs" job silently no-ops when the repo's "Automatically delete head branches" setting is on. On merge, GitHub deletes the merged head branch, which auto-retargets every child PR's base from the merged head branch to the merged base (e.g. main) *before* the job runs. The job discovers children via `gh pr list --base <merged-head>`, which then returns nothing — so it rebases nothing, force-pushes nothing, and still reports success. Run #257 (and #256 before it) did exactly this, leaving the downstream stack (#258/#259/#260) with doubled diffs that had to be rebased by hand. The fix is to keep "Automatically delete head branches" off (now done at the repo level) and let this workflow own head-branch cleanup. With the setting off, GitHub leaves child bases untouched, the lookup finds them, and the job rebases the stack and then deletes the merged branch itself — for both stacked and non-stacked PRs. ### What? - Document the hard requirement that "Automatically delete head branches" stays off, with the retarget-race rationale, in the workflow header. - Log a clear line when a merge has no child PRs instead of returning silently. - Clarify the branch-deletion step: it runs on every merged PR (stacked or not) and is skipped only when a child rebase failed; fix the misleading "All stacked PRs rebased successfully" message that printed even on no-op runs. No behavior change on the happy path — the job already deleted the merged branch on success; this makes the intent explicit and the logs legible. ## Test Plan - ✅ `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/rebase-stack.yml'))"` — YAML parses. - Confirmed `delete_branch_on_merge` is `false` via `gh api repos/uber/submitqueue`. - Post-merge: the next Rebase Stack run should log child rebases or "No open child PRs … nothing to rebase", then "Deleting merged branch …" and actually remove it. Co-authored-by: Cursor <cursoragent@cursor.com>
behinddwalls
added a commit
that referenced
this pull request
Jun 17, 2026
## Summary ### Why? The "Rebase Stacked PRs" job was silently no-opping: with the repo's "Automatically delete head branches" setting on, GitHub deleted the merged head branch on merge, which auto-retargeted every child PR's base from the merged head branch to the merged base (e.g. `main`) *before* the job ran. Its child lookup (`gh pr list --base <merged-head>`) then found nothing, so it rebased nothing and still went green. Run #257 (and #256 before it) did exactly this, leaving #258/#259/#260 with doubled diffs that had to be rebased by hand. The repo setting is now off, making this workflow the sole owner of head-branch cleanup. But that exposed a second gap: the job only deleted the merged branch at the end of its own run. When a child rebase conflicts, the run intentionally keeps the merged branch (the child still bases on it) and never fires again — so after the author manually rebases and retargets the child, nothing ever deletes the orphaned merged branch. ### What? - Document the hard requirement that "Automatically delete head branches" stays **off**, with the retarget-race rationale, in the workflow header. - Replace the outcome-gated single-branch delete with an invariant-based sweep, `cleanup_orphaned_merged_branches`, that runs on every invocation: for each recently merged PR whose head branch still exists, delete it **iff** no open PR references it as a base or head. This: - deletes the just-merged branch on a clean run (children retargeted away), - **keeps** it when an immediate child rebase failed (that child still bases on it — deleting it would retarget the child to `main` and recreate the broken diff), - reaps branches stranded by an earlier conflicted run on the next merge, once their children were manually fixed. - Branch existence is snapshotted in one `git ls-remote`; the merged base (e.g. `main`) is never touched. Conflicted runs also emit a `::warning::` for visibility while still exiting non-fatally, and no-stack merges log a clear line. ## Test Plan - ✅ `yaml.safe_load` parses the workflow. - ✅ `bash -n` on the extracted `run:` script. - ✅ Simulated the sweep loop with mock data (dedup, skip-gone, skip-base, consider-delete all behave correctly). - Post-merge: the next Rebase Stack run should log child rebases (or "No open child PRs … nothing to rebase"), then a sweep that deletes the merged branch when nothing depends on it. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
6c53dd4 to
c803f40
Compare
5c1798d to
557ebb7
Compare
c803f40 to
b8986df
Compare
557ebb7 to
58313d2
Compare
behinddwalls
added a commit
that referenced
this pull request
Jun 17, 2026
…y) (#261) ## Summary ### Why? Each service's gateway proto redefines its own `Change` message (and SubmitQueue its own `Strategy` enum), duplicating wire contracts that are meant to stay identical across domains. This is the proto-level gap mirroring what `platform/base/{change,mergestrategy}` already solved for the Go entities, and Stovepipe's upcoming gateway API would otherwise add a third copy of `Change`. ### What? Introduces an `api/base/` proto tree as the shared-wire-contract analog of `platform/base/`: - Adds `api/base/change` (`uber.base.change.Change`) and `api/base/mergestrategy` (`uber.base.mergestrategy.Strategy`), each a message-only contract that domains import instead of redefining. - Teaches the hermetic codegen rule (`tool/proto/proto_codegen.bzl`) to resolve cross-package proto `import`s (exec-root proto_path + imported sources as action inputs) and to skip the gRPC/YARPC generators for service-less contracts via a new `gen_services` attribute. - Migrates the SubmitQueue gateway proto to import the shared `Change`/`Strategy` and drops its local definitions; updates Go consumers (land controller + unit/integration/e2e tests) to the shared `protopb` packages. - Makefile now creates the `protopb/` output dir and marks copied stubs writable, so net-new proto packages generate cleanly. Known cosmetic gazelle warning: resolving the new cross-proto import prints "multiple rules may be imported" for the `# keep` rules_go go_proto_library alias. It does not change generated BUILD files, the build, or the committed-files path that consumers resolve via the root `# gazelle:resolve go` directives. ## Test Plan - ✅ `make proto` (regenerates stubs, including the message-only base protos) - ✅ `./tool/bazel build //...` (213 targets) - ✅ `make test` (54 unit tests pass) - ✅ `make fmt` ## Issues ## Stack 1. @ #261 1. #259 1. #260 Co-authored-by: Cursor <cursoragent@cursor.com>
823ec31 to
738e35e
Compare
839efc5 to
02f6e24
Compare
This was referenced Jun 17, 2026
8696ae5 to
e4512bb
Compare
b4860ba to
b514f27
Compare
e4512bb to
7f52edf
Compare
265fcfe to
802e3a1
Compare
b514f27 to
8ea502e
Compare
802e3a1 to
4316027
Compare
mnoah1
approved these changes
Jun 18, 2026
behinddwalls
added a commit
that referenced
this pull request
Jun 18, 2026
## Summary ### Why? The message queue topic-binding proto option was named `topics`, which reads as a concrete wire topic name. It is not — it carries a stable **logical topic key**. Each implementer maps the key to whatever topic name its broker/queue requires (subject to that backend's naming constraints); on our Go side the keys are `consumer.TopicKey` values, mapped to concrete names through `TopicRegistry`. The name `topics` invited the wrong mental model. ### What? Rename the `google.protobuf.MessageOptions` extension `topics` → `topic_keys` (field number `50001` unchanged, so the wire/extension layout is identical) and reframe its doc comment to say it carries a logical key, not a wire name. Regenerated `messagequeue.pb.go` (`E_Topics` → `E_TopicKeys`). This is the base of a stack; the runway contract and the contract RFC that consume the option are updated in the branches stacked on top. ## Test Plan ✅ `make proto` — descriptor field renamed (`name=topics` → `name=topic_keys`), field number `50001` unchanged ✅ `./tool/bazel build //...` ## Issues ## Stack 1. @ #264 1. #259 1. #260 1. #245 1. #247
5b0b01f to
8ddf4a3
Compare
4316027 to
107f266
Compare
behinddwalls
added a commit
that referenced
this pull request
Jun 18, 2026
## Summary
### Why?
Queue payloads are Go structs serialized with `encoding/json`, so the
wire shape is defined only by Go source. There is no language-neutral
contract a non-Go client can compile against, no explicit
topic-to-payload binding, and no distinction between a domain's private
wiring and a published cross-domain contract.
### What?
Doc-only — the design of record for message queue contracts
(`doc/rfc/messagequeue-contract.md`):
- **Contract language: Protobuf**, serialized as **protobuf JSON**
(`protojson`) so payloads stay self-describing JSON on the wire. The
`.proto` is the authority and the Go binding is generated from it, so it
cannot drift (no hand-authored struct, no drift test to keep them in
sync).
- **Topic binding:** a custom `topics` proto option (defined in
`api/base/messagequeue`) carries the wire topic names on the message
itself, read back by reflection — not on the publish/consume hot path,
which still resolves topics from a `consumer.TopicKey` via the registry.
- **Location by audience:** external contracts (something outside the
owning domain depends on them) live in `api/{domain}/messagequeue/`;
internal ones in `{domain}/core/messagequeue/`, with the split enforced
by Bazel `visibility`.
- Documents the accepted protojson conventions (snake_case field names,
UPPER_SNAKE enums, int64-as-string) and why JSON Schema, binary proto,
and Avro were rejected.
## Test Plan
- ✅ `make lint` (doc-only)
## Issues
## Stack
1. #264
1. @ #259
1. #260
1. #245
1. #247
Add Runway's published, language-neutral merge queue contract. merge.proto defines MergeRequest/MergeResult, reusing the shared Change and Strategy types and the topic_keys option, generated into protopb and serialized as protobuf JSON so the queue keeps storing self-describing JSON. The Go helpers wrap protojson and expose the topic binding via reflection; topic keys are co-located with the contract. A drift test round-trips the payloads and verifies every topic key is bound to exactly one message.
107f266 to
b50d730
Compare
sbalabanov
approved these changes
Jun 18, 2026
| // hash, a Subversion revision number, a Perforce changelist, and so on. | ||
| // Empty for a dry-run check, for a change already present on the target, or | ||
| // for a step that failed to apply. | ||
| repeated string output_ids = 2; |
Contributor
There was a problem hiding this comment.
may be it should be an object (message) so it is easier to extend in the future with additional metadata.
Contributor
There was a problem hiding this comment.
also probably important to contract if it is ordered or not.
| string id = 1; | ||
| // success is true if the whole ordered step sequence applied cleanly: | ||
| // mergeable for a dry-run check, merged for a committing merge. | ||
| bool success = 2; |
Contributor
There was a problem hiding this comment.
better use enum for future extensibility, even if it is binary now
| @@ -0,0 +1,107 @@ | |||
| // Copyright (c) 2025 Uber Technologies, Inc. | |||
Contributor
There was a problem hiding this comment.
can this be autogenerated?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
Runway's merge queues are the first cross-domain message queue contract: a client (potentially non-Go) publishes a merge request and consumes the result without access to Runway's Go types or storage. Per the RFC (#259) this needs a language-neutral, proto-defined contract.
What?
Establishes the message queue contract pattern using Runway's merge queues as the reference:
api/runway/messagequeue/proto/merge.protodefiningMergeRequest/MergeResult, reusing the sharedChangeandStrategytypes and thetopicsoption fromapi/base; generated intoprotopb.protojsonhelpers (MergeRequestToBytes/MergeRequestFromBytesand theMergeResultcounterparts); aTopics()reflection helper exposes the topic binding. Topic keys are co-located with the contract.runway/entityintoapi/runway/messagequeue.topicsoption.Test Plan
./tool/bazel test //api/runway/messagequeue:messagequeue_test./tool/bazel build //...Issues
Stack