feat: route AI provider requests to aibridged #21343
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
4d87f7d to
9b2593b
Compare
|
|
||
| // canonicalHost strips the port from a host:port string and lowercases it. | ||
| func canonicalHost(host string) string { | ||
| if i := strings.IndexByte(host, ':'); i != -1 { |
There was a problem hiding this comment.
Were you aware of net.SplitHostPort but chose this approach for some reason?
There was a problem hiding this comment.
I used the same approach as the POC. We can use net.SplitHostPort, but it errors when there's no port in the address. After MITM decryption, req.Host is the HTTP Host header, which omits the port when using default ports (443 for HTTPS, 80 for HTTP), so api.anthropic.com instead of api.anthropic.com:443.
I've updated providerFromURL to receive a url.URL and use url.Hostname(), which handles both cases correctly.
Addressed in: 5e6a35c
| // Check if this request is for a supported AI provider. | ||
| provider := providerFromHost(req.Host) | ||
| if provider == "" { | ||
| s.logger.Debug(s.ctx, "passthrough request to unknown host", |
There was a problem hiding this comment.
This shouldn't really happen, right? Since we only MITM allowlisted domains?
There was a problem hiding this comment.
That's correct, but right now we don't have the allowlist in place yet. Currently, the proxy always MITMs all requests, so unknown hosts reach handleRequest and get passed through after decryption.
Once coder/internal#1182 is implemented, unknown hosts will be tunneled (not decrypted), so they won't reach handleRequest and this case should never happen. At that point, we should probably update this to an error. I added a TODO comment: 82ddf4d
| targetPath: "/v1/messages", | ||
| expectedPath: "/api/v2/aibridge/anthropic/v1/messages", |
There was a problem hiding this comment.
Can you also please include a case for known a provider whose route is neither bridged nor passthru?
There was a problem hiding this comment.
What do you mean? 🤔 A route to a known provider is always bridged from the proxy to aibridge.
Do you mean a route that is not configured in aibridge as either a bridged route or a passthrough route? (Examples: bridged, passthrough). We could test that, but that's aibridge's responsibility to handle, not the proxy's. In these tests we're using a mock aibridge that just returns 200 OK.
bc4caed to
97198d5
Compare
88a304c to
56e1c5c
Compare
b82c306 to
5e6a35c
Compare
4157564 to
ac0129f
Compare
5a1d2dc to
4196397
Compare
ac0129f to
ae82ddd
Compare
644e954 to
4654e31
Compare
176e7e7 to
097fbc9
Compare
097fbc9 to
7145548
Compare
4654e31 to
e0906db
Compare
Description
Implements request routing for the AI Bridge Proxy. After MITM decryption, requests to known AI providers (Anthropic, OpenAI) are rewritten to the corresponding aibridged endpoint, while requests to unknown hosts are passed through to their original destination.
Changes
CoderAccessURLconfiguration option for specifying the Coder deployment URLhandleRequestto route decrypted requests based on target hostAuthorization: Bearerheader for aibridgedDepends on: #21342
Closes: coder/internal#1181