feat: route AI provider requests to aibridged #21343
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
4d87f7d to
9b2593b
Compare
|
|
||
| // canonicalHost strips the port from a host:port string and lowercases it. | ||
| func canonicalHost(host string) string { | ||
| if i := strings.IndexByte(host, ':'); i != -1 { |
There was a problem hiding this comment.
Were you aware of net.SplitHostPort but chose this approach for some reason?
| func providerFromHost(host string) string { | ||
| switch canonicalHost(host) { | ||
| case "api.anthropic.com": | ||
| return "anthropic" |
There was a problem hiding this comment.
Let's use the provider constants here pls.
e.g. https://github.com/coder/aibridge/blob/main/provider_anthropic.go#L29
| // Check if this request is for a supported AI provider. | ||
| provider := providerFromHost(req.Host) | ||
| if provider == "" { | ||
| s.logger.Debug(s.ctx, "passthrough request to unknown host", |
There was a problem hiding this comment.
This shouldn't really happen, right? Since we only MITM allowlisted domains?
| originalPath := req.URL.Path | ||
| aibridgePath := "/" + provider + originalPath | ||
|
|
||
| newURL := *s.coderAccessURL | ||
| newURL.Path = "/api/v2/aibridge" + aibridgePath |
There was a problem hiding this comment.
Please prefer https://pkg.go.dev/net/url#JoinPath instead; covers a bunch of edge-cases.
| originalPath := req.URL.Path | ||
| aibridgePath := "/" + provider + originalPath | ||
|
|
||
| newURL := *s.coderAccessURL |
There was a problem hiding this comment.
It's technically possible that this could be nil here, so let's avoid the nil-deref panic pls.
| targetPath: "/v1/messages", | ||
| expectedPath: "/api/v2/aibridge/anthropic/v1/messages", |
There was a problem hiding this comment.
Can you also please include a case for known a provider whose route is neither bridged nor passthru?
| w.WriteHeader(http.StatusOK) | ||
| _, _ = w.Write([]byte("hello from aibridged")) | ||
| })) | ||
| defer aibridgedServer.Close() |
| // - For AI providers, use their real hostnames to trigger routing. | ||
| var targetURL string | ||
| if tt.passthrough { | ||
| targetURL = targetServer.URL + tt.targetPath |
There was a problem hiding this comment.
Ditto about url.JoinPath
Description
Implements request routing for the AI Bridge Proxy. After MITM decryption, requests to known AI providers (Anthropic, OpenAI) are rewritten to the corresponding aibridged endpoint, while requests to unknown hosts are passed through to their original destination.
Changes
CoderAccessURLconfiguration option for specifying the Coder deployment URLhandleRequestto route decrypted requests based on target hostAuthorization: Bearerheader for aibridgedDepends on: #21342
Closes: coder/internal#1181