Skip to content

feat: add core AI MITM proxy daemon #21296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ssncferreira wants to merge 5 commits into
base: main
Choose a base branch
from
+622 −4
Draft

feat: add core AI MITM proxy daemon #21296

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
67307de
feat: add core AI MITM proxy daemon
ssncferreira Dec 16, 2025
c1825d6
chore: address comments
ssncferreira Dec 17, 2025
1651bbe
chore: rename aiproxyd to aibridgeproxyd
ssncferreira Dec 17, 2025
c90790f
chore: add missing aibridgeproxyd renames
ssncferreira Dec 17, 2025
db9ee42
chore: address comments
ssncferreira Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions cli/testdata/coder_server_--help.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,20 @@ AI BRIDGE OPTIONS:
Maximum number of AI Bridge requests per second per replica. Set to 0
to disable (unlimited).

AI BRIDGE PROXY OPTIONS:
--aibridge-proxy-cert-file string, $CODER_AIBRIDGE_PROXY_CERT_FILE
Path to the CA certificate file for AI Bridge Proxy.

--aibridge-proxy-enabled bool, $CODER_AIBRIDGE_PROXY_ENABLED (default: false)
Enable the AI Bridge MITM Proxy for intercepting and decrypting AI
provider requests.

--aibridge-proxy-key-file string, $CODER_AIBRIDGE_PROXY_KEY_FILE
Path to the CA private key file for AI Bridge Proxy.

--aibridge-proxy-listen-addr string, $CODER_AIBRIDGE_PROXY_LISTEN_ADDR (default: :8888)
The address the AI Bridge Proxy will listen on.

CLIENT OPTIONS:
These options change the behavior of how clients interact with the Coder.
Clients include the Coder CLI, Coder Desktop, IDE extensions, and the web UI.
Expand Down
14 changes: 14 additions & 0 deletions cli/testdata/server-config.yaml.golden
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -765,6 +765,20 @@ aibridge:
# (unlimited).
# (default: 0, type: int)
rateLimit: 0
aibridgeproxy:
# Enable the AI Bridge MITM Proxy for intercepting and decrypting AI provider
# requests.
# (default: false, type: bool)
enabled: false
# The address the AI Bridge Proxy will listen on.
# (default: :8888, type: string)
listen_addr: :8888
# Path to the CA certificate file for AI Bridge Proxy.
# (default: <unset>, type: string)
cert_file: ""
# Path to the CA private key file for AI Bridge Proxy.
# (default: <unset>, type: string)
key_file: ""
# Configure data retention policies for various database tables. Retention
# policies automatically purge old data to reduce database size and improve
# performance. Setting a retention duration to 0 disables automatic purging for
Expand Down
20 changes: 20 additions & 0 deletions coderd/apidoc/docs.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions coderd/apidoc/swagger.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

57 changes: 56 additions & 1 deletion codersdk/deployment.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1217,6 +1217,10 @@ func (c *DeploymentValues) Options() serpent.OptionSet {
Name: "AI Bridge",
YAML: "aibridge",
}
deploymentGroupAIBridgeProxy = serpent.Group{
Name: "AI Bridge Proxy",
YAML: "aibridgeproxy",
}
deploymentGroupRetention = serpent.Group{
Name: "Retention",
Description: "Configure data retention policies for various database tables. Retention policies automatically purge old data to reduce database size and improve performance. Setting a retention duration to 0 disables automatic purging for that data type.",
Expand Down Expand Up @@ -3443,6 +3447,49 @@ Write out the current server config as YAML to stdout.`,
Group: &deploymentGroupAIBridge,
YAML: "rateLimit",
},

// AI Bridge Proxy Options
{
Name: "AI Bridge Proxy Enabled",
Description: "Enable the AI Bridge MITM Proxy for intercepting and decrypting AI provider requests.",
Flag: "aibridge-proxy-enabled",
Env: "CODER_AIBRIDGE_PROXY_ENABLED",
Value: &c.AI.BridgeProxyConfig.Enabled,
Default: "false",
Group: &deploymentGroupAIBridgeProxy,
YAML: "enabled",
},
{
Name: "AI Bridge Proxy Listen Address",
Description: "The address the AI Bridge Proxy will listen on.",
Flag: "aibridge-proxy-listen-addr",
Env: "CODER_AIBRIDGE_PROXY_LISTEN_ADDR",
Value: &c.AI.BridgeProxyConfig.ListenAddr,
Default: ":8888",
Copy link
Contributor Author

@ssncferreira ssncferreira Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if there is a better standard for a proxy port 🤔

Copy link
Contributor

@dannykopping dannykopping Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8080is kinda common because HTTP is 80, but I don't think it matters.

I think we listen on 8080 already for our web frontend in dev mode; might make things complicated.

Copy link
Contributor Author

@ssncferreira ssncferreira Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we listen on 8080 already for our web frontend in dev mode; might make things complicated.

Yes, exactly, that is why I kept this as 8888.

Group: &deploymentGroupAIBridgeProxy,
YAML: "listen_addr",
Copy link
Contributor Author

@ssncferreira ssncferreira Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we have a convention for yaml between snake_case and camelCase, as I see some options of aibridge with both, for instance, rateLimit and inject_coder_mcp_tools

Copy link
Contributor

@dannykopping dannykopping Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, it should be underscore.

Would you mind raising an issue to address these inconsistencies with rateLimit?

Copy link
Contributor Author

@ssncferreira ssncferreira Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an issue here: coder/internal#1205

},
{
Name: "AI Bridge Proxy Certificate File",
Description: "Path to the CA certificate file for AI Bridge Proxy.",
Flag: "aibridge-proxy-cert-file",
Env: "CODER_AIBRIDGE_PROXY_CERT_FILE",
Value: &c.AI.BridgeProxyConfig.CertFile,
Default: "",
Group: &deploymentGroupAIBridgeProxy,
YAML: "cert_file",
},
{
Name: "AI Bridge Proxy Key File",
Description: "Path to the CA private key file for AI Bridge Proxy.",
Flag: "aibridge-proxy-key-file",
Env: "CODER_AIBRIDGE_PROXY_KEY_FILE",
Value: &c.AI.BridgeProxyConfig.KeyFile,
Default: "",
Group: &deploymentGroupAIBridgeProxy,
YAML: "key_file",
Copy link
Contributor Author

@ssncferreira ssncferreira Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, do we need to set this as a secret? Something like: Annotations: serpent.Annotations{}.Mark(annotationSecretKey, "true"),
This is the file path, so the content is the actual secret 🤔

},

// Retention settings
{
Name: "Audit Logs Retention",
Expand Down Expand Up @@ -3535,8 +3582,16 @@ type AIBridgeBedrockConfig struct {
SmallFastModel serpent.String `json:"small_fast_model" typescript:",notnull"`
}

type AIBridgeProxyConfig struct {
Enabled serpent.Bool `json:"enabled" typescript:",notnull"`
ListenAddr serpent.String `json:"listen_addr" typescript:",notnull"`
CertFile serpent.String `json:"cert_file" typescript:",notnull"`
KeyFile serpent.String `json:"key_file" typescript:",notnull"`
}

type AIConfig struct {
BridgeConfig AIBridgeConfig `json:"bridge,omitempty"`
BridgeConfig AIBridgeConfig `json:"bridge,omitempty"`
BridgeProxyConfig AIBridgeProxyConfig `json:"aibridge_proxy,omitempty"`
}

type SupportConfig struct {
Expand Down
6 changes: 6 additions & 0 deletions docs/reference/api/general.md
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 42 additions & 3 deletions docs/reference/api/schemas.md
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 42 additions & 0 deletions docs/reference/cli/server.md
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading