-
Notifications
You must be signed in to change notification settings - Fork 6
feat: add API proxy sidecar for secure LLM credential management #751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
230913c
40c1c65
2854f11
31a7c51
b63ac77
d1d03d3
e7c71fe
5b4c2d9
9efa4d1
a41b390
1e36afa
5071a89
ba91aaf
81a0b24
b6661e5
21a3776
b9d9559
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Envoy proxy for API key management and proxying | ||
| # This sidecar container holds API keys and proxies requests to LLM providers | ||
| # Supports OpenAI (Codex) and Anthropic (Claude) APIs | ||
|
|
||
| FROM envoyproxy/envoy:v1.31-latest | ||
|
|
||
| # Install curl for healthchecks | ||
| USER root | ||
| RUN apt-get update && \ | ||
| apt-get install -y curl && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Copy entrypoint script that generates envoy.yaml from environment variables | ||
| COPY entrypoint.sh /entrypoint.sh | ||
| RUN chmod +x /entrypoint.sh | ||
|
|
||
| # Create directory for generated config and ensure envoy can write to it | ||
| RUN mkdir -p /etc/envoy && \ | ||
| chown -R envoy:envoy /etc/envoy | ||
|
|
||
| # Switch back to envoy user for running the proxy | ||
| USER envoy | ||
|
|
||
| # Expose ports for proxying | ||
| # 10000 - OpenAI API proxy (Codex) | ||
| # 10001 - Anthropic API proxy (Claude) | ||
| EXPOSE 10000 10001 | ||
|
|
||
| ENTRYPOINT ["/entrypoint.sh"] | ||
| CMD ["-c", "/etc/envoy/envoy.yaml"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # Envoy API Proxy Sidecar | ||
|
|
||
| This container provides secure API key management for LLM providers (OpenAI Codex and Anthropic Claude). | ||
|
|
||
| ## Purpose | ||
|
|
||
| The Envoy proxy acts as a sidecar that: | ||
| - Holds API keys securely (isolated from agent container) | ||
| - Automatically injects authentication headers | ||
| - Proxies requests to LLM API endpoints | ||
|
|
||
| ## Architecture | ||
|
|
||
| - **IP Address**: 172.30.0.30 on awf-net | ||
| - **Ports**: | ||
| - 10000: OpenAI API proxy (Codex) | ||
| - 10001: Anthropic API proxy (Claude) | ||
|
|
||
| ## Configuration | ||
|
|
||
| API keys are passed via environment variables: | ||
| - `OPENAI_API_KEY` - Optional OpenAI API key | ||
| - `ANTHROPIC_API_KEY` - Optional Anthropic API key | ||
|
|
||
| The entrypoint script dynamically generates `envoy.yaml` with: | ||
| - Request header injection for authentication | ||
| - TLS termination for upstream connections | ||
| - HTTP/2 support for API endpoints | ||
|
|
||
| ## Security | ||
|
|
||
| - No API keys exposed to agent container | ||
| - Agent only receives proxy URLs (OPENAI_BASE_URL, ANTHROPIC_BASE_URL) | ||
| - All unnecessary Linux capabilities dropped | ||
| - Resource limits: 512MB memory, 100 PIDs | ||
|
|
||
| ## Usage | ||
|
|
||
| Enable via CLI flag: | ||
| ```bash | ||
| export OPENAI_API_KEY="sk-..." | ||
| export ANTHROPIC_API_KEY="sk-ant-..." | ||
| awf --enable-api-proxy --allow-domains api.openai.com,api.anthropic.com -- command | ||
| ``` | ||
|
|
||
| ## Example Workflows | ||
|
|
||
| ### Codex Usage | ||
| ```bash | ||
| export OPENAI_API_KEY="sk-..." | ||
| awf --enable-api-proxy \ | ||
| --allow-domains api.openai.com \ | ||
| -- curl http://api-proxy:10000/v1/completions -H "Content-Type: application/json" | ||
| ``` | ||
|
|
||
| ### Claude Code Usage | ||
| ```bash | ||
| export ANTHROPIC_API_KEY="sk-ant-..." | ||
| awf --enable-api-proxy \ | ||
| --allow-domains api.anthropic.com \ | ||
| -- curl http://api-proxy:10001/v1/messages -H "Content-Type: application/json" | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
| #!/bin/bash | ||
| set -e | ||
|
|
||
| # Generate Envoy configuration from environment variables | ||
| # This allows API keys to be injected at runtime without persisting to disk | ||
|
|
||
| # Start building the configuration | ||
| cat > /etc/envoy/envoy.yaml <<EOF | ||
| static_resources: | ||
| listeners: | ||
| # OpenAI API proxy (Codex) | ||
| - name: openai_listener | ||
| address: | ||
| socket_address: | ||
| address: 0.0.0.0 | ||
| port_value: 10000 | ||
| filter_chains: | ||
| - filters: | ||
| - name: envoy.filters.network.http_connection_manager | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager | ||
| stat_prefix: openai_ingress | ||
| codec_type: AUTO | ||
| route_config: | ||
| name: openai_route | ||
| virtual_hosts: | ||
| - name: openai_service | ||
| domains: ["*"] | ||
| routes: | ||
| - match: | ||
| prefix: "/" | ||
| route: | ||
| cluster: openai_cluster | ||
| timeout: 300s | ||
|
||
| EOF | ||
|
|
||
| # Add Authorization header injection for OpenAI if API key is provided | ||
| if [ -n "$OPENAI_API_KEY" ]; then | ||
| cat >> /etc/envoy/envoy.yaml <<EOF | ||
| request_headers_to_add: | ||
| - header: | ||
| key: "Authorization" | ||
| value: "Bearer ${OPENAI_API_KEY}" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| EOF | ||
| fi | ||
|
|
||
| cat >> /etc/envoy/envoy.yaml <<EOF | ||
| http_filters: | ||
| - name: envoy.filters.http.router | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router | ||
|
|
||
| # Anthropic API proxy (Claude) | ||
| - name: anthropic_listener | ||
| address: | ||
| socket_address: | ||
| address: 0.0.0.0 | ||
| port_value: 10001 | ||
| filter_chains: | ||
| - filters: | ||
| - name: envoy.filters.network.http_connection_manager | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager | ||
| stat_prefix: anthropic_ingress | ||
| codec_type: AUTO | ||
| route_config: | ||
| name: anthropic_route | ||
| virtual_hosts: | ||
| - name: anthropic_service | ||
| domains: ["*"] | ||
| routes: | ||
| - match: | ||
| prefix: "/" | ||
| route: | ||
| cluster: anthropic_cluster | ||
| timeout: 300s | ||
| EOF | ||
|
|
||
| # Add API key injection for Anthropic if provided | ||
| if [ -n "$ANTHROPIC_API_KEY" ]; then | ||
| cat >> /etc/envoy/envoy.yaml <<EOF | ||
| request_headers_to_add: | ||
| - header: | ||
| key: "x-api-key" | ||
| value: "${ANTHROPIC_API_KEY}" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| - header: | ||
| key: "anthropic-version" | ||
| value: "2023-06-01" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| EOF | ||
| fi | ||
|
|
||
| cat >> /etc/envoy/envoy.yaml <<EOF | ||
| http_filters: | ||
| - name: envoy.filters.http.router | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router | ||
|
|
||
| clusters: | ||
| # OpenAI API cluster | ||
| - name: openai_cluster | ||
| type: LOGICAL_DNS | ||
| dns_lookup_family: V4_ONLY | ||
| load_assignment: | ||
| cluster_name: openai_cluster | ||
| endpoints: | ||
| - lb_endpoints: | ||
| - endpoint: | ||
| address: | ||
| socket_address: | ||
| address: api.openai.com | ||
| port_value: 443 | ||
| transport_socket: | ||
| name: envoy.transport_sockets.tls | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext | ||
| sni: api.openai.com | ||
| typed_extension_protocol_options: | ||
| envoy.extensions.upstreams.http.v3.HttpProtocolOptions: | ||
| "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions | ||
| explicit_http_config: | ||
| http2_protocol_options: {} | ||
|
|
||
| # Anthropic API cluster | ||
| - name: anthropic_cluster | ||
| type: LOGICAL_DNS | ||
| dns_lookup_family: V4_ONLY | ||
| load_assignment: | ||
| cluster_name: anthropic_cluster | ||
| endpoints: | ||
| - lb_endpoints: | ||
| - endpoint: | ||
| address: | ||
| socket_address: | ||
| address: api.anthropic.com | ||
| port_value: 443 | ||
| transport_socket: | ||
| name: envoy.transport_sockets.tls | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext | ||
| sni: api.anthropic.com | ||
| typed_extension_protocol_options: | ||
| envoy.extensions.upstreams.http.v3.HttpProtocolOptions: | ||
| "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions | ||
| explicit_http_config: | ||
| http2_protocol_options: {} | ||
|
|
||
|
||
| admin: | ||
| address: | ||
| socket_address: | ||
| address: 127.0.0.1 | ||
| port_value: 9901 | ||
| EOF | ||
|
|
||
| echo "[INFO] Generated Envoy configuration" | ||
| if [ -n "$OPENAI_API_KEY" ]; then | ||
| echo "[INFO] OpenAI API key configured (first 8 chars: ${OPENAI_API_KEY:0:8}...)" | ||
| fi | ||
| if [ -n "$ANTHROPIC_API_KEY" ]; then | ||
| echo "[INFO] Anthropic API key configured (first 8 chars: ${ANTHROPIC_API_KEY:0:8}...)" | ||
| fi | ||
|
|
||
| # Start Envoy with the generated configuration | ||
| exec /usr/local/bin/envoy "$@" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The script comment says the API keys are injected “without persisting to disk”, but the keys are written into
/etc/envoy/envoy.yaml(and also end up in docker-compose.yml via the service environment). Update the comment and/or mount the config path as tmpfs and avoid writing secrets into a file if the intent is truly no-on-disk persistence.This issue also appears on line 157 of the same file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot implement the suggestion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed - updated the entrypoint.sh comments to clarify that keys are written to /etc/envoy/envoy.yaml and are present in docker-compose.yml service environment. The comment now accurately reflects reality instead of claiming no disk persistence. Commit: d918a43