> ## Documentation Index > Fetch the complete documentation index at: https://docs.cognisafe.uk/llms.txt > Use this file to discover all available pages before exploring further. # Azure OpenAI Insertion > Route Azure OpenAI traffic through Cognisafe for runtime security scoring Azure OpenAI uses a different endpoint shape than the standard OpenAI API: requests go to a per-deployment URL (`/openai/deployments/{deployment-id}/chat/completions?api-version=...`) rather than the canonical `/v1/chat/completions`. The Cognisafe proxy handles this transparently — it forwards the request to the upstream URL you configure, preserving the path and query string. The only change required in your application code is setting `base_url` to the Cognisafe proxy. ## The insertion pattern Replace the Azure OpenAI endpoint with your Cognisafe proxy URL in your SDK or HTTP client configuration. The proxy rewrites nothing — it forwards the full path, including the deployment name and `api-version` parameter, to the real Azure OpenAI endpoint configured as `UPSTREAM_URL`. ``` Your app │ base_url = https://proxy.cognisafe.yourcompany.com ▼ Cognisafe Proxy (Go, :8080) │ UPSTREAM_URL = https://.openai.azure.com │ Forwards: /openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview │ POSTs /internal/log to Cognisafe API (non-blocking) ▼ Azure OpenAI Service └── Private endpoint (optional — see private-endpoint blueprint) ``` ## Environment variable mapping | Azure OpenAI variable | Cognisafe proxy variable | Notes | | -------------------------- | ------------------------ | ------------------------------------------------------------ | | `AZURE_OPENAI_ENDPOINT` | `UPSTREAM_URL` | Set on the proxy deployment, not in your app | | `AZURE_OPENAI_API_KEY` | — | Do not send to Cognisafe; proxy handles upstream auth | | `AZURE_OPENAI_API_VERSION` | — | Passed as query param by the SDK; proxy forwards it | | `OPENAI_API_KEY` | `PROXY_API_KEY` | Your app authenticates to Cognisafe with this | | `AZURE_OPENAI_AD_TOKEN` | — | If using AAD auth, proxy forwards the `Authorization` header | The Cognisafe proxy does not strip or inspect the `api-version` query parameter. Whatever your SDK sends is forwarded verbatim to Azure OpenAI. You are responsible for using an `api-version` that your Azure OpenAI resource supports. ## Python: Azure OpenAI SDK The `openai` Python package supports Azure OpenAI via `AzureOpenAI`. Route it through Cognisafe by replacing `azure_endpoint` with the proxy URL and setting `api_key` to your `PROXY_API_KEY`. ```python theme={null} import os from openai import AzureOpenAI client = AzureOpenAI( # Point at the Cognisafe proxy, not the Azure OpenAI endpoint directly. # The proxy is configured with UPSTREAM_URL=https://.openai.azure.com azure_endpoint="https://proxy.cognisafe.yourcompany.com", # Cognisafe proxy API key — not your Azure OpenAI key. api_key=os.environ["PROXY_API_KEY"], # api-version is forwarded unchanged to Azure OpenAI by the proxy. api_version="2024-08-01-preview", ) response = client.chat.completions.create( # Azure OpenAI uses deployment names, not model names. # The proxy forwards this in the path: /openai/deployments/gpt-4o-prod/chat/completions model="gpt-4o-prod", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarise the OWASP LLM Top 10."}, ], max_tokens=500, ) print(response.choices[0].message.content) ``` ## Python: Cognisafe SDK (recommended) The Cognisafe SDK's `patch_openai()` function handles proxy routing automatically. For Azure OpenAI, set the standard Azure environment variables and configure the SDK before patching: ```python theme={null} import os import cognisafe from openai import AzureOpenAI cognisafe.configure( api_key=os.environ["PROXY_API_KEY"], project_id="my-azure-project", proxy_url="https://proxy.cognisafe.yourcompany.com", api_url="https://api.cognisafe.yourcompany.com", ) # patch_openai() rewrites the base_url on the default openai client. # For AzureOpenAI, patch the instance directly: client = AzureOpenAI( azure_endpoint="https://proxy.cognisafe.yourcompany.com", api_key=os.environ["PROXY_API_KEY"], api_version=os.environ.get("AZURE_OPENAI_API_VERSION", "2024-08-01-preview"), ) cognisafe.patch_openai(client=client) # instruments the client for tracing ``` ## Azure OpenAI request path handling Azure OpenAI endpoints follow this pattern: ``` https://.openai.azure.com/openai/deployments//chat/completions ?api-version=2024-08-01-preview ``` The Cognisafe proxy preserves the path and query string. The effective forwarded URL is: ``` {UPSTREAM_URL}/openai/deployments/{deployment-id}/chat/completions?api-version=... ``` Set `UPSTREAM_URL` on the proxy deployment to your Azure OpenAI resource base URL — **without** a trailing slash and **without** the `/openai/deployments/...` path: ```bash theme={null} # Correct UPSTREAM_URL=https://my-aoai-resource.openai.azure.com # Incorrect — do not include the deployment path UPSTREAM_URL=https://my-aoai-resource.openai.azure.com/openai/deployments/gpt-4o-prod ``` ## AAD token authentication (no static key) Azure OpenAI supports authentication via Azure AD bearer tokens in addition to API keys. If your deployment policy prohibits static API keys on the Azure OpenAI resource, configure the proxy to use workload identity. ### Proxy workload identity setup (AKS) Enable workload identity on the proxy pod (see the [AKS blueprint](/blueprints/aks) for the full setup). The proxy needs a managed identity with the **Cognitive Services User** role on the Azure OpenAI resource: ```bash theme={null} az role assignment create \ --assignee \ --role "Cognitive Services User" \ --scope /subscriptions//resourceGroups//providers/Microsoft.CognitiveServices/accounts/ ``` The proxy must then acquire an AAD token and inject it as the `Authorization` header on outbound requests to Azure OpenAI. The current Go proxy forwards the `Authorization` header from the incoming request. To use workload identity, implement token acquisition in the proxy using the Azure Identity SDK: ```go theme={null} // proxy/main.go — outbound request enrichment import "github.com/Azure/azure-sdk-for-go/sdk/azidentity" cred, err := azidentity.NewWorkloadIdentityCredential(nil) // acquire token for https://cognitiveservices.azure.com/.default token, err := cred.GetToken(ctx, policy.TokenRequestOptions{ Scopes: []string{"https://cognitiveservices.azure.com/.default"}, }) req.Header.Set("Authorization", "Bearer " + token.Token) req.Header.Del("api-key") // remove static key header if present ``` Workload identity eliminates the `AZURE_OPENAI_API_KEY` secret from your cluster entirely. Clients still authenticate to the Cognisafe proxy using `PROXY_API_KEY`; the proxy holds the credential for Azure OpenAI internally. This is the recommended pattern for enterprise deployments. ## Private endpoint option For zero-public-egress deployments, place both the Cognisafe proxy and the Azure OpenAI private endpoint in the same VNet. The proxy resolves `.openai.azure.com` via private DNS to the private endpoint IP — no traffic leaves the Microsoft backbone. ``` VNet: 10.0.0.0/16 ├── proxy-subnet: 10.0.1.0/24 │ └── Cognisafe proxy pods (AKS node pool) │ └── data-subnet: 10.0.3.0/24 └── Azure OpenAI private endpoint (10.0.3.10) Private DNS: privatelink.openai.azure.com → 10.0.3.10 ``` Full Terraform and NSG configuration is in the [Private Endpoint Architecture blueprint](/blueprints/private-endpoint). ## Testing ```bash theme={null} # Direct test via curl (bypasses SDK) curl -s -X POST \ "https://proxy.cognisafe.yourcompany.com/openai/deployments/gpt-4o-prod/chat/completions?api-version=2024-08-01-preview" \ -H "Authorization: Bearer $PROXY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "messages": [{"role": "user", "content": "Say hello in JSON."}], "max_tokens": 30 }' | jq . # Verify the request was logged in Cognisafe curl -s "https://api.cognisafe.yourcompany.com/requests?limit=1" \ -H "Authorization: Bearer $INTERNAL_API_SECRET" | jq '.data[0] | {id, model, created_at, project_id}' ``` ## Troubleshooting | Error | Root cause | Resolution | | --------------------------------------------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `404 Resource Not Found` from Azure OpenAI | Deployment name does not exist in the target Azure OpenAI resource | Verify `az cognitiveservices account deployment list --name --resource-group ` | | `401 Unauthorized` from Azure OpenAI | `UPSTREAM_URL` is set correctly but the proxy's auth header is wrong | If using API key auth, ensure `api-key` header is set by the proxy; if using AAD, check workload identity token scopes | | `400 Bad Request — api-version` | Azure OpenAI resource does not support the requested `api-version` | Update `api_version` in your client to one listed in the Azure OpenAI API changelog | | `403 Forbidden` | Managed identity lacks the `Cognitive Services User` role | Run `az role assignment list --assignee ` to verify | | `401` from Cognisafe proxy | Invalid `PROXY_API_KEY` | Check the `Authorization: Bearer ...` header sent to the proxy; verify the key in Key Vault matches the proxy's `PROXY_API_KEY` env var | | Proxy returns response but Cognisafe dashboard shows no request | `API_BACKEND_URL` misconfigured on proxy | The proxy logs to `/internal/log` asynchronously — check proxy container logs for connection errors to the API service | | Streaming responses cut off | Proxy buffer too small | Cognisafe proxy streams responses chunk-by-chunk; check for body-size limits in any intermediate nginx ingress (`proxy-body-size` annotation) |