Azure OpenAI Insertion

Azure OpenAI uses a different endpoint shape than the standard OpenAI API: requests go to a per-deployment URL (/openai/deployments/{deployment-id}/chat/completions?api-version=...) rather than the canonical /v1/chat/completions. The Cognisafe proxy handles this transparently — it forwards the request to the upstream URL you configure, preserving the path and query string. The only change required in your application code is setting base_url to the Cognisafe proxy.

The insertion pattern

Replace the Azure OpenAI endpoint with your Cognisafe proxy URL in your SDK or HTTP client configuration. The proxy rewrites nothing — it forwards the full path, including the deployment name and api-version parameter, to the real Azure OpenAI endpoint configured as UPSTREAM_URL.

Your app
  │  base_url = https://proxy.cognisafe.yourcompany.com
  ▼
Cognisafe Proxy (Go, :8080)
  │  UPSTREAM_URL = https://<your-resource>.openai.azure.com
  │  Forwards: /openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview
  │  POSTs /internal/log to Cognisafe API (non-blocking)
  ▼
Azure OpenAI Service
  └── Private endpoint (optional — see private-endpoint blueprint)

Environment variable mapping

Azure OpenAI variable	Cognisafe proxy variable	Notes
`AZURE_OPENAI_ENDPOINT`	`UPSTREAM_URL`	Set on the proxy deployment, not in your app
`AZURE_OPENAI_API_KEY`	—	Do not send to Cognisafe; proxy handles upstream auth
`AZURE_OPENAI_API_VERSION`	—	Passed as query param by the SDK; proxy forwards it
`OPENAI_API_KEY`	`PROXY_API_KEY`	Your app authenticates to Cognisafe with this
`AZURE_OPENAI_AD_TOKEN`	—	If using AAD auth, proxy forwards the `Authorization` header

The Cognisafe proxy does not strip or inspect the api-version query parameter. Whatever your SDK sends is forwarded verbatim to Azure OpenAI. You are responsible for using an api-version that your Azure OpenAI resource supports.

Python: Azure OpenAI SDK

The openai Python package supports Azure OpenAI via AzureOpenAI. Route it through Cognisafe by replacing azure_endpoint with the proxy URL and setting api_key to your PROXY_API_KEY.

import os
from openai import AzureOpenAI

client = AzureOpenAI(
    # Point at the Cognisafe proxy, not the Azure OpenAI endpoint directly.
    # The proxy is configured with UPSTREAM_URL=https://<your-resource>.openai.azure.com
    azure_endpoint="https://proxy.cognisafe.yourcompany.com",

    # Cognisafe proxy API key — not your Azure OpenAI key.
    api_key=os.environ["PROXY_API_KEY"],

    # api-version is forwarded unchanged to Azure OpenAI by the proxy.
    api_version="2024-08-01-preview",
)

response = client.chat.completions.create(
    # Azure OpenAI uses deployment names, not model names.
    # The proxy forwards this in the path: /openai/deployments/gpt-4o-prod/chat/completions
    model="gpt-4o-prod",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarise the OWASP LLM Top 10."},
    ],
    max_tokens=500,
)

print(response.choices[0].message.content)

Python: Cognisafe SDK (recommended)

The Cognisafe SDK’s patch_openai() function handles proxy routing automatically. For Azure OpenAI, set the standard Azure environment variables and configure the SDK before patching:

import os
import cognisafe
from openai import AzureOpenAI

cognisafe.configure(
    api_key=os.environ["PROXY_API_KEY"],
    project_id="my-azure-project",
    proxy_url="https://proxy.cognisafe.yourcompany.com",
    api_url="https://api.cognisafe.yourcompany.com",
)

# patch_openai() rewrites the base_url on the default openai client.
# For AzureOpenAI, patch the instance directly:
client = AzureOpenAI(
    azure_endpoint="https://proxy.cognisafe.yourcompany.com",
    api_key=os.environ["PROXY_API_KEY"],
    api_version=os.environ.get("AZURE_OPENAI_API_VERSION", "2024-08-01-preview"),
)

cognisafe.patch_openai(client=client)  # instruments the client for tracing

Azure OpenAI request path handling

Azure OpenAI endpoints follow this pattern:

https://<resource-name>.openai.azure.com/openai/deployments/<deployment-id>/chat/completions
  ?api-version=2024-08-01-preview

The Cognisafe proxy preserves the path and query string. The effective forwarded URL is:

{UPSTREAM_URL}/openai/deployments/{deployment-id}/chat/completions?api-version=...

Set UPSTREAM_URL on the proxy deployment to your Azure OpenAI resource base URL — without a trailing slash and without the /openai/deployments/... path:

# Correct
UPSTREAM_URL=https://my-aoai-resource.openai.azure.com

# Incorrect — do not include the deployment path
UPSTREAM_URL=https://my-aoai-resource.openai.azure.com/openai/deployments/gpt-4o-prod

AAD token authentication (no static key)

Azure OpenAI supports authentication via Azure AD bearer tokens in addition to API keys. If your deployment policy prohibits static API keys on the Azure OpenAI resource, configure the proxy to use workload identity.

Proxy workload identity setup (AKS)

Enable workload identity on the proxy pod (see the AKS blueprint for the full setup). The proxy needs a managed identity with the Cognitive Services User role on the Azure OpenAI resource:

az role assignment create \
  --assignee <proxy-managed-identity-object-id> \
  --role "Cognitive Services User" \
  --scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<aoai-resource>

The proxy must then acquire an AAD token and inject it as the Authorization header on outbound requests to Azure OpenAI. The current Go proxy forwards the Authorization header from the incoming request. To use workload identity, implement token acquisition in the proxy using the Azure Identity SDK:

// proxy/main.go — outbound request enrichment
import "github.com/Azure/azure-sdk-for-go/sdk/azidentity"

cred, err := azidentity.NewWorkloadIdentityCredential(nil)
// acquire token for https://cognitiveservices.azure.com/.default
token, err := cred.GetToken(ctx, policy.TokenRequestOptions{
    Scopes: []string{"https://cognitiveservices.azure.com/.default"},
})
req.Header.Set("Authorization", "Bearer " + token.Token)
req.Header.Del("api-key") // remove static key header if present

Workload identity eliminates the AZURE_OPENAI_API_KEY secret from your cluster entirely. Clients still authenticate to the Cognisafe proxy using PROXY_API_KEY; the proxy holds the credential for Azure OpenAI internally. This is the recommended pattern for enterprise deployments.

Private endpoint option

For zero-public-egress deployments, place both the Cognisafe proxy and the Azure OpenAI private endpoint in the same VNet. The proxy resolves <resource>.openai.azure.com via private DNS to the private endpoint IP — no traffic leaves the Microsoft backbone.

VNet: 10.0.0.0/16
  ├── proxy-subnet: 10.0.1.0/24
  │     └── Cognisafe proxy pods (AKS node pool)
  │
  └── data-subnet: 10.0.3.0/24
        └── Azure OpenAI private endpoint  (10.0.3.10)
              Private DNS: privatelink.openai.azure.com → 10.0.3.10

Full Terraform and NSG configuration is in the Private Endpoint Architecture blueprint.

Testing

# Direct test via curl (bypasses SDK)
curl -s -X POST \
  "https://proxy.cognisafe.yourcompany.com/openai/deployments/gpt-4o-prod/chat/completions?api-version=2024-08-01-preview" \
  -H "Authorization: Bearer $PROXY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Say hello in JSON."}],
    "max_tokens": 30
  }' | jq .

# Verify the request was logged in Cognisafe
curl -s "https://api.cognisafe.yourcompany.com/requests?limit=1" \
  -H "Authorization: Bearer $INTERNAL_API_SECRET" | jq '.data[0] | {id, model, created_at, project_id}'

Troubleshooting

Error	Root cause	Resolution
`404 Resource Not Found` from Azure OpenAI	Deployment name does not exist in the target Azure OpenAI resource	Verify `az cognitiveservices account deployment list --name <resource> --resource-group <rg>`
`401 Unauthorized` from Azure OpenAI	`UPSTREAM_URL` is set correctly but the proxy’s auth header is wrong	If using API key auth, ensure `api-key` header is set by the proxy; if using AAD, check workload identity token scopes
`400 Bad Request — api-version`	Azure OpenAI resource does not support the requested `api-version`	Update `api_version` in your client to one listed in the Azure OpenAI API changelog
`403 Forbidden`	Managed identity lacks the `Cognitive Services User` role	Run `az role assignment list --assignee <identity-object-id>` to verify
`401` from Cognisafe proxy	Invalid `PROXY_API_KEY`	Check the `Authorization: Bearer ...` header sent to the proxy; verify the key in Key Vault matches the proxy’s `PROXY_API_KEY` env var
Proxy returns response but Cognisafe dashboard shows no request	`API_BACKEND_URL` misconfigured on proxy	The proxy logs to `/internal/log` asynchronously — check proxy container logs for connection errors to the API service
Streaming responses cut off	Proxy buffer too small	Cognisafe proxy streams responses chunk-by-chunk; check for body-size limits in any intermediate nginx ingress (`proxy-body-size` annotation)

Getting Started

SDKs

LLM Providers

Self-hosted Models

Self-hosting

Integrations

Reference Architectures

Deployment Blueprints

Safety & Scoring

Azure OpenAI Insertion

The insertion pattern

Environment variable mapping

Python: Azure OpenAI SDK

Python: Cognisafe SDK (recommended)

Azure OpenAI request path handling

AAD token authentication (no static key)

Proxy workload identity setup (AKS)

Private endpoint option

Testing

Troubleshooting

​The insertion pattern

​Environment variable mapping

​Python: Azure OpenAI SDK

​Python: Cognisafe SDK (recommended)

​Azure OpenAI request path handling

​AAD token authentication (no static key)

​Proxy workload identity setup (AKS)

​Private endpoint option

​Testing

​Troubleshooting

The insertion pattern

Environment variable mapping

Python: Azure OpenAI SDK

Python: Cognisafe SDK (recommended)

Azure OpenAI request path handling

AAD token authentication (no static key)

Proxy workload identity setup (AKS)

Private endpoint option

Testing

Troubleshooting