AKS Deployment

This blueprint walks through a full production deployment of Cognisafe on Azure Kubernetes Service (AKS). It covers namespace layout, Kubernetes manifests for all four components, secret management via Azure Key Vault CSI, TLS termination via cert-manager, autoscaling, and observability. The goal is a cluster you can hand to an on-call engineer and have them understand without tribal knowledge.

Prerequisites

AKS cluster running Kubernetes 1.28+ (az aks get-versions to verify availability in your region)
kubectl configured against the target cluster (az aks get-credentials --resource-group <rg> --name <cluster>)
Helm 3.12+
Azure CLI 2.55+ with the aks-preview extension
Azure Container Registry (ACR) attached to the cluster (az aks update --attach-acr <acr-name>)
cert-manager 1.14+ installed in the cluster (see below)
NGINX Ingress Controller installed (see below)
Azure Key Vault CSI driver enabled on the cluster

The Key Vault CSI driver and cert-manager are not installed by default on AKS. Install them before applying these manifests. Installation commands are listed in the Install cluster add-ons section.

Architecture overview

Internet
    │
    ▼
Azure Load Balancer (public IP)
    │
    ▼
NGINX Ingress Controller  (ingress-nginx namespace)
    │  TLS terminated here; cert-manager issues Let's Encrypt certs
    ├──── proxy.cognisafe.yourcompany.com  ──▶  proxy Service (ClusterIP :8080)
    ├──── api.cognisafe.yourcompany.com    ──▶  api Service   (ClusterIP :8000)
    └──── cognisafe.yourcompany.com        ──▶  web Service   (ClusterIP :3000)

Inside cognisafe-system namespace:
  proxy  Deployment (2 replicas)
    └── forwards every LLM call → upstream LLM API (egress)
    └── POSTs /internal/log → api Service (ClusterIP)
  api    Deployment (2 replicas)
    └── reads/writes → PostgreSQL (Azure DB for PostgreSQL Flexible Server, private endpoint)
    └── pushes jobs  → Redis (Azure Cache for Redis, private endpoint)
  web    Deployment (2 replicas)
    └── calls api Service (ClusterIP) server-side
  safety-worker Deployment (3 replicas, HPA: 2–10)
    └── pulls from Redis queue, runs scoring, writes scores to PostgreSQL

Secrets sourced from Azure Key Vault via CSI SecretProviderClass.
ConfigMap holds non-secret env vars shared across deployments.

Install cluster add-ons

Enable Key Vault CSI driver

az aks enable-addons \
  --resource-group <rg> \
  --name <cluster> \
  --addons azure-keyvault-secrets-provider

# Verify
kubectl get pods -n kube-system -l app=secrets-store-csi-driver

Install cert-manager

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true \
  --set global.leaderElection.namespace=cert-manager

Install NGINX Ingress Controller

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.replicaCount=2 \
  --set controller.nodeSelector."kubernetes\.io/os"=linux \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz

Wait for the LoadBalancer IP to be assigned:

kubectl get service ingress-nginx-controller -n ingress-nginx \
  --watch -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Point your DNS A records (proxy.cognisafe.yourcompany.com, api.cognisafe.yourcompany.com, cognisafe.yourcompany.com) at this IP before cert-manager can issue certificates.

Namespace and ResourceQuota

# namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: cognisafe-system
  labels:
    app.kubernetes.io/managed-by: kubectl
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: cognisafe-quota
  namespace: cognisafe-system
spec:
  hard:
    requests.cpu: "16"
    requests.memory: 32Gi
    limits.cpu: "32"
    limits.memory: 64Gi
    pods: "50"
    services: "20"
    persistentvolumeclaims: "10"

kubectl apply -f namespace.yaml

ConfigMap

Non-secret configuration shared across deployments. Update the values for your environment.

# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cognisafe-config
  namespace: cognisafe-system
data:
  # Proxy
  UPSTREAM_URL: "https://api.openai.com"
  API_BACKEND_URL: "http://api:8000"

  # API
  STRIPE_PRICE_PRO: "price_REPLACE_ME_PRO"
  STRIPE_PRICE_TEAM: "price_REPLACE_ME_TEAM"

  # Safety worker
  SCORER_MODEL: "gpt-4o-mini"

  # Web
  NEXT_PUBLIC_API_URL: "https://api.cognisafe.yourcompany.com"
  API_URL: "http://api:8000"

kubectl apply -f configmap.yaml

Secret management via Azure Key Vault CSI

Store all sensitive values in Azure Key Vault, then mount them as Kubernetes Secrets using the Secrets Store CSI driver. This avoids storing secrets in etcd or in manifests.

Enable workload identity on the cluster

az aks update \
  --resource-group <rg> \
  --name <cluster> \
  --enable-oidc-issuer \
  --enable-workload-identity

OIDC_ISSUER=$(az aks show \
  --resource-group <rg> \
  --name <cluster> \
  --query "oidcIssuerProfile.issuerUrl" -o tsv)

Create a managed identity and Key Vault access policy

az identity create \
  --resource-group <rg> \
  --name cognisafe-kv-identity

IDENTITY_CLIENT_ID=$(az identity show \
  --resource-group <rg> \
  --name cognisafe-kv-identity \
  --query clientId -o tsv)

IDENTITY_OBJECT_ID=$(az identity show \
  --resource-group <rg> \
  --name cognisafe-kv-identity \
  --query principalId -o tsv)

az keyvault set-policy \
  --name <your-keyvault-name> \
  --object-id "$IDENTITY_OBJECT_ID" \
  --secret-permissions get list

# Federated credential so the ServiceAccount can use workload identity
az identity federated-credential create \
  --name cognisafe-federated \
  --identity-name cognisafe-kv-identity \
  --resource-group <rg> \
  --issuer "$OIDC_ISSUER" \
  --subject "system:serviceaccount:cognisafe-system:cognisafe-sa" \
  --audiences api://AzureADTokenExchange

ServiceAccount

# serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: cognisafe-sa
  namespace: cognisafe-system
  annotations:
    azure.workload.identity/client-id: "<IDENTITY_CLIENT_ID>"
  labels:
    azure.workload.identity/use: "true"

SecretProviderClass

# secretproviderclass.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: cognisafe-kv-secrets
  namespace: cognisafe-system
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "false"
    clientID: "<IDENTITY_CLIENT_ID>"
    keyvaultName: "<your-keyvault-name>"
    tenantId: "<your-tenant-id>"
    objects: |
      array:
        - |
          objectName: cognisafe-postgres-url
          objectType: secret
          objectAlias: POSTGRES_URL
        - |
          objectName: cognisafe-redis-url
          objectType: secret
          objectAlias: REDIS_URL
        - |
          objectName: cognisafe-proxy-api-key
          objectType: secret
          objectAlias: PROXY_API_KEY
        - |
          objectName: cognisafe-stripe-secret-key
          objectType: secret
          objectAlias: STRIPE_SECRET_KEY
        - |
          objectName: cognisafe-stripe-webhook-secret
          objectType: secret
          objectAlias: STRIPE_WEBHOOK_SECRET
        - |
          objectName: cognisafe-internal-api-secret
          objectType: secret
          objectAlias: INTERNAL_API_SECRET
        - |
          objectName: cognisafe-openai-api-key
          objectType: secret
          objectAlias: OPENAI_API_KEY
        - |
          objectName: cognisafe-clerk-secret-key
          objectType: secret
          objectAlias: CLERK_SECRET_KEY
        - |
          objectName: cognisafe-clerk-publishable-key
          objectType: secret
          objectAlias: NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
  secretObjects:
    - secretName: cognisafe-secrets
      type: Opaque
      data:
        - objectName: POSTGRES_URL
          key: POSTGRES_URL
        - objectName: REDIS_URL
          key: REDIS_URL
        - objectName: PROXY_API_KEY
          key: PROXY_API_KEY
        - objectName: STRIPE_SECRET_KEY
          key: STRIPE_SECRET_KEY
        - objectName: STRIPE_WEBHOOK_SECRET
          key: STRIPE_WEBHOOK_SECRET
        - objectName: INTERNAL_API_SECRET
          key: INTERNAL_API_SECRET
        - objectName: OPENAI_API_KEY
          key: OPENAI_API_KEY
        - objectName: CLERK_SECRET_KEY
          key: CLERK_SECRET_KEY
        - objectName: NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
          key: NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY

The Kubernetes Secret (cognisafe-secrets) is only created after the first pod mounts the CSI volume. You must deploy at least one pod (any of the four components) before the Secret object exists. The manifests below include the CSI volume mount in every Deployment precisely so the Secret is hydrated on first rollout.

Proxy Deployment

# deploy-proxy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: proxy
  namespace: cognisafe-system
  labels:
    app: proxy
    app.kubernetes.io/component: proxy
    app.kubernetes.io/part-of: cognisafe
spec:
  replicas: 2
  selector:
    matchLabels:
      app: proxy
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: proxy
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: cognisafe-sa
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: proxy
      containers:
        - name: proxy
          image: <acr-name>.azurecr.io/cognisafe/proxy:latest
          ports:
            - containerPort: 8080
              name: http
          env:
            - name: UPSTREAM_URL
              valueFrom:
                configMapKeyRef:
                  name: cognisafe-config
                  key: UPSTREAM_URL
            - name: API_BACKEND_URL
              valueFrom:
                configMapKeyRef:
                  name: cognisafe-config
                  key: API_BACKEND_URL
            - name: PROXY_API_KEY
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: PROXY_API_KEY
          resources:
            requests:
              cpu: 250m
              memory: 128Mi
            limits:
              cpu: "1"
              memory: 256Mi
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5
            failureThreshold: 2
          volumeMounts:
            - name: secrets-store
              mountPath: /mnt/secrets-store
              readOnly: true
      volumes:
        - name: secrets-store
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: cognisafe-kv-secrets
---
apiVersion: v1
kind: Service
metadata:
  name: proxy
  namespace: cognisafe-system
spec:
  selector:
    app: proxy
  ports:
    - port: 8080
      targetPort: 8080
      name: http
  type: ClusterIP

API Deployment

# deploy-api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: cognisafe-system
  labels:
    app: api
    app.kubernetes.io/component: api
    app.kubernetes.io/part-of: cognisafe
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: api
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: cognisafe-sa
      initContainers:
        - name: migrate
          image: <acr-name>.azurecr.io/cognisafe/api:latest
          command: ["alembic", "upgrade", "head"]
          env:
            - name: POSTGRES_URL
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: POSTGRES_URL
          volumeMounts:
            - name: secrets-store
              mountPath: /mnt/secrets-store
              readOnly: true
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: api
      containers:
        - name: api
          image: <acr-name>.azurecr.io/cognisafe/api:latest
          ports:
            - containerPort: 8000
              name: http
          envFrom:
            - configMapRef:
                name: cognisafe-config
            - secretRef:
                name: cognisafe-secrets
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: "2"
              memory: 1Gi
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 15
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 8
            periodSeconds: 5
            failureThreshold: 2
          volumeMounts:
            - name: secrets-store
              mountPath: /mnt/secrets-store
              readOnly: true
      volumes:
        - name: secrets-store
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: cognisafe-kv-secrets
---
apiVersion: v1
kind: Service
metadata:
  name: api
  namespace: cognisafe-system
spec:
  selector:
    app: api
  ports:
    - port: 8000
      targetPort: 8000
      name: http
  type: ClusterIP

The migrate init container runs alembic upgrade head on every pod start. Because Alembic migrations are idempotent, this is safe with multiple replicas — only the first pod to acquire the advisory lock will run pending migrations; the rest will no-op. This avoids the need for a separate migration Job.

Web Deployment

# deploy-web.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: cognisafe-system
  labels:
    app: web
    app.kubernetes.io/component: web
    app.kubernetes.io/part-of: cognisafe
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: web
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: cognisafe-sa
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: web
      containers:
        - name: web
          image: <acr-name>.azurecr.io/cognisafe/web:latest
          ports:
            - containerPort: 3000
              name: http
          env:
            - name: NEXT_PUBLIC_API_URL
              valueFrom:
                configMapKeyRef:
                  name: cognisafe-config
                  key: NEXT_PUBLIC_API_URL
            - name: API_URL
              valueFrom:
                configMapKeyRef:
                  name: cognisafe-config
                  key: API_URL
            - name: NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
            - name: CLERK_SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: CLERK_SECRET_KEY
            - name: INTERNAL_API_SECRET
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: INTERNAL_API_SECRET
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: "1"
              memory: 512Mi
          livenessProbe:
            httpGet:
              path: /api/health
              port: 3000
            initialDelaySeconds: 15
            periodSeconds: 15
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /api/health
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 5
            failureThreshold: 2
          volumeMounts:
            - name: secrets-store
              mountPath: /mnt/secrets-store
              readOnly: true
      volumes:
        - name: secrets-store
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: cognisafe-kv-secrets
---
apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: cognisafe-system
spec:
  selector:
    app: web
  ports:
    - port: 3000
      targetPort: 3000
      name: http
  type: ClusterIP

Safety Worker Deployment

The safety worker has no ingress — it only pulls from Redis and writes to PostgreSQL.

# deploy-safety-worker.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: safety-worker
  namespace: cognisafe-system
  labels:
    app: safety-worker
    app.kubernetes.io/component: safety-worker
    app.kubernetes.io/part-of: cognisafe
spec:
  replicas: 3
  selector:
    matchLabels:
      app: safety-worker
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: safety-worker
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: cognisafe-sa
      containers:
        - name: safety-worker
          image: <acr-name>.azurecr.io/cognisafe/api:latest
          command: ["python", "workers/safety_scorer.py"]
          env:
            - name: POSTGRES_URL
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: POSTGRES_URL
            - name: REDIS_URL
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: REDIS_URL
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: cognisafe-secrets
                  key: OPENAI_API_KEY
            - name: SCORER_MODEL
              valueFrom:
                configMapKeyRef:
                  name: cognisafe-config
                  key: SCORER_MODEL
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: "2"
              memory: 2Gi
          volumeMounts:
            - name: secrets-store
              mountPath: /mnt/secrets-store
              readOnly: true
      volumes:
        - name: secrets-store
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: cognisafe-kv-secrets

HorizontalPodAutoscaler for safety-worker

Scale the worker pool based on CPU, since safety scoring is CPU/network-bound during LLM calls.

# hpa-safety-worker.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: safety-worker-hpa
  namespace: cognisafe-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: safety-worker
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120

If your Redis queue depth is a more meaningful signal than CPU (e.g., burst scoring jobs after a traffic spike), consider adding a custom metric via KEDA (ScaledObject targeting the safety_score_jobs list length). KEDA’s Azure Cache for Redis scaler handles this directly.

TLS Ingress with cert-manager

ClusterIssuer (Let’s Encrypt production)

# clusterissuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform@yourcompany.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            class: nginx

Ingress

# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: cognisafe-ingress
  namespace: cognisafe-system
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "120"
    # Force HTTPS
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    # Rate limiting at ingress layer (supplement APIM if present)
    nginx.ingress.kubernetes.io/limit-rps: "200"
spec:
  tls:
    - hosts:
        - cognisafe.yourcompany.com
        - api.cognisafe.yourcompany.com
        - proxy.cognisafe.yourcompany.com
      secretName: cognisafe-tls
  rules:
    - host: cognisafe.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 3000
    - host: api.cognisafe.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 8000
    - host: proxy.cognisafe.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: proxy
                port:
                  number: 8080

Apply all manifests

kubectl apply -f namespace.yaml
kubectl apply -f serviceaccount.yaml
kubectl apply -f secretproviderclass.yaml
kubectl apply -f configmap.yaml
kubectl apply -f deploy-proxy.yaml
kubectl apply -f deploy-api.yaml
kubectl apply -f deploy-web.yaml
kubectl apply -f deploy-safety-worker.yaml
kubectl apply -f hpa-safety-worker.yaml
kubectl apply -f clusterissuer.yaml
kubectl apply -f ingress.yaml

# Verify rollout
kubectl rollout status deployment/proxy -n cognisafe-system
kubectl rollout status deployment/api -n cognisafe-system
kubectl rollout status deployment/web -n cognisafe-system
kubectl rollout status deployment/safety-worker -n cognisafe-system

Node pool recommendations

Pool	VM SKU	Taints/Labels	Use
`system`	Standard_D4s_v5 (2 nodes, min)	`CriticalAddonsOnly=true:NoSchedule`	kube-system, ingress-nginx, cert-manager
`app`	Standard_D4s_v5 (2–8 nodes)	—	proxy, api, web workloads
`worker`	Standard_D8s_v5 spot (0–10 nodes)	`workload=scoring:NoSchedule`	safety-worker (CPU-intensive, spot-tolerant)

# Create the spot worker pool
az aks nodepool add \
  --resource-group <rg> \
  --cluster-name <cluster> \
  --name workerpool \
  --node-count 2 \
  --min-count 0 \
  --max-count 10 \
  --enable-cluster-autoscaler \
  --priority Spot \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --node-vm-size Standard_D8s_v5 \
  --node-taints "workload=scoring:NoSchedule" \
  --labels workload=scoring

Add the corresponding toleration to the safety-worker Deployment spec:

tolerations:
  - key: "workload"
    operator: "Equal"
    value: "scoring"
    effect: "NoSchedule"
nodeSelector:
  workload: scoring

Spot nodes can be evicted with 30 seconds notice. The safety worker must handle SIGTERM gracefully — finish the current scoring job, then exit. The safety worker loop should catch KeyboardInterrupt/SIGTERM and drain the in-flight job before stopping. Ensure terminationGracePeriodSeconds is set to at least 60 seconds in the pod spec.

Monitoring: Azure Monitor + Container Insights

az aks enable-addons \
  --resource-group <rg> \
  --name <cluster> \
  --addons monitoring \
  --workspace-resource-id /subscriptions/<sub>/resourcegroups/<rg>/providers/microsoft.operationalinsights/workspaces/<workspace>

After enabling Container Insights, navigate to the AKS cluster in the Azure Portal → Monitoring → Insights to see per-node CPU/memory, per-pod logs, and live event streams. Key metrics to alert on:

Metric	Alert threshold	Notes
`safety-worker` CPU utilization	> 80% for 5 min	Indicates queue backlog; HPA should kick in
`api` p99 latency	> 2s	Check PostgreSQL connection pool
Pod restart count	> 3 in 10 min	OOMKill or crash loop
Redis queue depth (`safety_score_jobs`)	> 10,000	Workers not keeping up

# Stream live logs from all proxy pods
kubectl logs -n cognisafe-system -l app=proxy --follow --prefix

# Check HPA status
kubectl get hpa -n cognisafe-system

# Exec into a worker pod to inspect Redis queue depth
kubectl exec -n cognisafe-system \
  $(kubectl get pod -n cognisafe-system -l app=safety-worker -o name | head -1) \
  -- python -c "import redis, os; r=redis.from_url(os.environ['REDIS_URL']); print(r.llen('safety_score_jobs'))"

Getting Started

SDKs

LLM Providers

Self-hosted Models

Self-hosting

Integrations

Reference Architectures

Deployment Blueprints

Safety & Scoring

Prerequisites

Architecture overview

Install cluster add-ons

Namespace and ResourceQuota

ConfigMap

Secret management via Azure Key Vault CSI

Enable workload identity on the cluster

Create a managed identity and Key Vault access policy

ServiceAccount

SecretProviderClass

Proxy Deployment

API Deployment

Web Deployment

Safety Worker Deployment

HorizontalPodAutoscaler for safety-worker

TLS Ingress with cert-manager

ClusterIssuer (Let’s Encrypt production)

Ingress

Apply all manifests

Node pool recommendations

Monitoring: Azure Monitor + Container Insights

​Prerequisites

​Architecture overview

​Install cluster add-ons

​Namespace and ResourceQuota

​ConfigMap

​Secret management via Azure Key Vault CSI

​Enable workload identity on the cluster

​Create a managed identity and Key Vault access policy

​ServiceAccount

​SecretProviderClass

​Proxy Deployment

​API Deployment

​Web Deployment

​Safety Worker Deployment

​HorizontalPodAutoscaler for safety-worker

​TLS Ingress with cert-manager

​ClusterIssuer (Let’s Encrypt production)

​Ingress

​Apply all manifests

​Node pool recommendations

​Monitoring: Azure Monitor + Container Insights

Prerequisites

Architecture overview

Install cluster add-ons

Namespace and ResourceQuota

ConfigMap

Secret management via Azure Key Vault CSI

Enable workload identity on the cluster

Create a managed identity and Key Vault access policy

ServiceAccount

SecretProviderClass

Proxy Deployment

API Deployment

Web Deployment

Safety Worker Deployment

HorizontalPodAutoscaler for safety-worker

TLS Ingress with cert-manager

ClusterIssuer (Let’s Encrypt production)

Ingress

Apply all manifests

Node pool recommendations

Monitoring: Azure Monitor + Container Insights