Air-gapped Enterprise Deployment

Some data cannot leave the building. Defence, intelligence, government, and certain financial and healthcare workloads operate under data handling requirements that prohibit cloud egress entirely. This reference architecture deploys the complete Cognisafe stack — inference, scoring, observability, and identity — within an air-gapped enterprise perimeter with zero runtime internet dependency.

When to use this pattern

This architecture is appropriate when your workload must satisfy any of the following:

UK/US government classification — IL3, IL4, CUI, OFFICIAL-SENSITIVE, or SECRET handling requirements prohibit sending data to commercial cloud APIs
NHS DSPT — patient data must remain within NHS-approved infrastructure; commercial LLM APIs do not hold DSP Toolkit accreditation
Financial services — FCA SYSC 8.1 operational resilience requirements, or internal risk committees that prohibit sending trading or customer data to third-party AI providers
ISO 27001 with contractual data residency — customer contracts specify that data must remain within a defined geographic or network boundary

If your organisation uses Azure Government, AWS GovCloud, or a commercial cloud at a compliant classification level, consider the Azure OpenAI + APIM pattern instead — it may satisfy your requirements with less operational overhead.

Infrastructure requirements

Component	Minimum spec	Notes
GPU server (inference)	2× NVIDIA A100 80 GB or 4× H100 80 GB	Llama 3.1 70B requires ~140 GB VRAM; A100s can shard across 2 GPUs
GPU server (scoring)	1× NVIDIA A10G or A100 40 GB	Llama Guard 3 8B fits in 16 GB; use A10G for cost efficiency
Kubernetes cluster	3-node control plane + GPU worker nodes	RKE2 or OpenShift; do not use managed K8s (EKS/AKS) in a true air-gap
PostgreSQL	16-core, 128 GB RAM, 10 TB NVMe	TimescaleDB extension required
Redis	8-core, 32 GB RAM	Sentinel configuration for HA
Private container registry	Harbor or Artifactory	Required before internet is disconnected
Identity provider	Keycloak 24+	OIDC bridge to Active Directory / LDAP
Monitoring	Prometheus + Grafana (self-hosted)	No Datadog, no cloud-native APM

Architecture

╔═══════════════════════════════════════════════════════════════════════╗
║  ENTERPRISE PERIMETER (air-gap boundary)                              ║
║                                                                       ║
║  ┌──────────────────────────────────────────────────────────────┐     ║
║  │  Identity layer                                              │     ║
║  │  Active Directory ←→ Keycloak (OIDC bridge)                  │     ║
║  └───────────────────────────┬──────────────────────────────────┘     ║
║                              │ OIDC/JWT                               ║
║  ┌───────────────────────────▼──────────────────────────────────┐     ║
║  │  Cognisafe Web (Next.js)   :3000                              │     ║
║  │  Cognisafe API (FastAPI)   :8000                              │     ║
║  │  Cognisafe Proxy (Go)      :8080                              │     ║
║  │  Safety Worker (Python)                                           │     ║
║  │  PostgreSQL (TimescaleDB)  :5432                              │     ║
║  │  Redis (Sentinel)          :6379                              │     ║
║  └───────────────────────────┬──────────────────────────────────┘     ║
║                              │ OpenAI-compatible HTTP                 ║
║  ┌───────────────────────────▼──────────────────────────────────┐     ║
║  │  vLLM inference server     :8001                              │     ║
║  │  Model: Llama-3.1-70B-Instruct (or Mistral Large 2)          │     ║
║  │  GPU: 2× A100 80 GB (NVLink)                                  │     ║
║  └──────────────────────────────────────────────────────────────┘     ║
║                                                                       ║
║  ┌──────────────────────────────────────────────────────────────┐     ║
║  │  Llama Guard 3 inference   :8002                              │     ║
║  │  Model: Llama-Guard-3-8B                                      │     ║
║  │  GPU: 1× A10G 24 GB                                           │     ║
║  └──────────────────────────────────────────────────────────────┘     ║
║                                                                       ║
║  ┌──────────────────────────────────────────────────────────────┐     ║
║  │  Private container registry (Harbor)                          │     ║
║  │  Vulnerability scanner (Trivy)                                │     ║
║  └──────────────────────────────────────────────────────────────┘     ║
║                                                                       ║
╚═══════════════════════════════════════════════════════════════════════╝

Implementation

Mirror container images to the private registry

This step requires temporary internet access on an internet-connected machine (not in the air-gap). Pull, scan, and push all required images before disconnecting.

#!/usr/bin/env bash
# Run on an internet-connected build host, NOT inside the air-gap
set -euo pipefail

REGISTRY="registry.internal.example.com"
TRIVY_SEVERITY="CRITICAL,HIGH"

images=(
  "cognisafe/proxy:latest"
  "cognisafe/api:latest"
  "cognisafe/web:latest"
  "vllm/vllm-openai:v0.6.0"
  "ghcr.io/meta-llama/llama-guard-3-8b:latest"
  "postgres:16"
  "redis:7-alpine"
  "goharbor/harbor-core:v2.11.0"
)

for image in "${images[@]}"; do
  echo "Pulling $image"
  docker pull "$image"

  echo "Scanning $image with Trivy"
  trivy image --severity "$TRIVY_SEVERITY" --exit-code 1 "$image"

  tag="${REGISTRY}/${image##*/}"
  docker tag "$image" "$tag"

  echo "Pushing $tag"
  docker push "$tag"
done

Transfer model weights separately (they are too large for a container image). Use a secure transfer medium (encrypted USB, secure file transfer appliance):

# Download model weights from HuggingFace on the internet-connected host
huggingface-cli download meta-llama/Llama-3.1-70B-Instruct \
  --local-dir ./models/llama-3.1-70b \
  --token hf_...

huggingface-cli download meta-llama/Llama-Guard-3-8B \
  --local-dir ./models/llama-guard-3 \
  --token hf_...

# Checksum the weights before transfer
sha256sum ./models/**/*.safetensors > model-checksums.sha256

Verify checksums after transfer into the air-gap before loading into vLLM.

Deploy vLLM as the production inference server

Deploy vLLM on the GPU inference nodes. Mount model weights from a local NFS share or hostPath volume.

# vllm-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-llama3
  namespace: inference
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vllm-llama3
  template:
    metadata:
      labels:
        app: vllm-llama3
    spec:
      nodeSelector:
        nvidia.com/gpu.product: A100-SXM4-80GB
      containers:
        - name: vllm
          image: registry.internal.example.com/vllm-openai:v0.6.0
          args:
            - "--model=/models/Llama-3.1-70B-Instruct"
            - "--tensor-parallel-size=2"
            - "--max-model-len=8192"
            - "--port=8001"
            - "--served-model-name=llama-3.1-70b-instruct"
          resources:
            limits:
              nvidia.com/gpu: 2
          volumeMounts:
            - name: model-weights
              mountPath: /models
          ports:
            - containerPort: 8001
      volumes:
        - name: model-weights
          nfs:
            server: nfs.internal.example.com
            path: /exports/llm-models
---
apiVersion: v1
kind: Service
metadata:
  name: vllm-llama3
  namespace: inference
spec:
  type: ClusterIP
  selector:
    app: vllm-llama3
  ports:
    - port: 8001
      targetPort: 8001

vLLM’s API is OpenAI-compatible. The Cognisafe proxy UPSTREAM_URL points at http://vllm-llama3.inference.svc.cluster.local:8001.

Deploy Llama Guard 3 as the safety scoring model

Run a separate vLLM instance for Llama Guard 3. The Cognisafe safety worker calls this endpoint instead of the commercial OpenAI API.

# llama-guard-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llama-guard
  namespace: inference
spec:
  replicas: 1
  selector:
    matchLabels:
      app: llama-guard
  template:
    metadata:
      labels:
        app: llama-guard
    spec:
      nodeSelector:
        nvidia.com/gpu.product: NVIDIA-A10G
      containers:
        - name: vllm
          image: registry.internal.example.com/vllm-openai:v0.6.0
          args:
            - "--model=/models/Llama-Guard-3-8B"
            - "--port=8002"
            - "--served-model-name=llama-guard-3-8b"
          resources:
            limits:
              nvidia.com/gpu: 1
          volumeMounts:
            - name: model-weights
              mountPath: /models
          ports:
            - containerPort: 8002
      volumes:
        - name: model-weights
          nfs:
            server: nfs.internal.example.com
            path: /exports/llm-models

Configure the Cognisafe safety worker to use Llama Guard as its scoring backend:

# safety_worker environment variables
OPENAI_API_KEY=not-required-for-local
OPENAI_BASE_URL=http://llama-guard.inference.svc.cluster.local:8002/v1
SCORER_MODEL=llama-guard-3-8b

The LLM-as-judge scorer will query Llama Guard using the OpenAI-compatible chat completions API at the local endpoint. No internet egress occurs.

Deploy the Cognisafe stack pointing at local vLLM

Set environment variables to point all Cognisafe components at the local infrastructure. Use Kubernetes Secrets for sensitive values.

# cognisafe-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cognisafe-env
  namespace: cognisafe
data:
  UPSTREAM_URL: "http://vllm-llama3.inference.svc.cluster.local:8001"
  API_BACKEND_URL: "http://cognisafe-api.cognisafe.svc.cluster.local:8000"
  REDIS_URL: "redis://redis-sentinel.cognisafe.svc.cluster.local:26379"
  POSTGRES_URL: "postgresql+asyncpg://cognisafe:$(POSTGRES_PASSWORD)@postgres.cognisafe.svc.cluster.local:5432/cognisafe"
  OPENAI_BASE_URL: "http://llama-guard.inference.svc.cluster.local:8002/v1"
  SCORER_MODEL: "llama-guard-3-8b"
  # Disable Stripe — all subscriptions managed manually (see Step 6)
  STRIPE_SECRET_KEY: ""
  STRIPE_WEBHOOK_SECRET: ""
  # Email — use internal SMTP relay (see Step 7)
  SMTP_HOST: "smtp.internal.example.com"
  SMTP_PORT: "587"

Apply the Cognisafe Helm chart with these overrides:

helm upgrade --install cognisafe ./charts/cognisafe \
  --namespace cognisafe \
  --create-namespace \
  --values values-airgap.yaml \
  --set image.repository=registry.internal.example.com/cognisafe \
  --set image.pullPolicy=IfNotPresent

Configure Keycloak as the OIDC provider

Keycloak bridges your on-premises Active Directory to OIDC, which Cognisafe’s web UI uses for authentication (via Clerk-compatible OIDC or direct OIDC integration).

# Deploy Keycloak
helm upgrade --install keycloak bitnami/keycloak \
  --namespace identity \
  --set auth.adminUser=admin \
  --set auth.adminPassword="$(openssl rand -base64 32)" \
  --set externalDatabase.host=postgres.cognisafe.svc.cluster.local \
  --set externalDatabase.database=keycloak \
  --set image.registry=registry.internal.example.com

In the Keycloak admin console:

Create a realm: cognisafe
Add a User Federation: LDAP — point at your Active Directory domain controller
Sync users from AD
Create a Client: cognisafe-web — Client Protocol: openid-connect, Access Type: confidential
Set Valid Redirect URIs: https://cognisafe.internal.example.com/*

Configure Cognisafe to use Keycloak:

# web/ environment variables
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=""        # Clerk not used in air-gap
OIDC_ISSUER=https://keycloak.internal.example.com/realms/cognisafe
OIDC_CLIENT_ID=cognisafe-web
OIDC_CLIENT_SECRET=<keycloak-client-secret>
OIDC_REDIRECT_URI=https://cognisafe.internal.example.com/auth/callback

Disable Stripe — provision subscriptions manually

Set STRIPE_SECRET_KEY to an empty string. The Cognisafe API falls back to reading subscription tier directly from the subscriptions table when Stripe is not configured.Insert a manual subscription row for each project:

INSERT INTO subscriptions (
    project_id,
    tier,
    stripe_customer_id,
    stripe_subscription_id,
    requests_this_period,
    period_start,
    period_end
) VALUES (
    'my-project-id',
    'enterprise',
    NULL,           -- no Stripe customer
    NULL,           -- no Stripe subscription
    0,
    NOW(),
    NOW() + INTERVAL '1 year'
);

Enterprise tier has no request limit. All users on an air-gapped deployment should use the enterprise tier — billing is handled through your existing procurement process, not Stripe.

Route alerts to internal SMTP or Splunk

Disable Resend (the cloud email provider) and configure Cognisafe to deliver alerts via your internal SMTP relay or Splunk HTTP Event Collector.Internal SMTP:

# safety_worker + api environment variables
ALERT_BACKEND=smtp
SMTP_HOST=smtp.internal.example.com
SMTP_PORT=587
SMTP_FROM=cognisafe-alerts@example.com
SMTP_TO=security-ops@example.com
SMTP_TLS=true
SMTP_USERNAME=cognisafe-svc
SMTP_PASSWORD=<smtp-service-account-password>

Splunk HEC (preferred for SOC integration):

ALERT_BACKEND=webhook
ALERT_WEBHOOK_URL=https://splunk.internal.example.com:8088/services/collector/event
ALERT_WEBHOOK_HEADERS='{"Authorization": "Splunk <hec-token>"}'

The Cognisafe webhook payload is JSON and maps directly to a Splunk sourcetype. Create a Splunk saved search on sourcetype=cognisafe threat_detected to drive your SOC alerting rules.

Egress requirements

Phase	Egress required	What for
Initial setup (internet-connected host)	Yes	Pull container images, download model weights, download Helm charts
Model weight transfer	Physical media only	Encrypted USB or secure file transfer appliance
Runtime (steady state)	None	All inference, scoring, identity, and alerting are local
Updates	Physical media or one-way data diode	Periodic model updates, security patches (see operational runbook)

“Zero-egress runtime” means no network connections leave the air-gap boundary during normal operation. This includes NTP (use an internal NTP server), DNS (use internal resolver), and certificate revocation (pre-load CRLs or use OCSP stapling with cached responses).

Model selection

Production inference

Llama 3.1 70B Instruct — Best open-weight general model at time of writing. Requires 2× A100 80 GB with tensor parallelism. Competitive with GPT-4o on instruction following and coding.Mistral Large 2 — Strong alternative; slightly lower VRAM requirement. Consider for cost-constrained deployments.Llama 3.1 8B Instruct — For low-latency use cases or constrained GPU budgets. Fits on a single A10G.

Safety scoring

Llama Guard 3 8B — Meta’s purpose-built content safety classifier. Trained specifically for LLM input/output classification across OWASP LLM risk categories. Fits on a single A10G 24 GB.ShieldLM — Alternative if Llama Guard 3 is not available in your approved software list.

Compliance posture

Framework	Relevant control	How this architecture satisfies it
UK Government IL3 / IL4	Data must remain within HMG-approved infrastructure	All inference and logging within the perimeter; no commercial LLM API calls
NHS DSPT	Standard 07 — processing personal data	Patient data never leaves the network boundary; Llama Guard 3 scores for PII in every request
ISO 27001 A.12.4	Logging and monitoring	TimescaleDB audit log for every LLM and tool invocation; Keycloak provides identity audit trail
ISO 27001 A.13.1	Network segregation	Air-gap enforced at the network layer; Kubernetes NetworkPolicy restricts pod-to-pod communication
SOC 2 CC6.1	Logical access	Keycloak + AD enforces authentication and RBAC; Cognisafe API keys scoped per project
NCSC CAF (UK)	B1 — Service protection policies	Change management runbook for model updates; vulnerability scanning of all container images before admission

For accreditation evidence packs, export the following from Cognisafe:

llm_requests table (full, or time-bounded) — demonstrates audit trail
safety_scores table — demonstrates content screening
Keycloak audit log (realm events) — demonstrates access control
Trivy scan reports for all running container images

Operational runbook: model updates

Model updates require careful change management in an air-gapped environment. Follow this runbook for every update.

Prepare on the internet-connected build host

# Download the new model weights
huggingface-cli download meta-llama/Llama-3.1-70B-Instruct \
  --revision v2.0.0 \
  --local-dir ./models/llama-3.1-70b-v2

# Verify checksums from the official HuggingFace page
sha256sum -c official-checksums.sha256

# Scan with Trivy (model weights should be scanned for embedded malware)
trivy fs --severity CRITICAL,HIGH ./models/llama-3.1-70b-v2

# Create a manifest for the change management ticket
find ./models/llama-3.1-70b-v2 -name "*.safetensors" \
  -exec sha256sum {} \; > model-update-manifest.sha256

Transfer into the air-gap

Transfer the model weights and container image tarballs via your approved secure transfer mechanism. Log the transfer in your change management system (ServiceNow, Jira, etc.) with the SHA256 manifest as evidence.

# Inside the air-gap: verify checksums after transfer
sha256sum -c model-update-manifest.sha256

# Copy weights to the NFS share
rsync -av --checksum ./models/llama-3.1-70b-v2/ \
  nfs.internal.example.com:/exports/llm-models/Llama-3.1-70B-Instruct-v2/

Rolling restart with zero downtime

Update the vLLM Deployment to point at the new model path. Kubernetes performs a rolling update — the old model instance stays up until the new one is healthy.

kubectl set env deployment/vllm-llama3 \
  --namespace inference \
  "MODEL_PATH=/models/Llama-3.1-70B-Instruct-v2"

# Monitor the rollout
kubectl rollout status deployment/vllm-llama3 --namespace inference --timeout=10m

Run a smoke test against the new model before marking the change as complete:

curl -s http://vllm-llama3.inference.svc.cluster.local:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.1-70b-instruct", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 10}' \
  | jq '.choices[0].message.content'

Never run kubectl set image or patch a Deployment with a new model path unless you have verified the SHA256 checksums of the transferred weights against the official manifest. A supply chain compromise at the model weight level is indistinguishable from a legitimate update without this check.

Configure Prometheus alerts for GPU memory utilisation and model inference latency (p99). A sudden spike in p99 latency after a model update is an indicator that the new model has different compute characteristics and may require vLLM tuning (--max-model-len, --gpu-memory-utilization) before the rollout is fully complete.

Getting Started

SDKs

LLM Providers

Self-hosted Models

Self-hosting

Integrations

Reference Architectures

Deployment Blueprints

Safety & Scoring

Air-gapped Enterprise Deployment

When to use this pattern

Infrastructure requirements

Architecture

Implementation

Egress requirements

Model selection

Production inference

Safety scoring

Compliance posture

Operational runbook: model updates

​When to use this pattern

​Infrastructure requirements

​Architecture

​Implementation

​Egress requirements

​Model selection

Production inference

Safety scoring

​Compliance posture

​Operational runbook: model updates

When to use this pattern

Infrastructure requirements

Architecture

Implementation

Egress requirements

Model selection

Compliance posture

Operational runbook: model updates