Skip to main content

Documentation Index

Fetch the complete documentation index at: https://cognisafeltd.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

​
Two patching modes

Cognisafe uses different integration strategies depending on whether the LLM provider’s API is OpenAI-compatible.

​
Proxy mode

For OpenAI-compatible providers (OpenAI, Mistral, Ollama), the SDK rewrites the provider client’s base_url to point to the Cognisafe proxy running on port 8080. The Go proxy:
  1. Receives the request from your app
  2. Forwards it to the upstream LLM provider (unchanged)
  3. Streams the response back to your app
  4. In parallel, POSTs the full request and response payload to POST /internal/log on the FastAPI backend — non-blocking, so your app gets the response first
This means zero added latency on the round trip to the LLM.

​
Direct mode

For providers whose API isn’t OpenAI-compatible (Anthropic, Cohere), the SDK wraps the provider client’s create / messages.create method. After the LLM responds:
  1. Your app receives the response immediately
  2. The SDK fires a background task that ships the payload to POST /internal/log
A small amount of work happens on the SDK side, but it does not block the return of the response to your code.
ProviderModeHow
OpenAIProxybase_url rewritten to http://localhost:8080
MistralProxybase_url rewritten to http://localhost:8080
OllamaProxybase_url rewritten to http://localhost:8080
AnthropicDirectmessages.create wrapped
CohereDirectchat method wrapped

​
Async scoring pipeline

Safety scoring happens entirely off the hot path:
  1. POST /internal/log receives the payload and writes it to the llm_requests table in PostgreSQL (TimescaleDB).
  2. It immediately pushes a job to the safety_score_jobs Redis queue.
  3. One or more safety_worker processes read from the queue and run PyRIT scorers (currently content_safety, pii_detection, jailbreak_detection, data_poisoning, vector_weakness, unbounded_consumption).
  4. Scorer results are written to the safety_scores table — one row per scorer per request.
  5. The dashboard polls for new scores and displays them.
The safety workers are horizontally scalable: run as many as you need with docker compose up --scale safety_worker=N.

​
Data model

​
llm_requests

Every LLM API call captured by the proxy or SDK. A TimescaleDB hypertable partitioned on created_at for efficient time-range queries. Key columns: id, project_id, model, provider, prompt_tokens, completion_tokens, cost_usd, latency_ms, request_body (JSONB), response_body (JSONB), created_at.

​
safety_scores

One row per scorer per request. Foreign key to llm_requests. Key columns: id, request_id, scorer_name, score_value (1–5 Likert), score_label (safe / unsafe / unscored), rationale, created_at.

​
subscriptions

Per-project billing state managed by Stripe webhooks. Key columns: project_id, tier (free / pro / team / enterprise), stripe_customer_id, stripe_subscription_id, requests_this_period, period_reset_at. Rate limiting is enforced in POST /internal/log — the API returns HTTP 429 when requests_this_period exceeds the tier limit.

​
The @cognisafe.trace decorator

For non-standard LLM calls — custom HTTP clients, internal models, fine-tuned endpoints — use the generic decorator: 
import cognisafe

@cognisafe.trace(model="gpt-4o-fine-tuned")
def call_my_model(prompt: str) -> str:
    # any code that calls an LLM
    response = requests.post("https://my-internal-model/v1/chat", json={"prompt": prompt})
    return response.json()["output"]
The decorator captures the function’s input and output and ships them to /internal/log in the background.

​
Request flow diagram

Your app
   │
   │  (OpenAI call with patched base_url)
   ▼
proxy/ (Go, :8080)
   │  ◄── forwards response to your app immediately
   │
   │  POST /internal/log  (non-blocking)
   ▼
api/ (FastAPI, :8000)
   │  ── writes to llm_requests (PostgreSQL)
   │  ── pushes job to Redis queue
   ▼
safety_worker/ (Python)
   │  ── pulls from safety_score_jobs queue
   │  ── runs PyRIT scorers
   │  ── writes to safety_scores (PostgreSQL)
   ▼
web/ (Next.js, :3000)
   └── reads from PostgreSQL, renders dashboard