SOC Integration

This reference architecture describes the full workflow from Cognisafe detection through to analyst resolution, covering alert triage, ITSM ticket creation, investigation runbook, and the feedback loop that improves detection quality over time. The goal is not to add another dashboard to the SOC toolset. The goal is to make AI threat events appear in the systems your analysts already use — ServiceNow, PagerDuty, Jira — with enough context that they can act without opening a second application.

The SOC analyst perspective

Before designing the integration, understand the constraints:

Analysts are context-switched constantly. A Cognisafe alert arriving with only a score and a category will be triaged as low priority. An alert arriving with the affected agent name, a prompt snippet, the OWASP description, a direct link to the full request, and a pre-filled ticket body will be acted on.
Alert fatigue is real. A medium-severity finding on a known agent running a known pattern should not page anyone at 3am. Severity routing must be calibrated, not uniform.
Analysts are not AI experts. The ticket must explain what the detection means in plain language, not just say “LLM01 score=0.94”.

Design the integration for the analyst, not for the system.

Architecture overview

Cognisafe Platform
  (threat_detected webhook · HMAC-signed)
          │
          ▼
Webhook Receiver
(Azure Logic App / n8n / Tines / AWS Lambda)
  1. Verify HMAC signature
  2. Classify severity
  3. Enrich: pull full request details from Cognisafe API
  4. Route by severity
          │
          ├── critical ──▶  PagerDuty Events API v2
          │                 (immediate page, dedup by request_id)
          │
          ├── high ────▶  ServiceNow REST API
          │               (P2 incident, assignment by OWASP category)
          │
          └── medium ──▶  Jira REST API
                          (backlog ticket, label owasp-{category})

          ▲ Feedback loop
          │
Resolved ticket → custom scorer tuning → reduced false positive rate

Step 1: Webhook receiver

The receiver is the single entry point for all Cognisafe events. It must be:

Fast — respond 200 within 3 seconds or Cognisafe will retry
Idempotent — Cognisafe delivers at-least-once; check event.id before creating a ticket
Signature-validated — reject all payloads without a valid X-Cognisafe-Signature header

# receiver.py — FastAPI webhook receiver (self-hosted option)
import hashlib, hmac, json
import httpx
from fastapi import FastAPI, Request, HTTPException, BackgroundTasks

app = FastAPI()
WEBHOOK_SECRET = "your-cognisafe-webhook-secret"
COGNISAFE_API_URL = "https://api.cognisafe.io"
COGNISAFE_API_KEY = "your-api-key"

def verify_signature(body: bytes, signature: str) -> bool:
    expected = "sha256=" + hmac.new(
        WEBHOOK_SECRET.encode(), body, hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, signature or "")

@app.post("/webhook/cognisafe")
async def receive_webhook(request: Request, background: BackgroundTasks):
    body = await request.body()
    sig = request.headers.get("X-Cognisafe-Signature", "")

    if not verify_signature(body, sig):
        raise HTTPException(status_code=401, detail="Invalid signature")

    event = json.loads(body)

    # Respond immediately; process async to avoid timeout
    background.add_task(process_event, event)
    return {"status": "accepted"}

async def process_event(event: dict):
    if event.get("event") != "threat_detected":
        return

    data = event["data"]
    severity = data["severity"]
    request_id = data["request_id"]

    # Idempotency: check if we've already created a ticket for this event
    if await ticket_exists(event["id"]):
        return

    # Enrich: pull full request details
    enriched = await fetch_request_details(request_id)

    if severity == "critical":
        await create_pagerduty_alert(event, enriched)
    elif severity == "high":
        await create_servicenow_incident(event, enriched)
    elif severity == "medium":
        await create_jira_ticket(event, enriched)
    # low: aggregate in daily digest (not shown here)

async def fetch_request_details(request_id: str) -> dict:
    async with httpx.AsyncClient() as client:
        resp = await client.get(
            f"{COGNISAFE_API_URL}/api/requests/{request_id}",
            headers={"Authorization": f"Bearer {COGNISAFE_API_KEY}"},
            timeout=10,
        )
        resp.raise_for_status()
        return resp.json()

Step 2: Severity routing

Cognisafe severity	ITSM action	Target system	SLA
`critical`	Immediate page	PagerDuty Events API v2	Acknowledge in 15 min
`high`	P2 incident	ServiceNow	Respond in 4 hours
`medium`	Backlog ticket	Jira	Triage in next sprint
`low`	Daily digest	Email / Slack	Weekly review

ServiceNow integration

Create a P2 incident with enough context for the assigned engineer to start investigating without opening the Cognisafe dashboard.

SERVICENOW_URL = "https://yourco.service-now.com"
SERVICENOW_USER = "cognisafe-integration"
SERVICENOW_PASS = "your-servicenow-password"

# Assignment group routing by OWASP category
OWASP_ASSIGNMENT_GROUPS = {
    "LLM01": "AI Security - Prompt Injection",
    "LLM02": "AI Security - Output Validation",
    "LLM03": "AI Security - Training & Data",
    "LLM04": "AI Security - Model Denial",
    "LLM05": "AI Security - Supply Chain",
    "LLM06": "AI Security - Excessive Agency",
    "LLM07": "AI Security - Plugins",
    "LLM08": "AI Security - Excessive Permissions",
    "LLM09": "AI Security - Overreliance",
    "LLM10": "AI Security - Model Theft",
}

async def create_servicenow_incident(event: dict, enriched: dict) -> str:
    data = event["data"]
    owasp = data["owasp_category"]
    assignment_group = OWASP_ASSIGNMENT_GROUPS.get(owasp, "AI Security - General")

    description = f"""
Cognisafe has detected a high-severity AI threat.

OWASP Category: {owasp} — {data['owasp_description']}
Agent: {data['agent_name']} (tag: {data.get('agent_tag', 'untagged')})
Model: {data['model']}
Scorer: {data['scorer_name']} (score: {data['score_value']:.2f})
Detected at: {event['created_at']}

Prompt snippet (truncated):
{data.get('prompt_snippet', 'Not available')}

Full request and response: {data.get('report_url', 'Not available')}

Investigation checklist:
1. Open the report URL above and review the full prompt and response
2. Check the agent tag — is this version expected in production?
3. Check request frequency — is this a pattern or one-off?
4. Check if a red team run (run_id: {data.get('run_id', 'none')}) triggered this
5. Escalate to critical if pattern continues, or close with evidence if false positive

Remediation guidance: https://docs.cognisafe.io/safety/owasp-coverage#{owasp.lower()}
"""

    payload = {
        "short_description": f"[Cognisafe] {owasp} {data['owasp_description']} — {data['agent_name']}",
        "description": description.strip(),
        "urgency": "2",        # High
        "impact": "2",         # High
        "priority": "2",       # P2
        "category": "AI Security",
        "subcategory": owasp,
        "assignment_group": assignment_group,
        "caller_id": "cognisafe-integration",
        "correlation_id": event["id"],           # idempotency key
        "u_cognisafe_request_id": data["request_id"],
        "u_cognisafe_report_url": data.get("report_url", ""),
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{SERVICENOW_URL}/api/now/table/incident",
            json=payload,
            auth=(SERVICENOW_USER, SERVICENOW_PASS),
            headers={"Content-Type": "application/json", "Accept": "application/json"},
            timeout=15,
        )
        resp.raise_for_status()
        ticket_number = resp.json()["result"]["number"]
        return ticket_number

The correlation_id field maps to ServiceNow’s built-in correlation field. Configure a business rule in ServiceNow to check correlation_id before creating a new incident — this prevents duplicate tickets when the webhook is retried.

Jira integration

Medium-severity findings go into the AI Security backlog for review at the next sprint triage.

JIRA_URL = "https://yourco.atlassian.net"
JIRA_USER = "cognisafe@yourco.com"
JIRA_API_TOKEN = "your-jira-api-token"
JIRA_PROJECT_KEY = "AISEC"

async def create_jira_ticket(event: dict, enriched: dict) -> str:
    data = event["data"]
    owasp = data["owasp_category"]

    description = {
        "type": "doc",
        "version": 1,
        "content": [
            {
                "type": "paragraph",
                "content": [{"type": "text", "text": f"Cognisafe detected a medium-severity AI threat."}]
            },
            {
                "type": "bulletList",
                "content": [
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"OWASP: {owasp} — {data['owasp_description']}"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Agent: {data['agent_name']} ({data.get('agent_tag', 'untagged')})"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Score: {data['score_value']:.2f} via {data['scorer_name']}"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Report: {data.get('report_url', 'N/A')}"}
                    ]}]},
                ]
            }
        ]
    }

    payload = {
        "fields": {
            "project": {"key": JIRA_PROJECT_KEY},
            "summary": f"[Cognisafe] {owasp}: {data['agent_name']} — {data['owasp_description']}",
            "description": description,
            "issuetype": {"name": "Security Finding"},
            "priority": {"name": "Medium"},
            "labels": [f"owasp-{owasp.lower()}", "cognisafe", "ai-security"],
            "components": [{"name": "AI-Security"}],
            "customfield_10100": event["id"],       # External issue ID (idempotency)
        }
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{JIRA_URL}/rest/api/3/issue",
            json=payload,
            auth=(JIRA_USER, JIRA_API_TOKEN),
            headers={"Content-Type": "application/json"},
            timeout=15,
        )
        resp.raise_for_status()
        return resp.json()["key"]

PagerDuty integration

Critical findings page the on-call engineer immediately. The dedup_key is set to request_id so that if Cognisafe retries the webhook or the same request triggers multiple scorers, only one page is sent.

PAGERDUTY_ROUTING_KEY = "your-events-api-v2-routing-key"

async def create_pagerduty_alert(event: dict, enriched: dict) -> None:
    data = event["data"]

    payload = {
        "routing_key": PAGERDUTY_ROUTING_KEY,
        "event_action": "trigger",
        "dedup_key": data["request_id"],          # prevents duplicate pages
        "payload": {
            "summary": f"[CRITICAL] Cognisafe: {data['owasp_category']} {data['owasp_description']} on {data['agent_name']}",
            "severity": "critical",
            "source": "Cognisafe",
            "timestamp": event["created_at"],
            "component": data["agent_name"],
            "group": event["project_id"],
            "class": data["owasp_category"],
            "custom_details": {
                "scorer":          data["scorer_name"],
                "score":           data["score_value"],
                "model":           data["model"],
                "agent_tag":       data.get("agent_tag", ""),
                "prompt_snippet":  data.get("prompt_snippet", ""),
                "run_id":          data.get("run_id", ""),
                "report_url":      data.get("report_url", ""),
            }
        },
        "links": [
            {
                "href": data.get("report_url", ""),
                "text": "Open in Cognisafe"
            }
        ]
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            "https://events.pagerduty.com/v2/enqueue",
            json=payload,
            timeout=10,
        )
        resp.raise_for_status()

PagerDuty will resolve an existing incident if you send an event with event_action: resolve and the same dedup_key. Wire this up to your ticket closure flow so that when a ServiceNow incident or Jira ticket is marked “resolved as false positive”, the PagerDuty alert is also resolved. This keeps the on-call queue accurate.

Investigation runbook

This is the step-by-step procedure for a SOC analyst receiving a Cognisafe critical alert in PagerDuty or a high-severity incident in ServiceNow.

Open the Cognisafe report

Click the report URL in the ticket. This opens the full request detail view in the Cognisafe dashboard, showing the complete prompt, response, all scorer results, and the request metadata (agent, model, timestamp, latency).

Review the full prompt and response

Read the full prompt, not just the snippet. Confirm the score makes sense. A jailbreak_detection score of 0.94 on a prompt that says “Ignore all previous instructions” is a true positive. A score of 0.61 on a prompt that contains the word “ignore” in an unrelated context is likely a false positive. Document your assessment.

Check the agent name and tag

Is agent_name a known agent in your agent inventory? Is agent_tag the version currently expected in production? An unknown agent name or unexpected tag is a significant escalation signal — this may be an unauthorised deployment or a misconfigured CI rollout. Check your agent registry and deployment pipeline logs.

Check request frequency

In the Cognisafe dashboard, filter by agent_name and owasp_category over the last 24 hours. Is this a one-off event or part of a pattern? A pattern of 50 similar events in the last hour indicates an active attack or a broken agent in a loop. A single event may be an anomaly or a legitimate edge case.

Check red team history

If run_id is present in the ticket, this detection was triggered during a scheduled red team run, not by a real user request. Open the Red Team section of the Cognisafe dashboard and find the run. If the detection is from a red team run, it is expected — but still review the score to confirm the control is working as designed. Close the ticket with evidence: “Red team run , expected detection confirmed.”

Escalate or close

Escalate to critical: if the agent is in production, the event is from a real user (no run_id), and the pattern repeats. Update the ServiceNow ticket to P1, page the AI system owner, and consider disabling the affected agent via the Cognisafe project settings.Close as false positive: document the reasoning, add the prompt pattern to a suppression list in the custom scorer configuration, and update the ticket status to “Closed — False Positive”. Feed back to scorer tuning (see Feedback loop below).Close as expected: for red team events or known test agents. Document and close.

SOAR playbook: automated first response

For Sentinel users, deploy this Logic App playbook to fire automatically when a new Sentinel incident is created from a Cognisafe analytics rule. It adds the report URL as a comment on the incident and optionally sends a Slack notification to the AI Security channel.

{
  "definition": {
    "triggers": {
      "Microsoft_Sentinel_incident": {
        "type": "ApiConnectionWebhook",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "path": "/incident-creation"
        }
      }
    },
    "actions": {
      "Get_Incident_Details": {
        "type": "ApiConnection",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "method": "get",
          "path": "/Incidents/@{encodeURIComponent(triggerBody()?['object']?['name'])}"
        },
        "runAfter": {}
      },
      "Add_Report_URL_Comment": {
        "type": "ApiConnection",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "method": "post",
          "path": "/Incidents/@{encodeURIComponent(triggerBody()?['object']?['name'])}/Comments",
          "body": {
            "message": "@concat('Automated first response:\n\nCognisafe Report URL: ', triggerBody()?['object']?['properties']?['additionalData']?['ReportUrl'], '\n\nOWASP Category: ', triggerBody()?['object']?['properties']?['additionalData']?['OwaspCategory'], '\n\nFollow the investigation runbook: https://docs.cognisafe.io/reference-architectures/soc-integration#investigation-runbook')"
          }
        },
        "runAfter": { "Get_Incident_Details": ["Succeeded"] }
      },
      "Notify_Slack": {
        "type": "Http",
        "inputs": {
          "method": "POST",
          "uri": "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK",
          "body": {
            "channel": "#ai-security-soc",
            "text": "@concat(':rotating_light: *Cognisafe critical alert*\n', triggerBody()?['object']?['properties']?['title'], '\n', triggerBody()?['object']?['properties']?['additionalData']?['ReportUrl'])"
          }
        },
        "runAfter": { "Add_Report_URL_Comment": ["Succeeded"] }
      }
    }
  }
}

For Splunk SOAR, implement an equivalent Python automation script triggered by the cognisafe:threat sourcetype with severity=critical.

KPI targets for AI threat operations

Define and measure these KPIs from the first week of go-live. They are the leading indicators that the integration is working.

KPI	Definition	Target
MTTD (Mean Time to Detect)	Time from LLM request to Cognisafe `threat_detected` webhook	< 60 seconds (async scoring)
MTTA (Mean Time to Acknowledge)	Time from ticket creation to analyst acknowledgement	Critical: < 15 min; High: < 4 hours
MTTR (Mean Time to Respond)	Time from ticket creation to resolution (closed or escalated)	Critical: < 2 hours; High: < 24 hours
False positive rate	Tickets closed as “false positive” / total tickets	Target < 20% at steady state
Alert coverage	% of agent deployments with Cognisafe enabled	Target 100% of production agents

Track these monthly and review in the quarterly security review. A rising false positive rate is the signal that scorer tuning is needed (see below).

Feedback loop: closed tickets improve detection quality

Every resolved ticket is an opportunity to improve the scorer. When an analyst closes a ticket as “false positive”, that prompt pattern should be added to the scorer’s exclusion list. When an analyst confirms a true positive, that pattern should be added to the keyword or Likert scorer’s positive examples.

# evals/scorers.yaml — custom keyword scorer built from SOC feedback
scorers:
  - name: jailbreak_detection
    type: keyword_list
    severity: critical
    keywords:
      # Patterns confirmed as true positives by SOC analysts
      - "ignore all previous instructions"
      - "disregard your system prompt"
      - "you are now DAN"
      - "pretend you have no restrictions"
      - "your new instructions are"
    exclusions:
      # Patterns confirmed as false positives by SOC analysts
      - "ignore all previous instructions in the document"  # doc summarisation task
      - "disregard formatting instructions"                  # layout task

Run a monthly review with the SOC team and the AI system owners to review the keyword list and calibrate thresholds. The feedback loop from SOC resolution to scorer update should take less than one working day.

Shift-left: pre-briefing the SOC before agent deployment

The SOC should not encounter a new AI agent for the first time in a 3am alert. Before any agent goes to production, the deployment team should provide the SOC with:

Agent brief: name, purpose, LLM model, data access, OWASP risk profile from the most recent Cognisafe red team run
Expected alert patterns: what Cognisafe detections are expected during normal operation and should be auto-closed
Escalation contacts: the AI engineer on call for this specific agent
Rollback procedure: how to disable the agent if a critical threat is confirmed

Template this as a Jira ticket type “AI Agent Deployment Brief” filed by the development team 5 working days before production release. The SOC reviews and signs off before go-live. This is the AI equivalent of a change advisory board review.

Getting Started

SDKs

LLM Providers

Self-hosted Models

Self-hosting

Integrations

Reference Architectures

Deployment Blueprints

Safety & Scoring

The SOC analyst perspective

Architecture overview

Step 1: Webhook receiver

Step 2: Severity routing

ServiceNow integration

Jira integration

PagerDuty integration

Investigation runbook

SOAR playbook: automated first response

KPI targets for AI threat operations

Feedback loop: closed tickets improve detection quality

Shift-left: pre-briefing the SOC before agent deployment

​The SOC analyst perspective

​Architecture overview

​Step 1: Webhook receiver

​Step 2: Severity routing

​ServiceNow integration

​Jira integration

​PagerDuty integration

​Investigation runbook

​SOAR playbook: automated first response

​KPI targets for AI threat operations

​Feedback loop: closed tickets improve detection quality

​Shift-left: pre-briefing the SOC before agent deployment

The SOC analyst perspective

Architecture overview

Step 1: Webhook receiver

Step 2: Severity routing

ServiceNow integration

Jira integration

PagerDuty integration

Investigation runbook

SOAR playbook: automated first response

KPI targets for AI threat operations

Feedback loop: closed tickets improve detection quality

Shift-left: pre-briefing the SOC before agent deployment