> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognisafe.uk/llms.txt
> Use this file to discover all available pages before exploring further.

# SOC Integration

> Integrating Cognisafe AI threat intelligence into existing security operations workflows and ITSM systems

This reference architecture describes the full workflow from Cognisafe detection through to analyst resolution, covering alert triage, ITSM ticket creation, investigation runbook, and the feedback loop that improves detection quality over time.

The goal is not to add another dashboard to the SOC toolset. The goal is to make AI threat events appear in the systems your analysts already use — ServiceNow, PagerDuty, Jira — with enough context that they can act without opening a second application.

## The SOC analyst perspective

Before designing the integration, understand the constraints:

* **Analysts are context-switched constantly.** A Cognisafe alert arriving with only a score and a category will be triaged as low priority. An alert arriving with the affected agent name, a prompt snippet, the OWASP description, a direct link to the full request, and a pre-filled ticket body will be acted on.
* **Alert fatigue is real.** A medium-severity finding on a known agent running a known pattern should not page anyone at 3am. Severity routing must be calibrated, not uniform.
* **Analysts are not AI experts.** The ticket must explain what the detection means in plain language, not just say "LLM01 score=0.94".

Design the integration for the analyst, not for the system.

***

## Architecture overview

```text theme={null}
Cognisafe Platform
  (threat_detected webhook · HMAC-signed)
          │
          ▼
Webhook Receiver
(Azure Logic App / n8n / Tines / AWS Lambda)
  1. Verify HMAC signature
  2. Classify severity
  3. Enrich: pull full request details from Cognisafe API
  4. Route by severity
          │
          ├── critical ──▶  PagerDuty Events API v2
          │                 (immediate page, dedup by request_id)
          │
          ├── high ────▶  ServiceNow REST API
          │               (P2 incident, assignment by OWASP category)
          │
          └── medium ──▶  Jira REST API
                          (backlog ticket, label owasp-{category})

          ▲ Feedback loop
          │
Resolved ticket → custom scorer tuning → reduced false positive rate
```

***

## Step 1: Webhook receiver

The receiver is the single entry point for all Cognisafe events. It must be:

* **Fast** — respond 200 within 3 seconds or Cognisafe will retry
* **Idempotent** — Cognisafe delivers at-least-once; check `event.id` before creating a ticket
* **Signature-validated** — reject all payloads without a valid `X-Cognisafe-Signature` header

```python theme={null}
# receiver.py — FastAPI webhook receiver (self-hosted option)
import hashlib, hmac, json
import httpx
from fastapi import FastAPI, Request, HTTPException, BackgroundTasks

app = FastAPI()
WEBHOOK_SECRET = "your-cognisafe-webhook-secret"
COGNISAFE_API_URL = "https://api.cognisafe.io"
COGNISAFE_API_KEY = "your-api-key"

def verify_signature(body: bytes, signature: str) -> bool:
    expected = "sha256=" + hmac.new(
        WEBHOOK_SECRET.encode(), body, hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, signature or "")

@app.post("/webhook/cognisafe")
async def receive_webhook(request: Request, background: BackgroundTasks):
    body = await request.body()
    sig = request.headers.get("X-Cognisafe-Signature", "")

    if not verify_signature(body, sig):
        raise HTTPException(status_code=401, detail="Invalid signature")

    event = json.loads(body)

    # Respond immediately; process async to avoid timeout
    background.add_task(process_event, event)
    return {"status": "accepted"}

async def process_event(event: dict):
    if event.get("event") != "threat_detected":
        return

    data = event["data"]
    severity = data["severity"]
    request_id = data["request_id"]

    # Idempotency: check if we've already created a ticket for this event
    if await ticket_exists(event["id"]):
        return

    # Enrich: pull full request details
    enriched = await fetch_request_details(request_id)

    if severity == "critical":
        await create_pagerduty_alert(event, enriched)
    elif severity == "high":
        await create_servicenow_incident(event, enriched)
    elif severity == "medium":
        await create_jira_ticket(event, enriched)
    # low: aggregate in daily digest (not shown here)

async def fetch_request_details(request_id: str) -> dict:
    async with httpx.AsyncClient() as client:
        resp = await client.get(
            f"{COGNISAFE_API_URL}/api/requests/{request_id}",
            headers={"Authorization": f"Bearer {COGNISAFE_API_KEY}"},
            timeout=10,
        )
        resp.raise_for_status()
        return resp.json()
```

***

## Step 2: Severity routing

| Cognisafe severity | ITSM action    | Target system           | SLA                   |
| ------------------ | -------------- | ----------------------- | --------------------- |
| `critical`         | Immediate page | PagerDuty Events API v2 | Acknowledge in 15 min |
| `high`             | P2 incident    | ServiceNow              | Respond in 4 hours    |
| `medium`           | Backlog ticket | Jira                    | Triage in next sprint |
| `low`              | Daily digest   | Email / Slack           | Weekly review         |

***

## ServiceNow integration

Create a P2 incident with enough context for the assigned engineer to start investigating without opening the Cognisafe dashboard.

```python theme={null}
SERVICENOW_URL = "https://yourco.service-now.com"
SERVICENOW_USER = "cognisafe-integration"
SERVICENOW_PASS = "your-servicenow-password"

# Assignment group routing by OWASP category
OWASP_ASSIGNMENT_GROUPS = {
    "LLM01": "AI Security - Prompt Injection",
    "LLM02": "AI Security - Output Validation",
    "LLM03": "AI Security - Training & Data",
    "LLM04": "AI Security - Model Denial",
    "LLM05": "AI Security - Supply Chain",
    "LLM06": "AI Security - Excessive Agency",
    "LLM07": "AI Security - Plugins",
    "LLM08": "AI Security - Excessive Permissions",
    "LLM09": "AI Security - Overreliance",
    "LLM10": "AI Security - Model Theft",
}

async def create_servicenow_incident(event: dict, enriched: dict) -> str:
    data = event["data"]
    owasp = data["owasp_category"]
    assignment_group = OWASP_ASSIGNMENT_GROUPS.get(owasp, "AI Security - General")

    description = f"""
Cognisafe has detected a high-severity AI threat.

OWASP Category: {owasp} — {data['owasp_description']}
Agent: {data['agent_name']} (tag: {data.get('agent_tag', 'untagged')})
Model: {data['model']}
Scorer: {data['scorer_name']} (score: {data['score_value']:.2f})
Detected at: {event['created_at']}

Prompt snippet (truncated):
{data.get('prompt_snippet', 'Not available')}

Full request and response: {data.get('report_url', 'Not available')}

Investigation checklist:
1. Open the report URL above and review the full prompt and response
2. Check the agent tag — is this version expected in production?
3. Check request frequency — is this a pattern or one-off?
4. Check if a red team run (run_id: {data.get('run_id', 'none')}) triggered this
5. Escalate to critical if pattern continues, or close with evidence if false positive

Remediation guidance: https://docs.cognisafe.io/safety/owasp-coverage#{owasp.lower()}
"""

    payload = {
        "short_description": f"[Cognisafe] {owasp} {data['owasp_description']} — {data['agent_name']}",
        "description": description.strip(),
        "urgency": "2",        # High
        "impact": "2",         # High
        "priority": "2",       # P2
        "category": "AI Security",
        "subcategory": owasp,
        "assignment_group": assignment_group,
        "caller_id": "cognisafe-integration",
        "correlation_id": event["id"],           # idempotency key
        "u_cognisafe_request_id": data["request_id"],
        "u_cognisafe_report_url": data.get("report_url", ""),
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{SERVICENOW_URL}/api/now/table/incident",
            json=payload,
            auth=(SERVICENOW_USER, SERVICENOW_PASS),
            headers={"Content-Type": "application/json", "Accept": "application/json"},
            timeout=15,
        )
        resp.raise_for_status()
        ticket_number = resp.json()["result"]["number"]
        return ticket_number
```

<Note>
  The `correlation_id` field maps to ServiceNow's built-in correlation field. Configure a business rule in ServiceNow to check `correlation_id` before creating a new incident — this prevents duplicate tickets when the webhook is retried.
</Note>

***

## Jira integration

Medium-severity findings go into the AI Security backlog for review at the next sprint triage.

```python theme={null}
JIRA_URL = "https://yourco.atlassian.net"
JIRA_USER = "cognisafe@yourco.com"
JIRA_API_TOKEN = "your-jira-api-token"
JIRA_PROJECT_KEY = "AISEC"

async def create_jira_ticket(event: dict, enriched: dict) -> str:
    data = event["data"]
    owasp = data["owasp_category"]

    description = {
        "type": "doc",
        "version": 1,
        "content": [
            {
                "type": "paragraph",
                "content": [{"type": "text", "text": f"Cognisafe detected a medium-severity AI threat."}]
            },
            {
                "type": "bulletList",
                "content": [
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"OWASP: {owasp} — {data['owasp_description']}"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Agent: {data['agent_name']} ({data.get('agent_tag', 'untagged')})"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Score: {data['score_value']:.2f} via {data['scorer_name']}"}
                    ]}]},
                    {"type": "listItem", "content": [{"type": "paragraph", "content": [
                        {"type": "text", "text": f"Report: {data.get('report_url', 'N/A')}"}
                    ]}]},
                ]
            }
        ]
    }

    payload = {
        "fields": {
            "project": {"key": JIRA_PROJECT_KEY},
            "summary": f"[Cognisafe] {owasp}: {data['agent_name']} — {data['owasp_description']}",
            "description": description,
            "issuetype": {"name": "Security Finding"},
            "priority": {"name": "Medium"},
            "labels": [f"owasp-{owasp.lower()}", "cognisafe", "ai-security"],
            "components": [{"name": "AI-Security"}],
            "customfield_10100": event["id"],       # External issue ID (idempotency)
        }
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{JIRA_URL}/rest/api/3/issue",
            json=payload,
            auth=(JIRA_USER, JIRA_API_TOKEN),
            headers={"Content-Type": "application/json"},
            timeout=15,
        )
        resp.raise_for_status()
        return resp.json()["key"]
```

***

## PagerDuty integration

Critical findings page the on-call engineer immediately. The `dedup_key` is set to `request_id` so that if Cognisafe retries the webhook or the same request triggers multiple scorers, only one page is sent.

```python theme={null}
PAGERDUTY_ROUTING_KEY = "your-events-api-v2-routing-key"

async def create_pagerduty_alert(event: dict, enriched: dict) -> None:
    data = event["data"]

    payload = {
        "routing_key": PAGERDUTY_ROUTING_KEY,
        "event_action": "trigger",
        "dedup_key": data["request_id"],          # prevents duplicate pages
        "payload": {
            "summary": f"[CRITICAL] Cognisafe: {data['owasp_category']} {data['owasp_description']} on {data['agent_name']}",
            "severity": "critical",
            "source": "Cognisafe",
            "timestamp": event["created_at"],
            "component": data["agent_name"],
            "group": event["project_id"],
            "class": data["owasp_category"],
            "custom_details": {
                "scorer":          data["scorer_name"],
                "score":           data["score_value"],
                "model":           data["model"],
                "agent_tag":       data.get("agent_tag", ""),
                "prompt_snippet":  data.get("prompt_snippet", ""),
                "run_id":          data.get("run_id", ""),
                "report_url":      data.get("report_url", ""),
            }
        },
        "links": [
            {
                "href": data.get("report_url", ""),
                "text": "Open in Cognisafe"
            }
        ]
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(
            "https://events.pagerduty.com/v2/enqueue",
            json=payload,
            timeout=10,
        )
        resp.raise_for_status()
```

<Warning>
  PagerDuty will resolve an existing incident if you send an event with `event_action: resolve` and the same `dedup_key`. Wire this up to your ticket closure flow so that when a ServiceNow incident or Jira ticket is marked "resolved as false positive", the PagerDuty alert is also resolved. This keeps the on-call queue accurate.
</Warning>

***

## Investigation runbook

This is the step-by-step procedure for a SOC analyst receiving a Cognisafe critical alert in PagerDuty or a high-severity incident in ServiceNow.

<Steps>
  <Step title="Open the Cognisafe report">
    Click the report URL in the ticket. This opens the full request detail view in the Cognisafe dashboard, showing the complete prompt, response, all scorer results, and the request metadata (agent, model, timestamp, latency).
  </Step>

  <Step title="Review the full prompt and response">
    Read the full prompt, not just the snippet. Confirm the score makes sense. A `jailbreak_detection` score of 0.94 on a prompt that says "Ignore all previous instructions" is a true positive. A score of 0.61 on a prompt that contains the word "ignore" in an unrelated context is likely a false positive. Document your assessment.
  </Step>

  <Step title="Check the agent name and tag">
    Is `agent_name` a known agent in your agent inventory? Is `agent_tag` the version currently expected in production? An unknown agent name or unexpected tag is a significant escalation signal — this may be an unauthorised deployment or a misconfigured CI rollout. Check your agent registry and deployment pipeline logs.
  </Step>

  <Step title="Check request frequency">
    In the Cognisafe dashboard, filter by `agent_name` and `owasp_category` over the last 24 hours. Is this a one-off event or part of a pattern? A pattern of 50 similar events in the last hour indicates an active attack or a broken agent in a loop. A single event may be an anomaly or a legitimate edge case.
  </Step>

  <Step title="Check red team history">
    If `run_id` is present in the ticket, this detection was triggered during a scheduled red team run, not by a real user request. Open the Red Team section of the Cognisafe dashboard and find the run. If the detection is from a red team run, it is expected — but still review the score to confirm the control is working as designed. Close the ticket with evidence: "Red team run {run_id}, expected detection confirmed."
  </Step>

  <Step title="Escalate or close">
    **Escalate to critical:** if the agent is in production, the event is from a real user (no run\_id), and the pattern repeats. Update the ServiceNow ticket to P1, page the AI system owner, and consider disabling the affected agent via the Cognisafe project settings.

    **Close as false positive:** document the reasoning, add the prompt pattern to a suppression list in the custom scorer configuration, and update the ticket status to "Closed — False Positive". Feed back to scorer tuning (see Feedback loop below).

    **Close as expected:** for red team events or known test agents. Document and close.
  </Step>
</Steps>

***

## SOAR playbook: automated first response

For Sentinel users, deploy this Logic App playbook to fire automatically when a new Sentinel incident is created from a Cognisafe analytics rule. It adds the report URL as a comment on the incident and optionally sends a Slack notification to the AI Security channel.

```json theme={null}
{
  "definition": {
    "triggers": {
      "Microsoft_Sentinel_incident": {
        "type": "ApiConnectionWebhook",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "path": "/incident-creation"
        }
      }
    },
    "actions": {
      "Get_Incident_Details": {
        "type": "ApiConnection",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "method": "get",
          "path": "/Incidents/@{encodeURIComponent(triggerBody()?['object']?['name'])}"
        },
        "runAfter": {}
      },
      "Add_Report_URL_Comment": {
        "type": "ApiConnection",
        "inputs": {
          "host": { "connection": { "name": "@parameters('$connections')['azuresentinel']['connectionId']" } },
          "method": "post",
          "path": "/Incidents/@{encodeURIComponent(triggerBody()?['object']?['name'])}/Comments",
          "body": {
            "message": "@concat('Automated first response:\n\nCognisafe Report URL: ', triggerBody()?['object']?['properties']?['additionalData']?['ReportUrl'], '\n\nOWASP Category: ', triggerBody()?['object']?['properties']?['additionalData']?['OwaspCategory'], '\n\nFollow the investigation runbook: https://docs.cognisafe.io/reference-architectures/soc-integration#investigation-runbook')"
          }
        },
        "runAfter": { "Get_Incident_Details": ["Succeeded"] }
      },
      "Notify_Slack": {
        "type": "Http",
        "inputs": {
          "method": "POST",
          "uri": "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK",
          "body": {
            "channel": "#ai-security-soc",
            "text": "@concat(':rotating_light: *Cognisafe critical alert*\n', triggerBody()?['object']?['properties']?['title'], '\n', triggerBody()?['object']?['properties']?['additionalData']?['ReportUrl'])"
          }
        },
        "runAfter": { "Add_Report_URL_Comment": ["Succeeded"] }
      }
    }
  }
}
```

For Splunk SOAR, implement an equivalent Python automation script triggered by the `cognisafe:threat` sourcetype with `severity=critical`.

***

## KPI targets for AI threat operations

Define and measure these KPIs from the first week of go-live. They are the leading indicators that the integration is working.

| KPI                             | Definition                                                    | Target                                  |
| ------------------------------- | ------------------------------------------------------------- | --------------------------------------- |
| MTTD (Mean Time to Detect)      | Time from LLM request to Cognisafe `threat_detected` webhook  | \< 60 seconds (async scoring)           |
| MTTA (Mean Time to Acknowledge) | Time from ticket creation to analyst acknowledgement          | Critical: \< 15 min; High: \< 4 hours   |
| MTTR (Mean Time to Respond)     | Time from ticket creation to resolution (closed or escalated) | Critical: \< 2 hours; High: \< 24 hours |
| False positive rate             | Tickets closed as "false positive" / total tickets            | Target \< 20% at steady state           |
| Alert coverage                  | % of agent deployments with Cognisafe enabled                 | Target 100% of production agents        |

Track these monthly and review in the quarterly security review. A rising false positive rate is the signal that scorer tuning is needed (see below).

***

## Feedback loop: closed tickets improve detection quality

Every resolved ticket is an opportunity to improve the scorer. When an analyst closes a ticket as "false positive", that prompt pattern should be added to the scorer's exclusion list. When an analyst confirms a true positive, that pattern should be added to the keyword or Likert scorer's positive examples.

```yaml theme={null}
# evals/scorers.yaml — custom keyword scorer built from SOC feedback
scorers:
  - name: jailbreak_detection
    type: keyword_list
    severity: critical
    keywords:
      # Patterns confirmed as true positives by SOC analysts
      - "ignore all previous instructions"
      - "disregard your system prompt"
      - "you are now DAN"
      - "pretend you have no restrictions"
      - "your new instructions are"
    exclusions:
      # Patterns confirmed as false positives by SOC analysts
      - "ignore all previous instructions in the document"  # doc summarisation task
      - "disregard formatting instructions"                  # layout task
```

Run a monthly review with the SOC team and the AI system owners to review the keyword list and calibrate thresholds. The feedback loop from SOC resolution to scorer update should take less than one working day.

***

## Shift-left: pre-briefing the SOC before agent deployment

The SOC should not encounter a new AI agent for the first time in a 3am alert. Before any agent goes to production, the deployment team should provide the SOC with:

1. **Agent brief:** name, purpose, LLM model, data access, OWASP risk profile from the most recent Cognisafe red team run
2. **Expected alert patterns:** what Cognisafe detections are expected during normal operation and should be auto-closed
3. **Escalation contacts:** the AI engineer on call for this specific agent
4. **Rollback procedure:** how to disable the agent if a critical threat is confirmed

Template this as a Jira ticket type "AI Agent Deployment Brief" filed by the development team 5 working days before production release. The SOC reviews and signs off before go-live. This is the AI equivalent of a change advisory board review.