LM Studio

Overview

LM Studio is a desktop application for running LLMs locally on Mac, Windows, and Linux. It exposes a local OpenAI-compatible server, making it a natural fit for development and testing workflows where you want Cognisafe’s safety scoring and observability without sending data to cloud providers.

Start the LM Studio server

In LM Studio, go to Local Server → load a model → click Start Server. By default it listens on http://localhost:1234/v1.

Proxy configuration

# Cognisafe proxy env vars
UPSTREAM_URL=http://host.docker.internal:1234

If running the Cognisafe proxy in Docker, use host.docker.internal (Mac/Windows) or your machine’s LAN IP (Linux) to reach the LM Studio server on the host.

If running the proxy directly on the same machine:

UPSTREAM_URL=http://localhost:1234

SDK setup

import cognisafe
from openai import OpenAI

cognisafe.configure(
    api_key="csk_your_key_here",
    project_id="local-dev",
    proxy_url="http://localhost:8080",
)
cognisafe.patch_openai()

client = OpenAI(
    api_key="lm-studio",  # LM Studio ignores the API key
    base_url="http://localhost:8080/v1",
)

response = client.chat.completions.create(
    model="lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

Copy the exact model identifier from the LM Studio UI — it includes the GGUF file variant (e.g. Q4_K_M). The proxy passes the model name through unchanged.

Docker Compose for local development

Add Cognisafe alongside LM Studio in a local docker-compose.yml:

services:
  cognisafe-proxy:
    image: cognisafe/proxy:latest
    ports:
      - "8080:8080"
    environment:
      UPSTREAM_URL: http://host.docker.internal:1234
      PROXY_API_KEY: dev-key
      API_BACKEND_URL: http://cognisafe-api:8000
    extra_hosts:
      - "host.docker.internal:host-gateway"  # Linux only

  cognisafe-api:
    image: cognisafe/api:latest
    ports:
      - "8000:8000"
    environment:
      POSTGRES_URL: postgresql+asyncpg://postgres:postgres@postgres:5432/cognisafe
      REDIS_URL: redis://redis:6379

  postgres:
    image: timescale/timescaledb:latest-pg16
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: cognisafe

  redis:
    image: redis:7-alpine

Point your app at http://localhost:8080/v1 and LM Studio continues to run as normal on port 1234.

Typical dev workflow

Load your model in LM Studio and start the local server
Start the Cognisafe proxy pointing at localhost:1234
Run your agent or app — all calls are logged and scored
Open the Cognisafe dashboard to inspect prompts, responses, and safety scores
When ready for production, change UPSTREAM_URL to your cloud LLM provider — no app code changes needed

LM Studio is ideal for testing your safety scorer configuration. You can replay adversarial prompts against a local model to verify your custom scorers fire correctly before deploying to production.

Getting Started

SDKs

LLM Providers

Self-hosted Models

Self-hosting

Integrations

Reference Architectures

Deployment Blueprints

Safety & Scoring

Overview

Start the LM Studio server

Proxy configuration

SDK setup

Docker Compose for local development

Typical dev workflow

​Overview

​Start the LM Studio server

​Proxy configuration

​SDK setup

​Docker Compose for local development

​Typical dev workflow

Overview

Start the LM Studio server

Proxy configuration

SDK setup

Docker Compose for local development

Typical dev workflow