Documentation

AgentCache is the cost-governance and safety control plane for AI agents. It sits in front of any LLM workload and does four things gateways don't: cuts spend with savings you can verify to the dollar, puts a budget, quota, anomaly guard and kill-switch around autonomous agents, lets them resume reasoning across runs, and runs durable, human-in-the-loop background jobs.

Quick Start

import openai
client = openai.OpenAI(
    base_url="https://agentcache.ai/api/v1",
    api_key="ac_live_..."          # your AgentCache key
)
resp = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_headers={"X-OpenAI-Key": "sk-..."}  # your provider key
)

Authentication

Every request carries your key as a bearer token. Cache and control-plane endpoints are scoped per organization and per namespace.

Authorization: Bearer ac_live_...
X-Cache-Namespace: my-app        # tenant/workspace scope (cache + reasoning)

A cache miss returns 200 { "hit": false } — never an error. Keys are issued in live mode; keep them server-side.

OpenAI-compatible caching

Point the OpenAI SDK at AgentCache. Identical prompts return instantly from cache; misses pass through to your provider and are stored. Core endpoints:

POST /api/v1/chat/completions   # cache entry point (OpenAI-shaped)
POST /api/v1/embeddings         # vector entry point

Key/Value cache

Check the cache before you call a model; store the answer after. Declare the model and token counts so savings are priced automatically.

# 1 — is it cached? (a miss is 200 hit:false)
curl "https://agentcache.ai/api/cache/get?key=$HASH" \
  -H "Authorization: Bearer ac_live_..." \
  -H "X-Cache-Namespace: my-app" \
  -H "X-Model: claude-opus-5" \
  -H "X-Input-Tokens: 10000" -H "X-Output-Tokens: 2000"

# 2 — store the model's answer for next time
curl -X POST https://agentcache.ai/api/cache/set \
  -H "Authorization: Bearer ac_live_..." \
  -H "X-Cache-Namespace: my-app" \
  -d '{"key":"'$HASH'","value":{ ... }}'

Prefix cache

Agentic loops resend a large stable prefix on every step. Send the current messages and AgentCache tells you how much is an unchanged prefix of the previous turn and where to place a provider cache breakpoint.

POST /api/cache/prefix
Authorization: Bearer ac_live_...
X-Cache-Namespace: my-app
X-Model: claude-opus-5
X-Prefix-Tokens: 18000
{ "sessionId": "sess-9", "messages": [ ... ], "annotate": true }

→ { "hit": true, "reusedMessages": 7, "breakpointIndex": 6 }

Governance gate

The thing no gateway does. Before an expensive call, ask whether it's allowed under this account's budget, request quota, spend-anomaly guard and kill-switch. A well-behaved agent calls this first and stops when allow is false. A denial is a normal governed outcome — still 200, never an error.

POST /api/governance/gate
Authorization: Bearer ac_live_...
{ "model": "claude-opus-5", "inputTokens": 400000, "outputTokens": 80000 }

→ {
  "allow": false,
  "severity": "block",
  "estCostUsd": 4.00,
  "reasons": ["budget exceeded: $4,850 + est would pass $5,000"],
  "budget": { "state": "block", "remainingUsd": 150 },
  "quota":  { "state": "ok", "remaining": 66000 },
  "anomaly":{ "anomaly": false, "z": 0.7 }
}

Budgets & policy

Set the guardrails the gate enforces. Any dimension left at 0 is ungoverned (fail-safe: allow). The control-plane console drives these visually.

POST /api/governance/policy      # set budget / quota / kill-switch
{ "budgetUsd": 5000, "quotaRequests": 250000,
  "killSwitch": false, "blockOnAnomaly": false, "anomalySigma": 3 }

GET  /api/governance/status?days=30   # posture + spend/quota + anomaly + series

Savings & ROI

The verifiable number: net dollars saved over a window, with ROI and a breakdown by cache layer and model. Priced from real hits at current provider rates — unknown models return $0 so savings are never overstated.

GET /api/analytics/savings?days=30
Authorization: Bearer ac_live_...

→ {
  "windowDays": 30, "hits": 184203,
  "grossSavedUsd": 7530.42, "planCostUsd": 99, "netSavedUsd": 7431.42,
  "roi": 76,
  "byLayer": { "prefix": 4120, "exact": 2010, "semantic": 900, "reasoning": 401 },
  "byModel": { "claude-opus-5": 5200, "gpt-5.2": 2330 }
}

Reasoning memory

Durable per-agent, per-task state that survives across runs. Call resume at the start of a task to carry prior facts and decisions; commit at the end to remember what this run learned. Same task hash → resumable.

POST /api/cache/reasoning
Authorization: Bearer ac_live_...
X-Cache-Namespace: my-app

# resume prior state
{ "action": "resume", "agentId": "researcher", "task": {"repo":"x","goal":"triage"} }
→ { "hit": true, "runs": 9, "carry": { "facts": [...], "decisions": [...] } }

# commit what this run learned
{ "action": "commit", "agentId": "researcher", "task": {"repo":"x","goal":"triage"},
  "delta": { "facts": ["api rate-limited at 100rps"], "decisions": ["batch requests"] } }
→ { "committed": true, "runs": 10 }

MCP server (for agents)

Add one server to your MCP config and your agent gains the control plane — caching plus a pre-spend gate, savings, and cross-run memory — with no code changes.

{
  "mcpServers": {
    "agentcache": {
      "command": "npx",
      "args": ["-y", "agentcache-mcp"],
      "env": { "AGENTCACHE_API_KEY": "ac_live_..." }
    }
  }
}

Tools your agent gets: agentcache_get, agentcache_gate, agentcache_savings, agentcache_reasoning_resume, agentcache_reasoning_commit.

Background runs (human-in-the-loop)

Run long agents as durable jobs. Each step is gated (blocks a runaway before the spend), can pause for a human for up to 7 days with zero compute burned, and records its savings. Resume a paused run with the approval webhook.

// start a governed, durable run
inngest.send({ name: "agent/run.start", data: {
  runId: "run_123", agentId: "researcher", namespace: "acme",
  organizationId: "...", approvalThresholdUsd: 5,
  steps: [ { model: "claude-opus-5", inputTokens: 40000, outputTokens: 8000 } ]
}})

// resume a paused run
POST /api/agent/approve   { "runId": "run_123", "decision": "approve" }

API Reference

SDKs