Freshness
AI works from copies of data: indexes, caches, tool results, files. A copy matches its source only until the source changes, and nothing in the copy can show whether it has.
Freshness, provenance, and policy, checked down to the sentence before agents see it.
These four problems share one cause. Whatever hands an agent its data, a retriever, a tool result, a cached record, or a pasted paragraph, hands it bytes. The bytes do not carry what the agent needs to know about them: whether they are current, where they came from, who may see them, what must travel with them to be read correctly, and what they may be used for. That context exists, but it lives in file systems, identity providers, audit logs, and version histories that the agent never sees.
Why has nobody fixed this? Three facts about enterprise data explain why controlling how AI uses data is hard, and why file level permissions and prompt only scanners fall short. For the underlying studies, definitions, and a worked example, see Data Quality Problems Affecting AI Agents.
People copy and paste paragraphs, tables, and boilerplate across files, messages, and apps, so controls that track whole files miss most of the copies. Backup and enterprise storage deduplication numbers often range between 80 to 90% and up to 99% for data in use on networks.
Some data identifies itself: a payment card number has a fixed format and a check digit, so a pattern match finds it in any file. Most business data carries no such marker. Take a sentence from a board memo. Whether it is fresh (the current version), stale (because a newer document replaced it), or a trade secret (valuable because its owner keeps it secret) depends on facts outside the sentence. A classifier that reads only the text cannot tell these cases apart.
A sentence is a fixed, finite string of bytes, but the context that governs how it may be used (where each copy came from, who owns it, who may read it, which version is current, and which policies apply) keeps growing with every new copy, reader, version, and policy. Content copies freely, while context stays behind: a sentence pasted into an email loses the permissions of the folder it came from. Because the same sentence means different things in different places, handing raw text to an agent strips the rules that control its use.
Caber finds relationships between identical chunks of content across documents, tool calls, and API payloads. Then it connects the context attached to every instance of the same content, so it knows what that content means and how it may be used.
Every copy of a sentence carries a piece of its context. The file it sits in has an owner and an ACL. The email it was pasted into has a recipient list and a timestamp. The API response it came back in has a caller and a tenant. Any single copy, on its own, can be a bare string with none of that. Joining the copies by their content joins their contexts, so when an agent retrieves one copy, Caber answers from all of them: which version is current, where it originated, who may see it, and what it may be used for.
This turns the three facts above from obstacles into the mechanism. Duplication supplies the copies, the copies supply the context, and the context supplies the answer that the text alone cannot. Because the match is on content rather than on file identity, it works the same way whether the chunk arrives in a document, a tool result, or an API payload, and it needs no labels in the data itself.
In practice this is six steps. The Architecture Overview covers each in detail.
Most products that govern AI data use sit at a chokepoint. A proxy inspects prompts and completions. A permission filter trims search results by document ACL. A classifier labels objects at rest. Each one sees a single surface and reasons about whole documents or whole requests. Caber's unit is the content fragment, and its question is what every copy of that fragment, across every surface, says about how it may be used. The architecture below follows from that choice.
Figure 1: Caber architecture. Three sources of context (instrumented callers, storage and event connectors, and identity connectors) feed one core, which decides in the caller's process and records relationships in a context graph.
.rbix) in sub millisecond time. The version resolver (beta) orders overlapping copies to find which is current and withholds the superseded ones. Policy is Rego compiled to WebAssembly and hosted in process, so a decision makes no network or database call. Decisions act at fragment granularity: release, mask, replace, or withhold the specific span, and every decision emits an audit receipt.Taking inspiration from OpenTelemetry for zero-code integration, Caber instruments existing Python applications without code changes. Set the connection to your Caber deployment, then run your application under caber-run:
export CSI_API_KEY="your-api-key"
export CSI_CABER_HOST="https://your-caber-instance.example.com"
caber-run python app.py
caber-run uvicorn myapp:app --host 0.0.0.0Or enable it in code, before any HTTP or agent framework library is imported:
import pycaber.auto # must come before HTTP and agent framework imports
import requests
from langgraph.prebuilt import create_react_agent
# Tool calls, retrievals, and API responses are now captured and checked against policypycaber patches 15 transports, including requests, httpx, aiohttp, and urllib3; WSGI and ASGI servers; WebSocket clients; and gRPC clients and servers. It captures tool, retriever, and memory calls in LangChain, LangGraph, CrewAI, LlamaIndex, and AutoGen without changes to agent logic. For custom functions exposed to agents, the @pycaber.tool decorator attaches policy metadata. Policies are authored centrally in the Caber admin console and deployed into caber-core, not into application code.
| Property | Value | Status |
|---|---|---|
| Fragment Granularity | 40 byte floor; 160 byte mid ceiling; 768 byte coarse ceiling; 100% byte coverage | Implemented (WRL chunker production defaults) |
| Chunker Throughput | 76.6 MiB/s per core | Measured 21 Sep 2026 |
| Policy Evaluation Path | Rego compiled to Wasm, hosted in process (wasmtime); no network or database call | Implemented |
| SDK Added Delay, HTTP | p95 1.11 ms | Measured 5 Nov 2026 (wheel 0.2.5421, Apple M1 Max) |
| SDK Added Delay, Agent Tool Call | p95 1.27 ms | Measured 5 Nov 2026 (same run) |
| SDK Added Delay, WebSocket Turn | p95 0.88 ms | Measured 5 Nov 2026 (same run) |
| Inline Decision Budget | 200 ms end to end | Design budget |
| Stale Call Precision | 0.98 to 1.00 across 10 to 40 versions | Modeled operating point 27 Oct 2026 (beta) |
| Stale Call Coverage | 0.85 to 0.96 across 10 to 40 versions | Modeled operating point 27 Oct 2026 (beta) |
Measured SDK figures cover SDK overhead only, with on behalf of identity disabled. Stale call figures come from the version resolution accuracy model and await measurement on a production corpus. Design budgets are engineering goals, not measured results.
Caber connects across your existing AI and infrastructure stacks:
| Agent Frameworks | Models & Vector Stores | Protocols & Identity | Data Sources (Connectors) |
|---|---|---|---|
| • LangGraph • LangChain • LlamaIndex • CrewAI • AutoGen | Calls detected and typed (beta): • OpenAI, Anthropic • AWS Bedrock • Google (incl. Vertex AI) • Cohere, Mistral • Groq, Together, Fireworks, DeepSeek, Perplexity • Self-hosted endpoints (e.g., vLLM, by configuration) • Pinecone*, Weaviate*, Chroma* | • Model Context Protocol (MCP) • A2A (beta) • gRPC / protobuf, WebSocket • OIDC / JWT identity providers • AWS SigV4, SPIFFE, mTLS | • AWS S3 / MinIO • Google Cloud Storage* • Azure Blob* • PostgreSQL, MySQL, SQLite • File systems • CloudTrail, Kafka*, API polling |
* On near term roadmap.
Caber is self-hosted with Docker Compose, including a single-host trial mode. Internally, caber-core uses RocksDB with Roaring bitmaps for the fingerprint index, OpenSearch for events and content, and a graph database such as Neo4j for the context graph.