Table-context codec

Compacted data for agents.

tokade turns a table into compact, lossless context for a model. In the prompt it costs far fewer tokens than CSV or JSON. Behind a tool it serves schema, bounded reads, and exact per-column stats straight from the wire.

GitHub ↗

Fewer tokens

Far smaller than CSV or JSON.

Measured on Alibaba's public GenAI traces with o200k_base, tokade is 6.44× fewer tokens than compact JSON and 4.78× fewer than CSV. Every wire round-trips losslessly.

See benchmarks
tokens / formatalibaba genai traces
compact JSON41.2M
CSV30.6M
tokade6.4M
6.44× fewer tokens than compact JSON · lossless round-trip

Encode

One pass, more than twenty schemes.

tokade reads each column and picks the tightest encoding for its shape. This is a 1,000-row slice of Alibaba's public qps.csv trace, the kind of GenAI request log an agent gets asked about.

Wire format
qps.csv · 1,000 rows19,828 tokens
timestamp_anon,request_type,value 1662859176.0,API Requests,0.09000000000000001 1662859233.0,API Requests,0.09000000000000001 1662859404.0,Generative Requests,0.09000000000000001 1662859461.0,Generative Requests,0.09000000000000001 1662859461.0,API Requests,0.09000000000000001 1662859575.0,Generative Requests,0.09000000000000001 1662859689.0,API Requests,0.09000000000000001 1662859746.0,API Requests,0.18000000000000002 1662859803.0,API Requests,0.18000000000000002 1662859860.0,API Requests,0.18000000000000002 1662859917.0,Generative Requests,0.09000000000000001
encodeCSV()

tokade wire955 tokens

t=table
n=1000
timestamp_anon|s|d1:B:E…;L:1*3,2,1*11,2*4,1,3*3,2,1*3,2*3,1,4,6*2,13,9,5,3,…
request_type|s|B:…;M:118;API,Generative;R:2,2,4,4,2,2,4,2,3,2,3,8,10,8,5,2,2,…
value|s|S:1000;0.09000000000000001;Y:0.18000000000000002,0.63,0.36,0.27,1.69,…
20.8× fewer tokens · lossless round-trip

Behind a tool

One tool, in any MCP server.

Bind the wire server-side and expose tokade_table. The model asks for the exact cell, row, or column, and the server decodes just that from the wire, without reparsing the CSV.

Tool access
mcp-server.tstokade-mcp
import { encodeCSV } from "tokade"
import { tokade_table, tokadeMcpTools } from "tokade-mcp"

// your query already filtered and limited this
const { wire } = encodeCSV(csv)

// drop wire so the model sends selectors only;
// the table never reaches the model's context
const { wire: _w, ...props } = tokadeMcpTools[0].inputSchema.properties
const selectorSchema = { ...tokadeMcpTools[0].inputSchema, properties: props, required: ["action"] }

// bind the wire server-side
mcp.addTool("tokade_table", selectorSchema, (args) =>
  tokade_table({ ...args, wire }))

Querying

Bounded reads, straight from the wire.

Bound server-side, tokade-mcp serves schema and bounded reads without reparsing the table. Cold CPU per call on Node v22, against a tool that reparses raw CSV each time.

Runtime numbers
57×faster describe
2.6×faster column range
~2.5×faster cell · row · project

Install

Get started with tokade.

GitHub ↗