Using TOON in an LLM prompt

You have measured your payload, TOON comes out cheaper, and now you have to actually put it in a prompt. That part is rarely written down, so here it is.

The shape of the call

TOON belongs at the boundary, not in your application. Keep working with objects, encode immediately before the call, decode immediately after:

import { encode, decode } from '@toon-format/toon'

const prompt = [
  SYSTEM_EXPLAINING_TOON,
  '',
  'Data:',
  encode(records),        // ← objects in, TOON text out
  '',
  'Return JSON matching the schema.',
].join('\n')

Two things follow from putting it here and nowhere else. Your business logic never sees TOON, so nothing else in the codebase has to understand it. And turning the optimisation off is deleting one line — which you will want when you switch models and have to re-measure.

Do not ask the model to reply in TOON. Ask for JSON. Models generate JSON extremely reliably because they have seen enormous amounts of it, and every structured-output and function-calling API speaks JSON schemas. TOON’s saving is on input tokens, which is where the bulk of the volume sits anyway. Generating an unfamiliar format trades a reliable path for a small saving on the smaller half of the traffic.

Explaining the format

The model has to know how to read what you sent. How much explanation you need depends on the model, and the honest answer is that you have to test it — but the shape of the explanation is stable, and short beats thorough:

The data below is in TOON format: an array header names the fields
once in braces, then each following line is one record with values
in that same order, comma-separated.

users[2]{id,name,role}:
  1001,ada,admin
  1002,alan,member

is equivalent to:
[{"id":1001,"name":"ada","role":"admin"},
 {"id":1002,"name":"alan","role":"member"}]

One worked example does more than three paragraphs of specification, because it lets the model pattern-match rather than parse a description.

This explanation is a fixed cost per call. Roughly 80–120 tokens in the form above. That number is why small payloads do not benefit: saving 40 tokens per call while spending 100 explaining how to read them is a loss, and no comparison table that omits the prompt will tell you so.

If your payloads are large, put the explanation in a cached system prompt. Most providers bill cached input tokens at a fraction of the normal rate, which turns the fixed cost into nearly nothing across a long conversation.

Getting structured output back

Unchanged from any other pipeline: ask for JSON, validate it, do not trust it.

const parsed = MySchema.parse(JSON.parse(response))

Worth saying because it is easy to lose sight of: sending TOON does not make the response less reliable, and it does not make it more reliable either. The input format and the output contract are separate problems.

Two failure modes

Silent misreads on non-uniform data. If your records are not perfectly uniform, the encoder falls back to a per-record form that looks similar but reads differently — and a model that learned the tabular pattern from your example may misalign fields. Check uniformity before you switch, not after. The converter reports it as a percentage for exactly this reason.

Ambiguous values. In tabular form values are bare, so 007 and true and 2026-09-02 arrive without quotes to mark them as strings. The specification handles quoting correctly, and the reference encoder applies it — but if you build the format by hand, or post-process the output, this is where you break round-tripping. Use the library.

What it is worth

At the measured saving for a typical enveloped API response, priced across a range of input rates:

Price per 1M input tokens	Saved per month
$0.10	$14.30
$0.50	$71.50
$1.00	$143
$3.00	$429
$10.00	$1,430

The column that matters is the one nearest your actual rate. Two things worth noticing: at cheap-model prices the whole exercise buys you a coffee, and at frontier prices with real volume it becomes a line item worth having.

Which is the practical summary of this entire site — the format is a lever whose length depends on your payload shape, your call volume and your price per token. Measure it before you plan around it.

Checklist

• Encode at the call boundary; keep objects everywhere else.
• Ask for JSON back, never TOON.
• Include one worked example, not a specification.
• Put the explanation in a cached system prompt if payloads are large.
• Check field uniformity before switching; re-check after schema changes.
• Re-measure when you change models. The tokenizer changes with them.