Structured output¶

smartpipe can give you back plain text or structured JSON. Structured output is what turns messy text into data you can pipe into jq, a spreadsheet, or a database.

There are two ways to ask for it: inline braces (quick) and a --schema file (production). Both are map features.

Inline braces - for quick work¶

Put field names in {braces} and smartpipe asks the model for exactly those fields as a JSON object:

echo 'Invoice from Acme Corp, dated 2026-01-15, total $1250' \
| smartpipe map "Extract {vendor, date, total}"
# → {"vendor": "Acme Corp", "date": "2026-01-15", "total": 1250}

Everything outside the braces is the instruction the model follows; the braces declare the fields to return. A single field works too: map "Extract {total}". Types are inferred by the model - suitable for exploration. To put a literal brace in a prompt, double it: {{ and }}. Long instruction? Keep it in a file and pass @prompt.md (or --prompt-file prompt.md); braces inside the file are still live grammar.

The ladder, top to bottom¶

Five rungs; each teaches the next. Climb only as far as your task needs:

Rung	You write	You get
1	`map "Extract {vendor, total}"`	fields, model-inferred types
2	`{vendor: the supplier name, total}`	+ plain-English guidance per field
2.5	`{vendor string: the supplier, status enum(paid, unpaid)}`	+ real types inline (same vocabulary as the DSL); a fully-typed group regains server-side strict mode
3	`--schema-from "vendor string; total number >= 0; status enum(paid, unpaid)"`	+ real types and constraints - parsed deterministically, no model call, typos fail free
4	`smartpipe schema "an invoice with …" > invoice.json`	a drafted schema file (one model call, meta-validated; a failed draft exits 3 with empty `stdout`)
5	`--schema invoice.json`	full JSON Schema control

Braces carry names, types, and descriptions (ident [type] [: description]). Constraints (>=, lengths, optional) stay in the DSL or the file; braces don't carry them.

Two of the types are temporal: date (a calendar day) and datetime (a point in time). {due date} compiles to JSON Schema's format: "date", {ts datetime} to format: "date-time", and smartpipe canonicalizes whatever the model answers ("Jan 15, 2026", "15/01/2026", ISO with an offset) to ISO-8601 on the way out - YYYY-MM-DD for dates, full ISO for datetimes. An explicit UTC offset is preserved; a value without one stays naive. That makes the fields safe for where, sort --by, summarize's bin(), and chart --by-time downstream. These are deliberately the only two: no time-alone, no durations, no epoch numbers.

Object lists - one level of nesting¶

An inner brace group followed by [] asks for a list of objects with those (typed) fields:

echo 'Acme acquired Globex in 1996; Globex was founded in 1989' \
| smartpipe map "Extract {triples {subject, relation, object}[]}"
# → {"triples": [{"subject": "Acme", "relation": "acquired", "object": "Globex"}, {"subject": "Globex", "relation": "founded", "object": "1989"}]}

Inner fields speak the full type vocabulary and take : guidance exactly like outer fields - {events {name string, when date, severity enum(low, high)}[]: every notable event} - and an inner date canonicalizes per record. Add --explode events for one row per inner object. The same type works in the DSL: --schema-from "events {name string, when date}[]". Nesting stops at one level, deliberately: an object list inside an object list is refused (object lists nest one level deep - flatten the inner structure or extract in two passes).

Already have Pydantic or Zod models? Export them¶

JSON Schema is the interchange format, and both libraries emit it in one line. No smartpipe plugin needed:

python -c "import json; from myapp.models import Invoice; print(json.dumps(Invoice.model_json_schema()))" > invoice.json
smartpipe map "Extract the invoice" --schema invoice.json

npx zod-to-json-schema src/schemas.ts InvoiceSchema > invoice.json   # zod v3
# zod v4 has it built in: z.toJSONSchema(InvoiceSchema)
smartpipe map "Extract the invoice" --schema invoice.json

One caveat: providers' strict mode rejects some valid JSON Schema (optional fields, missing per-property types, $refs). smartpipe detects that and falls back to client-side validation automatically, so exported schemas still work; they just may not get the server-side guarantee.

(There is no native --schema-from-pydantic flag; the one-liner above already covers it.)

`--schema-from` - the deterministic DSL¶

field type constraints; field type … - semicolon-separated:

Types: string · number · integer · boolean · date · datetime · enum(a, b, …) · string[] · number[] · {a, b}[] (an object list)
Constraints: >= N · <= N (numbers) · minLength=N · maxLength=N (strings) · optional

Everything is required unless marked optional (which also stops smartpipe from claiming the provider's strict mode). Any typo is a usage error naming the exact fragment - before a single model call.

`--schema` file - for production¶

When you need the output to strictly conform - exact types, no surprise fields - point --schema at a standard JSON Schema file:

// invoice.json
{
  "type": "object",
  "properties": {
    "vendor": { "type": "string" },
    "date":   { "type": "string" },
    "total":  { "type": "number" }
  },
  "required": ["vendor", "total"],
  "additionalProperties": false
}

cat invoices.txt \
| smartpipe map "Extract the invoice data" --schema invoice.json

Two layers make this reliable. The schema is sent to the provider as guidance (their native JSON mode; smartpipe only claims the provider's strict variant when the schema qualifies - every field required, no open objects - because claiming it for a schema with optional fields, like date above, would be rejected outright). The guarantee, either way, is client-side: every reply is validated against your schema, repaired once if it fails, and skipped with a warning if it fails again.

With a schema, smartpipe:

enforces it via the model's native structured-output mode where available;
coerces types - a model that returns "1250" (a string) for a number field gets it turned into 1250;
drops extra fields when additionalProperties is false;
retries once if the first reply doesn't validate, re-asking the model with the specific error - and skips the item (with a warning) only if that retry also fails.

Checking a dataset: `schema --check`¶

smartpipe schema EXPR --check file.jsonl validates every row and exits 0 only when all pass. The check is open-world by default: only the fields you declared are judged - each must exist (unless marked ?/optional) and match its type/enum; undeclared fields, your own originals and the __ spine alike, are ignored. So map/extend output checks cleanly as-is - no --bare detour needed:

smartpipe extend "Add {label enum(spam, genuine)}" < posts.jsonl > out.jsonl
smartpipe schema '{label enum(spam, genuine)}' --check out.jsonl
# → note: schema check: 200 of 200 rows pass
# → note: (extras ignored - add --strict to forbid unknown fields)

--strict restores the closed world - a contract check that forbids unknown fields, byte-alike with the errors you'd get at extraction time. The extraction-time request schema is a different artifact and stays closed either way (provider strict modes demand it).

When to use which¶

	Inline `{braces}`	`--schema` file
Speed to write	Instant	Write the schema once
Type guarantees	Model-inferred	Enforced + coerced
Best for	Exploration, one-offs	Pipelines, production

The brace grammar, across verbs¶

The same {…} syntax means different things depending on the verb - one sentence covers it:

In map, braces describe the output. In filter and reduce, {field} references the input.

map "Extract {vendor, total}" → asks for those output fields.
filter "{priority} is wrong given {description}" → substitutes each item's priority and description values into the condition.

Comma-separated groups ({a, b}) are a map-only shorthand; in filter/reduce each {field} is a single input reference.

Field paths - reading nested data¶

Anywhere a verb reads a field, the field can be a path into nested records:

user.plan            a key inside an object
items[0].total       a list index, then a key (negative indexes count from the end)
a.b['weird key']     a quoted key, for names that aren't identifiers

One grammar, every surface that reads a field:

Surface	Example
`filter` / `reduce` `{braces}`	`filter "is {user.plan} priced fairly given {items[0].total}?"`
`where` left-hand sides	`where 'user.plan has "pro" and items[0].total >= 100'`
`sort --by`	`sort --by user.score --desc`
`chart`	`chart user.plan` (also `--facet` and `--by-time meta.ts:1h`)
`summarize`	`summarize 'count(), avg(metrics.score) by user.plan'`
`--fields` projection	`map … --fields user.plan,items[0].sku` (join's `left.id`, `right.name` reach the nested sides)
`write` templates	`write 'by-plan/{user.plan}.jsonl'` - reserved vars (`{path}` `{name}` `{stem}` `{ext}` `{index}`) always win over a same-named field
`--explode`	`map … --explode user.tags` - each element lands as a flat column named by the full path string (`"user.tags"`), readable downstream by the flat-column-wins rule
`--tally`	`map … --tally user.plan`
`join --on`	`join --on 'left.order.sku == right.product.sku' --right …` (alone or as blocking for a predicate)

The rules, in order:

A literal flat column wins. If a record really has a column named user.name (CSV headers produce these), that column is read - the path is tried second. Plain names behave exactly as before.
A miss at any hop is an ordinary missing field. Same skip, census note, (missing) bar, missing-last placement, or null group as a flat miss - and a key hop into a list (or an index hop into an object) is a miss too, not an error.
A malformed path is a loud, deterministic error - a.b[x] - index must be a number, a.b. - trailing dot - never a silent no-match.
Extraction stays flat. Paths read; they never name extraction output: map "Extract {user.name}" is refused with can't extract into 'user.name' - extraction field names are flat.