Models & providers¶

What's a "model"?¶

A model is the AI that reads your instruction and produces the answer. smartpipe doesn't contain one - it sends your text to a model running either in the cloud (OpenAI, Anthropic, or any compatible service) or locally (on your machine, via Ollama). You choose which; smartpipe just talks to it.

Cloud vs. local¶

	Cloud	Local (Ollama)
Cost	Small charge per use	Free
Privacy	Text and supported media go to the provider	Input stays on your machine when Ollama is local; first use may download model weights
Setup	Log in with ChatGPT, or get an API key	Install Ollama, pull a model
Speed / quality	Usually faster and stronger	Depends on your hardware

With no configured chat model, smartpipe tries local Ollama first. If it finds a non-embedding model, it uses that model and prints the exact ollama/... name.

If no model is configured and no usable Ollama model is found, smartpipe stops with a setup screen. It does not silently fall through to a cloud provider just because a key or login exists.

Once you choose a cloud model or ChatGPT login path, the relevant item data for that run goes to that provider. That is an explicit provider choice, not a hidden fallback.

Model strings¶

You name a model with a string. Two forms:

Explicit provider: ollama/qwen3:8b, openai/gpt-5.4-mini, anthropic/claude-opus-4-8, mistral/mistral-large-latest, gemini/gemini-3.1-flash-lite, openrouter/deepseek/deepseek-chat (OpenRouter is explicit-only - its names are other vendors' names).
Bare name: smartpipe routes by shape - claude-* → Anthropic, gpt-* / o* / text-embedding-* → OpenAI, the Mistral family (mistral-*, ministral-*, codestral-*, magistral-*, devstral-*, pixtral-*, open-mistral-*, open-mixtral-*, voxtral-*) → Mistral, gemini-* → Gemini, anything else → Ollama. OpenRouter never routes bare. So gpt-5.4-mini and openai/gpt-5.4-mini mean the same thing.

Namespaced Ollama models keep working as bare names too: hf.co/org/model routes to Ollama whole - including hf.co/mistralai/..., which is an Ollama name, not a Mistral cloud model.

Cloud credentials¶

Keys never live in smartpipe's config file. They come from the environment:

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export MISTRAL_API_KEY=...            # console.mistral.ai
export GEMINI_API_KEY=...             # aistudio.google.com
export OPENROUTER_API_KEY=sk-or-...   # openrouter.ai/keys
export JINA_API_KEY=...               # jina.ai (embeddings only)

...or from smartpipe auth login PROVIDER, which prompts for a key (masked), validates it with one catalog request (a failed check offers retry / store anyway / skip - the provider may just be having a bad minute), and stores it at ~/.local/share/smartpipe/auth.json with owner-only permissions. An exported variable always wins over a stored key. smartpipe auth list shows what's connected with keys masked (sk-...9f2) and the live source; smartpipe auth logout PROVIDER removes an entry.

The anthropic SDK ships with smartpipe - Claude models work out of the box. (smartpipe tells you if it's missing.) Mistral needs nothing extra - chat, mistral-embed embeddings, and pixtral-* vision all ride the built-in adapter. Any other OpenAI-compatible endpoint - Groq, OpenRouter, a local llama.cpp server - works by pointing smartpipe at it:

export SMARTPIPE_OPENAI_BASE_URL=https://api.groq.com/openai

Log in with ChatGPT (no API key)¶

If you have a ChatGPT Plus/Pro plan, you can use it directly:

smartpipe auth login              # opens your browser (or --headless for a code)
echo "hi" \
| smartpipe map "translate to French" --model gpt-5.4

What to know:

Which models: the ChatGPT login serves the Codex-era family (gpt-5.x, gpt-*-codex). The low-cost platform tiers - gpt-5.4-mini and gpt-5.4-nano - are rejected on this wire ("not supported ... with a ChatGPT account") and need an API key. For cheap, high-volume runs, set OPENAI_API_KEY and use gpt-5.4-nano, the low-cost OpenAI tier.
Precedence: an exported OPENAI_API_KEY always wins over a login - a key is an explicit, billable choice. Unset it to use your plan.
No embeddings: embed/top_k need an API key or a local model.
Where tokens live: ~/.config/smartpipe/auth.json, permissions 0600, refreshed automatically, removed with smartpipe auth logout. (This file holds only login tokens; API keys stored via auth login live in a separate file, ~/.local/share/smartpipe/auth.json.)
Two OpenAI entries on purpose: auth login lists openai (API key) and openai (ChatGPT login) separately - they are different wires with different capabilities. smartpipe auth login openai keeps meaning the ChatGPT flow (back-compat); the key wire is smartpipe auth login openai-api.
Why no login for Anthropic/Mistral: they don't offer one to third-party tools. OpenAI's login uses the same public OAuth client the Codex CLI and other open-source tools use, and smartpipe identifies itself (originator: smartpipe).

Forcing a path (D24)¶

There is deliberately no --auth knob and no --api-key/--base-url flags (argv leaks into ps and shell history). The environment is the override:

OPENAI_API_KEY=sk-... smartpipe map …          # force the key path
env -u OPENAI_API_KEY smartpipe map …          # force the ChatGPT login path
SMARTPIPE_OPENAI_BASE_URL=https://… smartpipe …  # point the wire elsewhere

Gemini rides its native wire (and watches video)¶

Gemini chat uses Google's native endpoint, the only wired endpoint that takes video input: map "what happens?" demo.mp4 --model gemini-3.1-flash-lite sends the actual video (visuals and soundtrack heard together, no conversion). On every other model the video ladder converts to frames + audio automatically. Structured output translates to Gemini's response-schema dialect; embeddings stay on the compat wire. SMARTPIPE_GEMINI_BASE_URL still points both.

Context windows: probed, not guessed¶

smartpipe keeps a conservative context-window table per provider. When an input exceeds that table, it asks providers that publish metadata for the real number: Ollama, Mistral, Gemini, and OpenRouter. OpenAI and Anthropic do not publish it.

Example: the table floors Gemini at 128k, but the probe can discover that gemini-2.5-flash holds 1M and widen the budget.

SMARTPIPE_CONTEXT_TOKENS=32000 overrides everything. If an estimate is still wrong, reduce self-corrects: a rejected chunk is split in half and retried.

One command, one coherent setup: `smartpipe use`¶

smartpipe use stamps a COMPLETE bundle - a chat model plus the embedder that pairs with it - in one command:

smartpipe use                    # interactive: text model, embeddings, OCR
smartpipe use gemini             # gemini-3.1-flash-lite + gemini-embedding-001
smartpipe use openai             # gpt-5.4-mini + text-embedding-3-small
smartpipe use ollama             # the best installed local model + a local embedder
smartpipe use gpt-5.4-mini       # any model ref: that model + its provider's pairing

A provider whose key is absent refuses with the smartpipe auth login fix and stamps nothing - never a partial setup. Re-running use refreshes the whole bundle, which is also the cure for a drifted config.

Cloud picks are multimodal by default: they stamp allow-captions = true, so images and audio convert to text through your own model when a run needs it (fractions of a cent each, every conversion disclosed per row). Picking the cloud setup is the consent; the command says so. Local setups keep the conservative --allow-captions opt-in.

Every save also writes a receipt at the top of config.toml - # stamped by: smartpipe use (…) - so the file documents how it got that way. (Profiles from older versions are retired: their keys load with one warning, are ignored, and disappear on the next save.)

Setting a default¶

smartpipe use gpt-5.4-mini                  # save a default (any provider: smartpipe use ollama works the same)
smartpipe using                             # see the effective settings + where each comes from

Override the default for a single command with --model:

cat data.txt \
| smartpipe map "summarize" --model claude-opus-4-8

Precedence¶

When the same setting is specified more than one way, the most specific wins:

--model flag  >  SMARTPIPE_MODEL env var  >  config file  >  Ollama autodetect

smartpipe using prints each value with its origin, so you can see which value takes effect.

Two models: chat and embedding¶

Most verbs use a chat model. embed and top_k use a separate embedding model, configured independently - the embeddings stage of smartpipe use sets it (or edit embed-model in config.toml directly).

The stt-model role¶

The stt-model config key - the speech-to-text stage of smartpipe use sets it - names a dedicated transcriber. When set, it runs FIRST in the audio ladder (a configured transcriber signals wanting verbatim text - LLM hearing paraphrases), falling back to the LLM rung and local whisper on failure. Two forms:

stt-model = "openai/whisper-1" - the remote wire. A paid cloud conversion, so the allow-captions consent gates it like every other one.
stt-model = "local" - pins on-device whisper as the first rung: free, verbatim, and audio never leaves your machine (no consent needed - nothing is uploaded or billed).

Unset, smartpipe picks the sensible strategy automatically:

Your situation	Transcription
OpenAI API key	`whisper-1` via the API (it supports it)
OpenAI ChatGPT login only	local whisper (the login wire has no speech-to-text (STT))
Gemini	the model hears audio natively
Ollama	local whisper (no STT endpoint)

SMARTPIPE_STT_MODEL / stt-model override the matrix per run or per account (SMARTPIPE_STT_MODEL=local works too). Only the openai remote wire exists today; the key accepts provider/model so more can land behind the same seam. smartpipe doctor has an stt row showing which path the ladder would take and whether it is healthy, and doctor --probe goes further for a remote resolution: one tiny disclosed transcription through the real wire, so a bad key or a dead endpoint reports as itself instead of charting healthy (a local resolution is verified but never run — no model download inside doctor).

graph's scanning modes (--fast and --name-top) honor the role too: --stt-model > SMARTPIPE_STT_MODEL > config, and there configuring or flagging the role IS the consent (the ocr-model rule) — audio and video tracks transcribe through the resolved wire, disclosed once per run when it is paid. Bare --fast with nothing configured stays local and free: the scanning modes never consult the auto-matrix with a chat model, so an ambient OpenAI key changes nothing. Full mode's native audio ladder does not honor the role yet — the flag refuses outside the scanning modes.

The ocr-model role¶

The ocr-model config key - the OCR stage of smartpipe use sets it - names a dedicated document parser. When set, ingested PDFs and images run through it instead of the local extraction ladder - page markdown becomes the item text, and a multi-page PDF becomes one item per page (the __source spine carries the page cut, exactly like split --by pages). Setting the role is the consent; every parsed page is disclosed per row:

⚠ degraded: report.pdf p.3 document → markdown (parsed by mistral/mistral-ocr-latest)

The role is provider-agnostic:

A mistral ref rides the dedicated /v1/ocr wire (needs MISTRAL_API_KEY; the endpoint charges per page).
Any other ref reads pages through the normal chat-vision wire with an extract-the-text framing - ocr-model = "ollama/llava" is a free local OCR. For PDFs on this rung, pages with a healthy text layer keep their locally extracted text (zero calls); only thin, image-bearing pages (scans) go through the model.

Inside the conversion ladder, a configured ocr-model also outranks the vision-chat rung for page images, so scanned documents read as text everywhere. If a parse fails, the item falls back to the local extraction ladder with a note - never a hard stop. --ocr-model (per run) > SMARTPIPE_OCR_MODEL > the config key. Unset: nothing changes.

Every ingesting verb honors the role - the per-item verbs, the embedding verbs, cluster/diff/distinct/outliers, split, join (the --right file included), and reader mode - with the same disclosures, the same --max-calls belt, and a preflight note before more than 20 billable pages start parsing. Mistral bills per page, so smartpipe reserves the complete PDF page count before upload; a document that does not fit in the remaining belt is not sent. The OCR stage of smartpipe use offers the dedicated mistral wire, your chat model, and the vision-capable catalog entries (capped at the menu height; typing a name covers the rest).

The media-embed-model role¶

The media-embed-model config key (e.g. media-embed-model = "jina/jina-clip-v2" in config.toml) names a JOINT text+image embedder. When set, embed and top_k route media items to it as pixels while text items keep using embed-model - and a text query can rank an image corpus, because both live in the same space.

One vector space per run is the law: if a run holds BOTH text and media items while the two roles name different models, smartpipe refuses loudly (vectors from two models can't be compared). The fix it names: set embed-model to the joint model too, or feed media-only input. Unset, media items keep today's behavior (native embedding when embed-model itself is media-capable, the caption pivot otherwise).

--media-embed-model (per run) > SMARTPIPE_MEDIA_EMBED_MODEL > the config key.

Request batching¶

By default, small items in map, extend, and filter coalesce: eligible items wait about 75 ms (or until 12 gather) and fly as ONE request, each in its own labeled <input id="r1"> block; the model answers with one JSON object keyed per item, and the answers fan back out in input order. Grouping follows (model, output schema) - prompts that interpolate fields still batch, each carrying its own instruction inside its block. Instructions and bodies are XML-text escaped before wrapping, so a literal </input> in text or a YAML scalar remains data and cannot forge a neighboring record.

Never batched: media items, oversized items, repair retries, and every other verb - they take the solo path unchanged. A missing or invalid per-item answer sends exactly that item back through the solo path (which owns the repair ladder); valid neighbors keep their answers.

The accounting stays per real call: --max-calls counts a batch of 12 items as 1 call, the circuit breaker watches real wire failures, the meter counts real usage, and --concurrency caps simultaneous outbound calls rather than objects waiting to be processed. Batch size 6 + concurrency 1 + 18 eligible items therefore means three sequential calls. One stderr note per run discloses attempts and recovery: note: batching: 500 items in 42 packed calls · 3 solo recoveries. The result cache stays per item - hits skip the batch entirely, and batched answers are cached individually.

Missing/invalid keys recover solo once. A packed-schema incompatibility also splits to solo once. Auth/model/setup faults stop immediately; an exhausted rate-limit or transport retry is fanned back to the affected rows without launching one new retry ladder per row. Every recovery crosses the same budget, breaker, and concurrency gate.

Kill switch: smartpipe config batching off, or SMARTPIPE_BATCH=off for one run. SMARTPIPE_BATCH_SIZE accepts 2..12 (default 12; larger values fault) and SMARTPIPE_BATCH_WINDOW_MS (default 75) tunes the coalesce window.

The usage ledger¶

smartpipe usage shows what the meter observed over the past hour, day, week, month, and lifetime - runs, tokens in/out, media, audio time, paid conversions. smartpipe usage reset zeroes it (printing the previous lifetime so the number isn't lost) and remembers the reset time. Only model-touching runs count; the ledger lives in ~/.local/state/smartpipe/ and never leaves your machine.