Skip to content

1 · Your first pipeline

Ten minutes, zero assumptions - including that you know what a "model" is. By the end you'll have run a real semantic transform over your own text.

What you're about to build

A one-line pipeline that reads text and rewrites every line through an AI model:

cat notes.txt \
| smartpipe map "translate to French"

map is the verb: it applies your instruction to each line and prints the result. To make it work you need two things - smartpipe (the tool) and a model (the AI that does the thinking). Let's get both.

1. Install smartpipe

pip install smartpipe-cli

Prefer an isolated install? The install guide covers pipx and uv.

2. Get a model

A "model" is the AI that reads your instruction and produces the answer. smartpipe doesn't include one; it talks to a model running in the cloud or locally. Pick a path:

Path A - log in with ChatGPT

Have a ChatGPT Plus/Pro plan? Log in and use it directly - no API key:

smartpipe auth login              # opens your browser
smartpipe use gpt-5.4

Path B - a cloud API key

If you have an API key (OpenAI, Anthropic, Mistral, Gemini, or OpenRouter), point smartpipe at a cloud model. Cloud models are typically faster and stronger, and cost a small amount per use.

smartpipe use gpt-5.4-mini
export OPENAI_API_KEY=sk-...           # the environment always wins; or store it with: smartpipe auth login

smartpipe auth login can also store a key for you (masked prompt, live check, owner-only file) - smartpipe auth logout removes it, and an exported variable always wins over a stored key.

Path C - local & free

Ollama runs open models on your own machine - no account, no API key, and model requests stay on that machine.

# 1. Install Ollama from https://ollama.com
# 2. Download a small, capable model (~5 GB):
ollama pull qwen3:8b
# 3. Tell smartpipe to use it:
smartpipe use ollama/qwen3:8b

Whichever path you pick, smartpipe use (with no arguments) walks you through it interactively: the text model, then embeddings, then an optional OCR model - connecting any provider you pick inline (masked key prompt, live validation), with a back row at every stage.

Not sure everything took? smartpipe doctor checks the whole setup in one screen - without spending a model call.

3. Your first transform

echo "hello world" \
| smartpipe map "translate to Spanish"
# → hola mundo

echo feeds one line in; map transforms it; the result comes out. Try it with a file:

printf "good morning\nthank you\n" \
| smartpipe map "translate to French"
# → bonjour
# → merci

One line in, one line out, in the same order. (Why lines? Everything in a pipe is an item - the item explains the five laws.) Text always arrives one item per line; pass --as file to treat a whole file as a single item (feeding smartpipe).

4. Your first extraction

Put field names in {braces} and smartpipe asks the model for structured data back, as JSON. Everything outside the braces is the instruction the model follows; the braces declare the fields to return:

echo 'Invoice from Acme Corp, dated 2026-01-15, total $1250' \
| smartpipe map "Extract {vendor, date, total number}"
# → {"vendor": "Acme Corp", "date": "2026-01-15", "total": 1250}

Because that's JSON, it composes with jq (never met jq? one-line intro in the Unix toolbox):

echo 'Invoice from Acme Corp, total $1250' \
| smartpipe map "Extract {vendor, total number}" \
| jq -r .total                 # jq pulls one field out of the JSON
# → 1250

That's the whole idea: smartpipe turns messy text into structured data you can pipe into the tools you already use.

Where to next

Continue the track: 2 · Structured data teaches braces, types, and schemas properly. Or jump by need:

  • Working with files? smartpipe map "summarize" 'reports/*.pdf' - PDFs, images, audio, and video are first-class (inputs).
  • Cutting costs? Put the free verbs first: smartpipe where 'level == "error"' before any paid stage, smartpipe sample 20 while iterating, and watch the live token/media counts in the status bar (models & providers).
  • Prepping data at scale? distinct to fold near-duplicates, extend to add judge scores, summarize/chart for the balance tables - the training-data cookbook walks the whole loop.
  • Same pipeline every week? Save it as a multi-stage .sem file or a custom verb; turn on the result cache so re-runs stop costing.