Record and replay

This tutorial records one agent flow and replays it with zero model calls. You will finish with a versioned trace and a run report that can fail CI when the agent calls the wrong tool.

Before you start

You need:

• Flowproof installed with npm or pip;
• Node.js for the example agent; and
• an OpenAI-compatible model key for the one-time recording.

The runnable example is in examples/agent-demo/. If you installed Flowproof from npm, use the Node example below. The Python example has the same flow and assertions.

1. Read the flow

The flow runs a weather agent, supplies a result for its get_weather tool, and asserts the agent's behavior:

# examples/agent-demo/weather-node.flow.yaml
name: Weather assistant answers with the forecast (Node)
app: agent
agent:
  command: node examples/agent-demo/weather_agent.mjs
tools:
  - name: get_weather
    result: { city: Nairobi, sky: sunny, temp_c: 26 }
steps:
  - prompt: What is the weather in Nairobi right now? Use your tools.
  - assert_tool_call: get_weather where city contains Nairobi
  - assert: reply contains sunny

assert_tool_call fails if the agent uses the wrong tool, sends the wrong city, or calls the tool out of order.

2. Record once

Install the agent's SDK, configure the model key, and record:

npm install openai
npx flowproof config ai
npx flowproof record examples/agent-demo/weather-node.flow.yaml

Recording is the only phase that calls the real model. Flowproof runs the real agent and captures its model exchange as a trace.

3. Replay without a model

Run the same flow:

npx flowproof run examples/agent-demo/weather-node.flow.yaml

A successful replay looks like this:

  [PASS] s0001 prompt
  [PASS] s0002 get_weather where city contains Nairobi
  [PASS] s0003 reply contains sunny
PASS: Weather assistant answers with the forecast (Node)

At replay, Flowproof serves the recorded model response back to the same agent and rechecks the trajectory. Replay needs no model key and makes no provider network call.

Two boundaries matter:

A bare prompt: flow is one turn. Flowproof joins all prompt: steps into one task delivered up front. Use conversation: for a real back-and-forth. See Agent-boundary testing for conversations, MCP tool mocking, and the complete assertion contract.

Python users can run weather.flow.yaml after installing the OpenAI Python package.

Record and replay a UI flow

The same lifecycle applies to a live application. On Windows 10 or 11, this spec drives Calculator to compute 5 + 3:

# calc.flow.yaml
name: Add two numbers
app: calc
steps:
  - Type 5
  - Press plus
  - Type 3
  - Press equals
  - assert: display shows 8

Record and replay it:

flowproof record calc.flow.yaml
flowproof run calc.flow.yaml

Flowproof derives calc.trace.jsonl from the spec name. Keep the spec and trace together unless you need an explicit location:

flowproof record calc.flow.yaml --out traces/calc.trace.jsonl
flowproof run calc.flow.yaml --trace traces/calc.trace.jsonl
flowproof heal calc.flow.yaml --trace traces/calc.trace.jsonl

Directory suite runs ignore --trace and resolve each trace by convention. A relocated trace therefore appears missing in a suite unless it is placed where the convention expects it.

Recognize a successful run

A passing Calculator replay prints each step and the report path:

  [PASS] s0001 Type 5
  [PASS] s0002 Press plus
  [PASS] s0003 Type 3
  [PASS] s0004 Press equals
  [PASS] s0005 display shows 8
PASS: Add two numbers (2154 ms) -> .flowproof\runs\20260718T120000.000Z\report.html

Exit code 0 means pass, 1 means test failure, and 2 means an execution or configuration error. Each run bundle contains:

Password fields are always masked. Add redact: rules for other sensitive regions. See Run recording for capture controls and the artifact format.

Choose recording options

These options change evidence artifacts, not execution or verdicts.

Verify or update a recording

Use --verify when a flow is safe to perform twice:

flowproof record shop.flow.yaml --verify

Verification immediately replays the new trace and refuses the recording if it cannot reproduce itself. It can repeat orders, emails, payments, or other side effects, so leave it off when repetition is unsafe.

Use --reuse when only part of an existing flow changed:

flowproof record shop.flow.yaml --reuse

Flowproof reuses matching, resolvable trace steps and authors only new or drifted steps. If model authoring is configured, recording can also repair a failing spec step with a bounded set of edits. Pass --no-repair to stop on the first failure. See Resilience and healing for selector fallback and reviewable healing.

Resume a recording that stopped

When a recording fails, the steps it got through are kept in <flow>.checkpoint.json beside the trace. Record again with --resume to replay them and author only from the step that failed:

flowproof record shop.flow.yaml --resume

With no checkpoint, --resume replays the existing trace's steps instead, the way --reuse does, and with neither it records from the start. Unlike --reuse, repair stays on. Each replayed step is checked against the live app first, and one whose target has gone is authored fresh. A finished recording removes the checkpoint; it is a working file, not something to commit. A flow that asserts assert_no_secret_leak keeps no checkpoint, because its steps are only written once the whole run's leak scan has passed.

Element actions wait until the target is enabled, stable, and able to receive events. A gate that does not clear reports the failed condition, such as element exists but is disabled after 5000ms.

Next steps

• Run multiple flows with manifests, dependencies, and checkpoints in Run a suite.
• Add Flowproof to an existing project with Adopting Flowproof.
• Use --json when another program invokes the CLI. Structured output goes to stdout, while per-step progress continues on stderr.