Phasing
What landed in v1, v2, and v3 of agent-boundary testing.
The agent-testing surface shipped in three compatibility-preserving phases.
v1: OpenAI-compatible model boundary
OpenAI-compatible chat-completions proxy (non-streaming),
app: agent process driver, cassette in trace v1 (additive header +
step artifacts), assert_tool_call grammar, trajectory diff on
re-record.
v2: Anthropic, streaming, and HTTP targets
Landed: the Anthropic Messages API (/v1/messages) and
streaming replay for both dialects. A request with stream: true is served
the recorded turn as a synthetic SSE stream in the client's own dialect
(OpenAI chat-completion chunks, or Anthropic message_start /
content_block_* / message_delta / message_stop events), so every
existing cassette serves a streaming client with no re-record and no schema
change. Chunk boundaries are synthesized rather than recorded (they carry
no test signal, and recording them would break turn matching); the
assembled turn is still what matches, and stream is transport, never part
of the comparison. Both wire protocols normalize into one neutral cassette,
tagged per turn (protocol, defaulting to openai so v1 traces are
byte-unchanged); a turn recorded in one dialect and replayed in another
diverges on that first. To keep record and replay symmetric, the record
path forwards non-streaming to the upstream and synthesizes the same stream
back to the agent. Also landed: http-target agents (drive an
already-running service via agent.url instead of spawning a process; see
"Driving a running service" above). v2 is complete.
v3: MCP tool boundary
MCP servers are a second mockable boundary for systems whose
tools are external MCP processes rather than internal functions.
Landed (v3.1): the stdio transport, with per-tool result mocks and
per-server strict-positional lanes in the trace (see "Mocking MCP tool
servers" above). Landed (v3.2): the streamable-HTTP transport
(url:/port:), an in-process loopback listener that forwards to the
real server at record (reading application/json or text/event-stream
answers) and replays the lane as single JSON bodies with zero network.
The trace shape is unchanged, so a lane is transport-blind: one recorded
through stdio replays through an HTTP-declared server and vice versa.
Landed (v3.3): server-initiated NOTIFICATIONS and the standalone
server-push SSE stream. A notification is recorded (inline in a POST's SSE
body, or off the bridged GET stream) into its lane with an after
anchor, and replayed over the GET stream flowproof now serves (a second
concurrent GET is a 409); anchors are recorded and replayed but never
matched, so the verdict is unchanged. The remaining v3.4 slice is
server-initiated REQUESTS (sampling, elicitation, roots-list), which need
answer correlation: on both transports a request mid-record fails by name
rather than corrupt a lane, and a JSON-RPC batch is a 400.