Skip to content

AI changes silently. Now you'll see it.

ReplayCI records tool calls, builds trusted references, and shows what changed before a model reaches production.

Get Beta Access Read the docs →
ReplayCI
observe Record tool calls from real traffic and draft contracts automatically.
changes Inspect the exact argument, tool call, or verdict that drifted.
gate Keep rollout blocked until shadow and review both clear.
Capture, compare, and promote in one review loop.
01 — Observe

Capture what your agent did

ReplayCI records tool calls from real runs, drafts contracts from them, and brings the important fields into review.

Real traffic · Draft contracts · No code changes
Observe - Captured runs
captured get_weather(city: "Montreal")
drafted contract.get_weather.location
review important fields promoted into diff review
Real traffic captured into reusable references
02 — Changes

See the changes that matter

Compare new runs to trusted references and inspect the exact tool call, argument, or verdict that changed.

Exact diff · Run context · Clear verdict
Changes - Semantic diff
Tool call: get_weather changed
- temperature_unit: "celsius"
+ temperature_unit: "fahrenheit"
context Reference: gpt-4o-mini · Candidate: claude-sonnet-4
Verdict: review required before promotion
03 — Shadow

Run new models in parallel

Replay a candidate against the same inputs and contracts, compare it to production, and inspect differences before rollout.

Same inputs · Same contracts · No production risk
Shadow - Candidate comparison
Production
gpt-4o-mini
4/4 pass
Candidate
claude-sonnet-4
3/4 pass
diff Candidate added an unsupported retry policy
Shadow safe - production stays untouched while you inspect the drift
04 — Merge

Promote with a clear gate

ReplayCI turns rollout into an explicit decision. If contracts fail or shadow looks unsafe, merge stays blocked.

Example CI gate
Pull request #184

Upgrade candidate model to claude-sonnet-4

Ready to merge
ReplayCI blocks promotion until observation, changes review, and shadow evaluation all clear.
pass Contract compliance
pass Deterministic reference review
pass Shadow comparison within thresholds
This card is an example CI workflow, not a ReplayCI dashboard screen.
05 — Ship

Ship AI you trust

From observation to enforcement, ReplayCI helps teams move faster without losing control.

Get Beta Access Read the docs →