Market data QA: two-judge review and report revision
Official cookbook recipe
- Industry
- Finance · Market data QA
- Source
- Official open-multi-agent cookbook
- Stage
- Runnable example
This is an official open-multi-agent cookbook example, maintained by YuanASI. It checks a stretch of BTCUSDT order book data before it goes into a backtest: the report agent sees only locally continuous deltas and trades and gives a provisional ACCEPT; two judges, each holding a data source the report agent cannot see, review it, overturn it and require QUARANTINE, and the report is final only when both vote to approve.
Scenario
Typical scenario: a quant team records an exchange's L2 order book deltas for backtesting. Looking at the delta file alone, update IDs chain end to end, the book never crosses and quantities are valid, so it looks ready to use; but whether the snapshot lines up with the delta stream, and whether the collector disconnected and reconnected midway, are recorded separately in the snapshot metadata and the collector log. A verdict drawn from one source lets intervals with gaps slip into the backtest.
This example writes "conclude first, then have reviewers holding different material look for errors" into the task definition: the report task carries a verify config, and each judge receives its own data source through judgePrompt; if anyone dissents, the objections go back to the report agent for revision, the verdict changes from ACCEPT to QUARANTINE, and the report states the missing update ID range, the disconnect window and the backfill action.
How it works
| Roles | Task DAG | Tools | Model | Deployment |
|---|---|---|---|---|
| depth-sequence-extractor | Task 1 extract-depth-sequence (root task, no dependencies) | tools: [], no tools | claude-sonnet-4-6 / anthropic | Local script, single run via npx tsx |
| trade-activity-extractor | Task 2 extract-trade-activity (root task, alongside task 1) | tools: [], no tools | claude-sonnet-4-6 / anthropic | Same |
| integrity-report-proposer | Task 3 propose-integrity-report, depends on tasks 1 and 2, carries verify review | tools: [], no tools | claude-sonnet-4-6 / anthropic | Same |
| protocol-continuity-judge | Judge for task 3, sole holder of snapshot-metadata.json | tools: [], no tools | claude-sonnet-4-6 / anthropic | Same |
| backtest-risk-judge | Judge for task 3, sole holder of collector-log.txt and data-quality-policy.md | tools: [], no tools | claude-sonnet-4-6 / anthropic | Same |
The models in the table are the example repository's defaults; the actual choice is made per scenario at the design stage.
All five agents have an outputSchema, each defined with zod: two audits, one integrity report (decision takes one of three values, ACCEPT / QUARANTINE / REPAIR_REQUIRED), and the judge verdict { accept, critique }. Task 3 sets dependencyPayload: 'structured', so only the validated JSON from the two extraction tasks is injected into its prompt.
The verify config is quorum: 2, maxRounds: 2, onDissent: 'revise'. In consensus.ts the judges rule one after another, and the round passes once approvals reach the quorum; if a round falls short and rounds remain, the previous answer and all objections are combined into a revision prompt and handed back to task 3's agent to rewrite. Only an accepted revision replaces the task result.
judgePrompt is a function that returns instructions by judge name; the snapshot metadata, collector log and data quality rules are spliced only into the matching judge's prompt, so the report agent does not see them in the first round. Once judgePrompt is set, the framework stops applying its built-in mode instructions.
The report agent's afterRun hook records every version of the structured report, and the script uses it to print the decision path (for example ACCEPT -> QUARANTINE).
Result
Runnable output and how to verify it:
It needs an API key and Node.js 20 or later. The script header lists ANTHROPIC_API_KEY as a prerequisite, and the model can be overridden with the MODEL environment variable; run it with npx tsx packages/core/examples/cookbook/market-data-integrity-verify-loop.ts.
The console prints, in order, the decision path, each judge's ACCEPT / DISSENT and reasons for each round, and the final JSON report.
After the run it checks 9 assertions: the first version is ACCEPT, the last is QUARANTINE, both judges take part in the first round, the second round has two approvals, the report records the 900101–900104 update ID gap and the 09:29:59.940 to 09:30:00.210 disconnect window, it cites snapshot-metadata.json and collector-log.txt respectively, and it gives a remediation action of re-downloading or rebuilding the data. Any FAIL exits with code 1; when runTasks returns a failure, the script throws The market-data integrity workflow failed. and also exits with code 1.
The order book, trades, snapshot, collector log and quality rules in the fixtures directory are all labeled MOCK synthetic data; the script connects to no exchange and makes no market data requests. The output is model-generated report text, used to demonstrate and verify the workflow.
Sources
How this relates to YuanASI
The material on this page is a cookbook example from the official open-multi-agent repository, maintained by YuanASI.
If you run a real market or trade data ingestion process and want several agents to check snapshots, collector logs and quality rules separately, with a verdict that passes review before data enters backtesting or risk control, connected to your own data pipeline, the matching YuanASI service is Custom AI Agent Development →
Related services
Last updated
Want to know how a workflow like this would land on your side?