Translation QA: cross-model back-translation and semantic drift grading

Scenario

Typical scenario: technical docs, product descriptions or contract clauses need versions in several languages, and the reviewers often do not read the target language, so all they can judge is whether the translation reads smoothly. Back-translation is a standard check in the localization industry: translate the text back into the source language and compare it with the original, and paragraphs whose meaning shifted stand out. If the same model that translated also back-translates, it tends to reuse its own phrasing, and the drift stays hidden.

This example splits translation, back-translation and comparison across three agents, with the back-translation handed to a different model provider. Each step produces structured JSON against a Zod schema, the next step reads the previous step's fields directly, and the result is a per-paragraph drift table that can feed later human review or an automated workflow.

How it works

The models in the table are the example repository's defaults; the actual choice is made per scenario at the design stage.

The three agents sit in new AgentPool(1) and the script calls pool.run() for each in turn, splicing the previous step's structured result into the next prompt; all three use temperature: 0 and maxTurns: 1. The back-translator is chosen in code: native Gemini when GEMINI_API_KEY is present, otherwise OpenAI, which also reads OPENAI_BASE_URL.

Each agent has a Zod schema as its outputSchema: a translation batch, a back-translation batch, and a drift table (each row holds original, translation, backtranslation, drift_severity, notes). In agent.ts the framework extracts the JSON and then validates it; on a first schema failure it feeds the validation error back to the model and tries once more, and if that still fails the step returns a failure.

The script registers the built-in tools with buildAgent(), but none of the three agents declares tools or toolPreset. OMA's tool grants deny by default (src/tool/grants.ts: with no positive grant, it resolves to zero built-in tools), so this pipeline only makes model calls. The source text is four English paragraphs inlined in the script, split on blank lines into numbered paragraphs.

Result

Runnable output and how to verify it:

It needs API keys from two providers. The script header lists ANTHROPIC_API_KEY as a prerequisite, plus one of GEMINI_API_KEY or OPENAI_API_KEY; run it with npx tsx packages/core/examples/cookbook/translation-backtranslation.ts.

Missing keys mean a clean skip. When the conditions are not met it prints [skip] This example needs ANTHROPIC_API_KEY plus GEMINI_API_KEY or OPENAI_API_KEY. and exits with code 0.

At runtime it first prints the paragraph count and the provider and model on each side, then the progress of the three steps as [1/3], [2/3], [3/3]. If any step fails or returns no structured result, it prints Agent A / B / C failed: with the model's raw output and exits with code 1.

The final output is the drift table. On success it prints a JSON array under FINAL DRIFT TABLE, one row per paragraph, with the original, translation, back-translation, a drift level of none / minor / major, and a note; it then lists input / output token counts per agent and in total, and finally prints "Done."

The source text is four short English technical passages inlined in the script (CI/CD, observability, schema validation, cross-model checking), written for the demo; the translation, back-translation and drift verdicts are model-generated text, used to demonstrate and verify the workflow.

Sources

How this relates to YuanASI

The material on this page is a cookbook example from the official open-multi-agent repository, maintained by YuanASI.

If you run a real multilingual translation or localization review process and want models from different providers to cross-check translations, route drifted paragraphs to human review, and connect it to your own termbase and publishing system, the matching YuanASI service is Custom AI Agent Development →