WMT 2026 results: what human evaluation means for localization

WMT 2026 gives localization teams a useful starting point for comparing AI translation. Key insights for us: use the rankings to build a shortlist, then test the model together with your terminology, context, technical requirements, and review process.

For this article, we reviewed the official human evaluation from WMT, the Conference on Machine Translation, alongside studies of terminology and translation failures. We focus on what those findings mean for product UI and marketing copy, where a fluent sentence can still change a warning, break a placeholder, or overstate a claim.

Key takeaways for localization teams

• Start with your language directions. Use the results to select candidates, then compare them on your content.

• Make terminology useful. Select glossary entries by concept, context, and locale.

• Test context and revision. Measure whether extra metadata or another pass fixes errors and whether it introduces new ones.

• Review UI and marketing separately. Check technical constraints alongside meaning, and give brand voice its own criteria.

What was tested at WMT 2026?

The General Machine Translation task evaluated 46 systems across 23 translation directions. The study includes six domains across its test sets: news, social conversations, speech, software, education, and fact-checking. Domain coverage differs by source language.

Systems worked with documents and explicit instructions. In software content, that included preserving JSON keys and placeholders. Meaning and instruction compliance both mattered.¹

This makes the benchmark relevant to real localization work. A usable translation must preserve its purpose and fit the place where it will appear.

How did WMT assess translation quality?

WMT26 used contrastive Error Span Annotation, or “cESA”. Reviewers saw three translations together with the source and document context. They marked minor and major errors, then assigned a separate quality score to each translation on a 0-100 scale. This produces more detail than a preference vote alone.²

Review effort was also allocated dynamically, with stronger systems receiving more evaluation. This helps concentrate a limited budget on separating leading systems. It means exact positions further down the ranking deserve care.³

For an internal evaluation, the useful lesson is to compare outputs on the same sources, hide their identities, and keep the intended use visible to reviewers.

What do the rankings actually tell us?

The leading broad-coverage systems illustrate two ways to summarize performance:

Wayfinder had the widest top-cluster coverage, while Gemini had the best average rank among systems entering all 23 directions. Lumen reached the constrained-track top cluster in 20 directions. Top-cluster membership does not mean an outright win over every alternative, and specialists entering fewer directions are not directly comparable through average rank alone.⁴

Use these results to build a shortlist for your language directions. But rankings alone don’t tell you how well a model handles your product’s terminology or brand voice. That’s where additional research becomes useful, starting with how AI translation systems use glossaries.

What do the terminology results tell us about glossaries?

The separate terminology task compared document translation with explicit dictionaries against translation using example bitext from which systems had to infer terminology. Across submitted systems, supplied dictionaries increased term success by 26.4 percentage points on average. Example bitext produced a smaller terminology gain and a larger chrF++ quality gain. These are task-specific results across different tracks, rather than a universal comparison at equal resource budgets.⁵

The product lesson includes how terminology reaches the model. Which terms are relevant to this document? Which meaning of a source word applies? Is the target entry a fixed product name or a term that needs grammatical inflection?

CUNI’s experiments provide a useful example. Filtering a dictionary to the current document improved judged terminology coverage. An additional revision pass supplied only with alleged errors could change terms that were already correct. Its final submission used a filtered glossary and one-pass translation. These development comparisons relied largely on an automatic judge, so they motivate local testing rather than guarantee an improvement.⁶

For localization teams, a practical glossary should communicate concept, scope, locale, and preservation policy. Exact matching suits a locked brand string. Ordinary terms also need context and grammar: a correctly inflected translation should not fail simply because it differs from a dictionary’s base form.⁷

Glossaries help models choose the right words, but terminology is only one part of the context a translation needs. Information about the interface, audience, or intended meaning can also influence the result. The question is whether providing more context always leads to better translations.

Does more context always improve translation?

Context can tell a translator whether “Open” is a button action or a status. It can explain who is speaking, what a product feature does, or how a campaign should sound.

It can also be overinterpreted. In KUEST’s English-to-Korean game-localization study, added relationship metadata lowered automatic persona scores for 19 of 27 systems, though the overall mean was unchanged. The analysis found overapplication of social-status and honorific cues. This is a specific result about the tested models and setup, but it challenges the assumption that additional metadata reliably improves tone.⁸

For a translation brief, define what context is allowed to influence. A screen description can clarify a label; it should not add an unrelated feature claim. A confident brand voice can shape expression; it should not turn “may help” into “will solve.”

Compare useful context with a version that omits it. You can also add irrelevant information to test whether the system uses context selectively. What counts as useful context, however, depends on the content you’re translating. A software interface has different requirements from a marketing campaign.

What should you check in UI strings and marketing copy?

Both need accurate meaning. Their additional requirements differ.

Consider two original examples. In a UI string, You may cancel after {date} must retain its permission, time condition, and placeholder. In marketing copy, Save up to 20% on annual plans must retain both the maximum discount and its scope. Fluency cannot compensate for changing those meanings.

Score the two streams separately. Otherwise, a gain on routine interface text can conceal a loss in expressive copy, or an attractive rewrite can conceal an unsupported claim.

Even with clear requirements, the first translation may contain errors. Some AI translation systems address this with an additional revision step. But does another pass actually improve quality, or can it introduce new mistakes?

When does an extra revision pass help?

Some leading systems combine several candidate translations with judging and repair. Wayfinder, for example, uses candidate selection or rewriting followed by conservative checks for severe defects, including truncation, repetition, and broken structure. Its repair stage deliberately restricts broad stylistic changes.⁹

A useful first experiment is therefore a strong single pass followed by targeted correction. Compare it with the same workflow without repair. Count successful fixes and new errors, and include the added latency and cost.

Automatic judging can help identify problems, but it also needs evaluation. The WMT26 quality-evaluation task assesses error spans, quality scores, and error-free classification as separate targets. Good agreement with a system ranking is not the same as reliably recognizing every clean translation.¹⁰

Use code for deterministic checks such as schema validity and protected tokens. Use appropriately calibrated linguistic evaluation for meaning and style. Keep human review where the content and measured reliability call for it.

Together, these findings show that translation quality depends on more than the model itself. Terminology, context, technical checks, and revision all play a role. The next step is to test how these elements work together in your own localization workflow.

How to apply these findings to your localization workflow

The findings give your team concrete hypotheses for improving translation briefs and evaluation. Validate each change against your existing workflow before claiming a quality gain.

Start with the existing prompt as a baseline, then compare:

• A clearer brief: specify the locale, content type, and intended function using available inputs.

• Relevant terminology: compare no glossary, a full glossary, and a document-specific selection.

• Controlled context: compare missing, useful, and distracting information.

• Selective repair: correct supported defects and measure regressions against a single pass.

Keep representative content and deliberately difficult cases separate. Reserve a held-out set by document, feature, or campaign. Review candidate translations without revealing their model or prompt identity, and report meaning errors, technical failures, terminology, voice, and correction effort separately.

In LingoHub, organize approved terminology in your glossary, record tone and locale rules in your style guide, and use quality checks as part of your review workflow. Assess the complete setup against the requirements of the content being translated.

Choose one important language direction and a small, varied set of UI and marketing examples. Compare the complete workflow, including what reviewers still need to fix.

Ready to put that evaluation into practice? Book a LingoHub demo to discuss how terminology, style guidance, and review can fit your localization workflow.

Sources

¹ General MT findings

² cESA protocol

³ Dynamic evaluation method

⁴ General MT findings, Figure 3 and Section 6.4

⁵ Terminology findings

⁶ CUNI terminology system

⁷ TermJudge (Related research)

⁸ KUEST

⁹ Wayfinder

¹⁰ Evaluation findings