On this page
Step 1: tier your content by riskStep 2: agree how to score errorsStep 3: design the sampleStep 4: let automation pick where to lookWhat never to sampleWhat to expectA company pays for a second linguist to review every word of its Korean help center: thousands of articles, updated weekly. The reviewers are thorough, the bill is large, and a Korean customer still finds the wrong price on the pricing page, which was translated in a rush outside the process. The QA budget went to the text that mattered least.
Quality assurance in translation is often all or nothing: full review of everything, or none at all once budgets tighten. There is a better middle. Risk-based QA decides how much checking each content type deserves. Sampling-based QA checks a well-chosen part of lower-risk content and uses the result to decide whether the rest needs more attention. Together they cut review cost and put more scrutiny where errors actually hurt.
Step 1: tier your content by risk
Ask two questions for each content type: if a sentence is wrong, how much harm does it do, and how many people see it? The answers sort content into tiers.
| Tier | Typical Korean content | QA method |
|---|---|---|
| Critical | Contracts, terms of service, privacy policy, regulated labels, safety instructions, prices and payment screens, patent claims | Full revision by a second qualified linguist, plus domain or legal review where relevant |
| High | Product UI, onboarding, website, marketing campaigns | Full revision, in context, on screenshots or devices |
| Medium | Help center, release notes, product descriptions at scale | Sampling with a score threshold; full review only of failed batches |
| Low | Internal documentation, support macros with low traffic, user-generated content | Automated checks plus light periodic sampling |
The critical tier is not negotiable. ISO 17100, the international standard for translation services, builds revision by a second person into its required process, and the critical tier is exactly where that second pair of eyes earns its cost. The savings come from the medium and low tiers, which usually make up most of the volume.
Step 2: agree how to score errors
Sampling only works if two reviewers would score the same sample the same way. That needs a shared error typology and severity scale. MQM (Multidimensional Quality Metrics) is an open framework widely used for this: it groups errors into categories such as terminology, accuracy, linguistic conventions, style, locale conventions, audience appropriateness, and design and markup, and rates them by severity. The worked example on the MQM site computes a simple score as one minus penalty points divided by word count, and treats any critical error as an automatic fail.
For Korean, add guidance on what counts as an error in your style sheet, for example:
Numbers and units
EN: 12 million usersKorean figure converted into the wrong ten-thousand unit, so the target says 120 millionMajor at least; critical on pricing or legal text. Large Korean numbers are grouped by ten thousand, which makes conversion errors common.Speech level drift
Style sheet: polite conversational style throughoutOne article switches to the formal written style halfwayMinor in a help article; would be major on an onboarding screen.Termbase not followed
Approved term for "workspace"A different Korean word for the same featureMajor when the term is a UI label users must find.Step 3: design the sample
- Choose the sample before looking. Pick segments or articles at random or at fixed intervals across the batch. Reviewing the first few pages, or the ones that look suspicious, biases the result.
- Size it so one segment cannot decide the result. A sample of a few sentences either passes or fails by luck. Start with a fixed word count per batch and adjust once you have a few months of scores.
- Stratify. If a batch mixes new translation, fuzzy matches and machine-translated segments, sample each group, because they fail in different ways.
- Set a threshold and an action. For example: pass, accept the batch; fail, review the whole batch and re-sample the next one at double size.
- Record scores per translator, engine and content type. Trends tell you where to tighten and where to relax.
Step 4: let automation pick where to look
Automated checks are cheap and should run on everything: placeholders, numbers, tags, length limits, glossary terms, untranslated text. Quality estimation goes further and scores segments. Phrase's Quality Performance Score, for example, can be used with thresholds to decide which segments skip or receive human editing. Use such scores to aim the human sample at low-scoring segments, as well as at a random share of high-scoring ones to check the scores themselves.
Do not treat a metric as the verdict. COMET and similar neural metrics are trained on human judgments and are useful for comparisons, but a WMT 2024 paper on pitfalls in using COMET documents how results can be misleading when the metric is applied carelessly. Keep the final decision with a human who reads Korean.
What never to sample
- Anything in the critical tier, however large the volume.
- The first deliverables from a new translator, vendor or engine: review fully until you have a baseline.
- Text going somewhere new: a new market, a new regulator, a new product line.
- Any batch where the source changed late and was patched in a hurry.
What to expect
We will not quote a universal saving, because it depends on how much of your volume sits in each tier. The method shows you your own figure: compare review hours and error scores for a quarter before and after tiering. The usual pattern is fewer hours on low-risk volume and more attention, often for the first time, on the small set of critical text. For the levels that sit behind each tier, see defining fit-for-purpose quality levels. For the post-editing side, see light versus full post-editing, and for review gates before a launch, pre-launch review gates and sign-off roles. In patents, where every claim is critical tier, see why QA is essential in patent translation.
For large document sets, tiering is easier when it is decided at the scoping stage; see how to scope and run large document translation programs.
Paying for review in the wrong places?
We tier your Korean content, set up MQM-style sampling and report scores you can act on. Talk to us →