The AI Translation Debugging Playbook: How to Diagnose and Fix Broken Localization Files After a Bad LLM Run¶

You ran your strings through an LLM, shipped the output, and something is wrong in production — a crash, a garbled UI, missing text, or a user reporting that the French version of your app is displaying German. You have 400 translated strings and no obvious starting point. This post gives you a repeatable, platform-specific playbook to find the damage, triage it by severity, and fix it before it compounds.

Think of this as the "what to do after the fire" companion to the prevention side of the problem. If you haven't read our coverage of 5 ways AI translation corrupts mobile resource files, that's the before-you-translate reading. This post picks up where that one leaves off. For a broader look at the failure modes that make one-shot LLM runs risky in the first place, /solutions/ai-translation-problems/ is worth a visit before you design your next workflow.

Step 0: Triage — Identify the Failure Class First¶

Before you run a single validation script, you need to know what kind of problem you're dealing with. AI-introduced localization bugs cluster into four distinct failure classes, and each one requires a different debugging path.

Structural failures are the loudest. The app crashes on launch, the resource compiler throws an error, or a locale fails to load entirely. On Android, you'll see Resources$NotFoundException. On iOS, the NSBundle call returns nil and your fallback language kicks in silently — which is almost worse, because it looks like the translation worked. On React Native, a malformed JSON file causes the i18n library to throw at initialization.

Placeholder failures show up at runtime when a string is actually used. On iOS you'll see NSInvalidArgumentException if the format specifier count doesn't match the arguments passed. On Android, java.util.MissingFormatArgumentException or a silent wrong-value substitution if the LLM dropped a positional index. These bugs are insidious because they only fire on the code paths that hit the affected string.

Plural failures produce wrong grammatical forms — always singular regardless of count, missing text for specific numeric values, or strings that silently fall through to the wrong CLDR category. Arabic and Russian are the most common victims because they have six and three plural forms respectively, and LLMs frequently collapse them to two. See our Android plurals guide and iOS stringsdict guide for the full taxonomy of what goes wrong here.

Semantic failures are the hardest class to detect. The file is structurally valid, the placeholders are intact, plurals have the right keys — but the translation means something subtly or embarrassingly wrong. No automated tool catches these. They surface through user reports, app store reviews, or a native speaker doing a spot-check.

Use this decision tree to route yourself:

Production symptom
│
├─ App crash / resource not found?
│   └─ → Step 1: Structural Validation
│
├─ Wrong value displayed / format exception?
│   └─ → Step 2: Placeholder Audit
│
├─ Wrong count form / missing text for N items?
│   └─ → Step 3: Plural Completeness Check
│
└─ "Something feels off" / user complaint about meaning?
    └─ → Step 4: Semantic Spot-Check

You may end up running all four steps. Start with the one the symptoms point to, resolve those issues, then continue down the list.

Step 1: Structural Validation (Platform-Specific)¶

This is the fastest step to automate and the one with the highest signal-to-noise ratio. Run these checks before you look at content.

Android strings.xml

Start with xmllint:

xmllint --noout res/values-fr/strings.xml

A non-zero exit code tells you the file is malformed. Common LLM-introduced breaks:

• Unescaped apostrophes. French text is full of them (l'application, c'est). XML requires \' inside unquoted string values, or the entire attribute value must be quoted. LLMs frequently output the raw apostrophe.
• Translated XML attribute names. An LLM asked to "translate the file" will sometimes translate name="app_name" to nom="app_name" or translate the key value itself. Your getString(R.string.app_name) call now finds nothing.
• Broken entity references. & becomes & (invalid XML), or & becomes & (double-escaped, renders literally as & in the UI).

After xmllint passes, use aapt to verify the compiled output:

aapt dump resources app/build/outputs/apk/debug/app-debug.apk | grep -A2 "string/app_name"

If a string is absent from the compiled APK, the resource compiler silently dropped it — often due to a duplicate key, which xmllint won't catch.

iOS .strings

plutil -lint Localizable.strings

Two failure modes that plutil catches and that LLMs introduce regularly:

• Encoding issues. plutil expects UTF-16 for traditional .strings files, but modern Xcode projects use UTF-8. An LLM asked to generate a .strings file sometimes outputs a UTF-16 BOM into what should be a UTF-8 file, or vice versa. Xcode's error message ("The data couldn't be read because it isn't in the correct format") is unhelpful.
• Duplicate keys. The LLM, especially when given a long context window, sometimes outputs a key twice with different values. plutil doesn't flag this — the later value wins silently. Run this to catch it:

awk -F'"' '/^"[^"]+" =/{print $2}' Localizable.strings | sort | uniq -d

Any output from that command is a duplicate key that needs manual resolution.

iOS .stringsdict

plutil -lint -s Localizable.stringsdict

Passing plist validation is necessary but not sufficient. An LLM can produce a structurally valid plist that has the wrong schema for NSStringPluralRuleType. Specifically, check that each plural rule entry has the required NSStringFormatSpecTypeKey, NSStringFormatValueTypeKey, and at minimum the other category populated. Missing the NSStringFormatSpecTypeKey key entirely produces a file that compiles but returns empty strings at runtime.

React Native JSON

node -e "JSON.parse(require('fs').readFileSync('locales/fr.json', 'utf8')); console.log('valid');"

Beyond parse validity, check that your key structure matches what your i18n library expects. Libraries like i18next support nested keys ("auth.login.button"); react-intl typically expects flat namespaced keys. An LLM that was given a flat file sometimes returns a nested structure, or the reverse. This won't cause a JSON parse error — it causes silent lookup misses at runtime.

Step 2: Placeholder Audit¶

Placeholder corruption is the second-most-common LLM failure after structural breaks, and it's almost impossible to catch by reading the file manually at scale. You need a script.

The core logic: for each string, extract the set of placeholder tokens from the source and the set from the translation, then diff them. Any addition, removal, or reordering is a bug.

Here's a Python script that covers the main format families:

import re
import json
import sys

# Adjust patterns for your platform
PLACEHOLDER_PATTERNS = [
    r'%\d+\$[sd@fg]',   # positional: %1$s, %2$d (Android/iOS)
    r'%[@sdifg]',        # simple: %s, %@, %d (iOS/Android)
    r'\{[a-zA-Z_][a-zA-Z0-9_]*\}',  # ICU/named: {name}, {count}
    r'%\([a-zA-Z_]+\)[sd]',         # Python-style named: %(key)s
]

def extract_placeholders(text):
    found = []
    for pattern in PLACEHOLDER_PATTERNS:
        found.extend(re.findall(pattern, text))
    return found

def audit_placeholders(source_path, translated_path):
    with open(source_path) as f:
        source = json.load(f)
    with open(translated_path) as f:
        translated = json.load(f)

    issues = []
    for key, source_value in source.items():
        if key not in translated:
            issues.append(f"MISSING KEY: {key}")
            continue
        source_tokens = extract_placeholders(source_value)
        translated_tokens = extract_placeholders(translated[key])
        if source_tokens != translated_tokens:
            issues.append(
                f"PLACEHOLDER MISMATCH [{key}]:\n"
                f"  Source:     {source_tokens}\n"
                f"  Translated: {translated_tokens}"
            )
    return issues

if __name__ == "__main__":
    issues = audit_placeholders(sys.argv[1], sys.argv[2])
    for issue in issues:
        print(issue)
    sys.exit(1 if issues else 0)

Run it as: python audit_placeholders.py locales/en.json locales/fr.json

The exit code makes it CI-friendly. Wire this into your pipeline and it becomes impossible to merge a translated file with broken placeholders. Our post on build-time locale validation covers the CI integration side in more detail.

Common LLM placeholder mutations to search for specifically:

• %s → % s — a space inserted between % and the specifier. The string renders as literal % s in the UI.
• %1$d → %d — the positional index is dropped. On Android this means argument reordering breaks; on iOS it may cause an argument count mismatch crash.
• {name} → { name } — spaces added inside the braces. Many i18n libraries do exact-match lookups on the token, so this silently produces the unreplaced literal { name } in output.
• %@ → %s — iOS format specifier replaced with C-style. This is silent in most cases but will cause NSInvalidArgumentException if the argument is actually a Swift String bridged through specific code paths.

For deeper background on why placeholders are such a common casualty of AI translation, this post on placeholder safety covers the root causes.

Step 3: Plural Completeness Check¶

This step requires knowing what CLDR requires for each target locale, which most engineers don't have memorized. The authoritative source is the CLDR Plural Rules table. Bookmark it.

Android strings.xml plurals

For each <plurals> block, list the <item quantity="..."> entries and compare against the CLDR-required categories for the locale. Example: Russian requires one, few, many, and other. An LLM will frequently produce only one and other, leaving few and many to fall through to other — which is grammatically wrong for most counts.

<!-- What you might get (wrong for Russian) -->
<plurals name="item_count">
    <item quantity="one">%d элемент</item>
    <item quantity="other">%d элементов</item>
</plurals>

<!-- What Russian actually requires -->
<plurals name="item_count">
    <item quantity="one">%d элемент</item>     <!-- 1, 21, 31... -->
    <item quantity="few">%d элемента</item>    <!-- 2-4, 22-24... -->
    <item quantity="many">%d элементов</item>  <!-- 5-20, 25-30... -->
    <item quantity="other">%d элемента</item>  <!-- fractions -->
</plurals>

iOS .stringsdict

Check that the dict for each localized plural string contains all CLDR categories for the target language. The missing category issue surfaces identically to Android — the system falls back to other silently.

React Native / ICU MessageFormat

ICU-format plural expressions inline all categories in a single string:

{
  "item_count": "{count, plural, one {# item} other {# items}}"
}

For Arabic (ar), the correct form requires six categories: zero, one, two, few, many, other. An LLM almost never produces all six without explicit prompting — and even when prompted, it frequently gets the numeric ranges wrong.

Run a check that parses each ICU string and validates that the set of plural categories matches CLDR requirements for the file's locale. A red flag heuristic: any locale that isn't English-family (en, de, nl, etc.) should raise suspicion if it has only one and other in its plural expressions.

Step 4: Semantic Spot-Check (The Human Step)¶

This is where things get interesting — because there is no tool that reliably catches a translation that is confidently, fluently, completely wrong. Automated validation tells you the file is well-formed and the placeholders are intact. It says nothing about whether the text means what you think it means.

Prioritize ruthlessly. You can't manually review 400 strings per locale. Focus on:

• Onboarding strings — first impressions matter, and LLMs frequently over-formalize or under-formalize the register depending on the target language.
• Error messages — these need to be clear and actionable. A bad translation here increases support volume.
• Paywall and subscription strings — a mistranslation of "free trial" or "cancel anytime" can create legal exposure in some jurisdictions. Any string that mentions price, billing, or terms goes to a human reviewer, full stop.
• Any string with a placeholder — context collapse is highest risk here. The LLM translated "Welcome back, {name}!" without knowing that name would be a first name in this context. The result might be grammatically correct for a different kind of noun entirely.

Back-translation as a quick signal. Take the LLM-translated string, paste it back into the LLM, and ask for an English translation. This isn't a reliable accuracy test, but it's an efficient way to surface obvious semantic drift. If the back-translation doesn't resemble the original English source, you have a problem worth escalating.

Source (en):   "Your subscription renews on {date}."
LLM output (fr): "Votre abonnement se renouvelle le {date}."
Back-translation: "Your subscription renews on {date}."  ✓

Source (en):   "Tap to dismiss"
LLM output (ar): "اضغط للرفض"  (literal: "Tap to reject/refuse")
Back-translation: "Tap to refuse"  ← flag this for review

Escalation rules: any string that touches money, legal terms, safety instructions, or medical information gets reviewed by a native speaker with domain knowledge before it ships. This is not optional. The cost of a professional review for a slice of high-risk strings is small compared to the cost of a regulatory complaint or a viral screenshot of a mistranslation.

Step 5: Lock It Down for Next Time¶

You've found the bugs and fixed them. Now close the loop so this debugging session doesn't happen again.

Wire the validation scripts into CI. Steps 1 through 3 are fully automatable. The structural checks (xmllint, plutil, JSON parse), the placeholder audit script, and a CLDR-aware plural completeness checker should all run on every PR that touches a localization file. A CI failure blocks the merge. This is the minimum viable localization quality gate.

If you're building this from scratch, our build-time locale validation guide has a complete breakdown of the checks worth running at each stage of the pipeline.

Establish a translation memory baseline. The output you've just validated and fixed is now your known-good baseline. Store it as a translation memory so that future AI translation runs are diffed against it. Strings that already have a validated translation don't need to go through the LLM again — and strings that do go through the LLM can be compared against the TM for consistency on shared terminology.

This is the structural difference between a one-shot LLM run and a localization pipeline. GetTranslated.AI's translation memory workflow is built around this pattern — validated strings anchor future runs and reduce the surface area for new bugs.

Document the failure modes you found. Add a LOCALIZATION.md to your repo (or a section in your existing docs) that lists the specific mutations you found in this incident: the placeholder format that got corrupted, the plural categories that were dropped, the encoding issue that appeared in one locale. This becomes the test case list for your validation scripts and the reference for whoever runs translations next.

For a comprehensive view of what a safe AI translation workflow looks like end-to-end, /solutions/safe-ai-translation/ covers the pipeline design.

Conclusion¶

Here's the five-step playbook in brief:

1. Triage — identify the failure class (structural, placeholder, plural, semantic) before you do anything else.
2. Structural validation — xmllint, plutil, JSON parse, and resource compilation checks catch file-level breaks fast.
3. Placeholder audit — script a token diff between source and translated strings; any mismatch is a bug.
4. Plural completeness — validate against CLDR requirements for every target locale; two-category plurals in Russian or Arabic are almost certainly wrong.
5. Semantic spot-check — prioritize high-stakes strings, use back-translation as a quick signal, escalate money and legal strings to human review.

The real lesson here isn't that LLMs are bad at translation. It's that one-shot LLM runs produce bugs that require a manual debugging session to clean up, and the cost of that session scales with the number of locales and strings. The fix isn't a better prompt — it's a pipeline with validation baked in from the start.

If you want to go deeper on the plural-specific failure modes that show up in AI translation runs, our posts on Android plurals and iOS stringsdict and React Native pluralization cover each platform in detail. And if you're thinking about what to do before you run the LLM rather than after, the before-you-translate checklist post covers the prevention side of this problem.

Build the pipeline. Validate on merge. Keep the known-good baseline. That's how you stop running this playbook every release cycle.