Why do language models sometimes insert unexpected words from another language?

First identify what kind of event occurred

Not every non-English-looking string is a language failure. Classifying it first prevents both unnecessary alarm and ineffective fixes.

The table is a triage tool, not a claim that one symptom has one cause. A company name in Japanese is not evidence that the model has lost the English instruction. A paragraph that switches into Russian after a retrieved Russian document may be normal copying behavior, poor context design, or an attack. The raw text and request trace decide which explanation is credible.

Why multilingual models can mix languages

Training material and code-switching

Multilingual models learn statistical patterns from text in many languages. That text includes translations, bilingual websites, source citations, product documentation, foreign titles, names, quotations, transliteration, and genuine code-switching. Code-switching means alternating languages in a single conversation or text. It is ordinary language use in many communities, not necessarily model corruption.

Research on multilingual model pre-training explicitly studies code-switching as a source of cross-lingual alignment. A 2025 ACL study reports that code-switching in a pre-training corpus can contribute to multilingual capability, although the study does not establish the composition of any proprietary model's data. ACL code-switching study This explains why a model can recognize and produce mixed-language patterns. It does not imply that a mixed-language answer is correct for an English-only task.

Shared tokenization and representations

Models do not usually work from a separate full-word dictionary for each language. They convert text into tokens, often subword pieces, then operate on numerical representations. SentencePiece is an example of language-independent subword tokenization that can be trained directly on raw sentences. SentencePiece Large multilingual systems commonly share vocabulary and parameters across languages. XLM-R, for example, demonstrated multilingual pre-training at scale across 100 languages. XLM-R paper

Shared machinery is useful because it enables multilingual understanding and transfer. It also means that related spellings, translations, familiar phrases, and foreign names occupy one prediction space. Tokenization is not a language selector. A tokenizer can represent Arabic, Devanagari, Cyrillic, Latin, or CJK text, but it does not impose a rule that only one script may be emitted. The model and decoding process still choose the next token from its active vocabulary.

Context can make another language locally plausible

The next-token distribution is conditioned on everything the model receives. A user may ask in English but include a pasted Spanish quotation. A document retrieval system may return a Chinese source passage. A system instruction may request exact citations. A few nearby examples in another language can make a foreign word or copied phrase more probable at the point where the model needs to continue.

Named entities create a legitimate boundary case. A model should normally preserve “São Paulo,” “München,” “東京,” a book title, a standard name, or an exact legal quote when fidelity is required. A product team must decide whether that is permitted output, translated output with the original in parentheses, or an error. Without this rule, an evaluator may count correct source preservation and unwanted language drift as the same phenomenon.

Decoding can select an unintended token

At every step, a generative model assigns a score or probability to possible next tokens. Decoding is the method used to turn those probabilities into text. With greedy decoding, the most probable allowed token is chosen. With sampling, a token is drawn from a distribution, commonly after controls such as temperature, top-p, or top-k are applied. Hugging Face's generation documentation notes that multinomial sampling can select any token with nonzero probability, unlike greedy search. Transformers generation strategies

A simple hypothetical example

Suppose the current text is: “Choose a more concise phrase, or”. The model may assign these illustrative probabilities to the next word token:

With sampling, the Russian token has a chance of selection in this invented distribution. The preceding words and a first unexpected token also affect later probabilities, so one short switch can sometimes grow into a phrase. The numbers are only an illustration. They are not a measurement of a particular model, prompt, or provider.

Temperature changes how concentrated the distribution is. Lowering it usually makes high-probability tokens relatively more favored and reduces run-to-run variation. It cannot guarantee English because the top token may itself be non-English, because constraints might be overridden by copied context, or because some systems use non-sampling strategies, hidden defaults, tools, or post-processing. The temperature setting is a diversity control, not a language-policy enforcement mechanism.

Retrieval, prompts, and injection are separate causes to investigate

When an application uses retrieval-augmented generation, the model receives retrieved documents as additional context. A foreign-language source can be relevant and harmless, but it can affect style, terms, and copied wording. Poor retrieval can surface irrelevant foreign content. A document with incorrect or stale terminology can contaminate output even if there is no malicious intent.

Prompt injection is a different security concern. An attacker can place instructions in a user message, web page, file, ticket, email, or retrieved document to influence the model's behavior. OWASP warns that retrieval and fine-tuning do not eliminate prompt-injection risk, and explicitly calls out multilingual or obfuscated attacks. OWASP prompt injection guidance A sudden shift into another language can be an accidental copy, but it can also be a signal to inspect untrusted context rather than merely retrying the generation.

Use these controls for retrieval systems:

• Maintain source provenance, document owner, language, ingestion time, trust level, and access permissions for every retrieved chunk.
• Treat retrieved text as untrusted data, not as instructions. State that boundary in system design, but also enforce it with tool permissions, allowlists, output validation, and human approval for consequential actions.
• Filter or rank retrieved documents by the target language when the task requires it, while preserving an explicit path for approved foreign-language sources.
• Log which chunks reached the model so a suspect phrase can be traced to a source, not guessed at from the final answer alone.
• Test the pipeline with benign foreign-language documents and with approved security test cases before granting the model access to sensitive data or actions.

Do not assume that an English-only prompt neutralizes hostile or irrelevant retrieved text. The application must separate trusted instructions from untrusted data and must make the downstream action safe even if the text is misleading.

Check for encoding and display failures

If the raw API response is correct but a web page, PDF, chat client, spreadsheet, or downstream system shows different characters, the failure is outside generation. Compare the raw response body, the value after JSON parsing, the stored database value, and the final rendered text. Record UTF-8 decoding, HTTP headers where relevant, font selection, normalization, and any copy-paste or export step.

Look at code points, not just glyphs. Latin a and Cyrillic а can look almost identical while being different characters. Unicode Technical Standard 39 describes mixed-script and whole-script confusables and provides mechanisms for mixed-script detection. Unicode security mechanisms Unicode normalization is also important when comparing text because visually equivalent sequences can have different underlying representations. Unicode normalization forms

For a user-facing language constraint, test both language and script. Language identification can flag a paragraph that has drifted, and script detection can flag unexpected Cyrillic, Arabic, or Devanagari characters. The open-source fastText language-identification models recognize many languages, but a one-word detector result is weak evidence because names, short words, and borrowed terms are ambiguous. Use a tool such as fastText as one signal alongside a script allowlist and task-specific exceptions. fastText language identification

Practical output controls

Write an enforceable language rule

Vague instructions such as “answer in English” are better than nothing, but a production rule should say what counts as compliance. For example:

Write the answer in English using the Latin script. Preserve proper names, official product names, and direct quotations exactly. For any non-English quotation, provide an English translation and label the original. Do not introduce non-English words for style or emphasis.

This rule distinguishes permitted fidelity from forbidden drift. Tailor it to the task. A transliteration tool, language-learning product, international catalog, or scholarly citation workflow needs different exceptions from a customer-support reply.

Provide a terminology contract

Give the model an approved glossary for terms that must remain untranslated, terms that must be translated, preferred spellings, abbreviations, and source-language rules. Include a small number of positive examples if the distinction is hard. Do not paste an unbounded list of multilingual web text and expect an English-only instruction to dominate it.

For important content, attach the source material that the model may use and say what to do with non-English material. For example, “Use the French regulation as evidence, cite its original title, but write all explanatory prose in English.” This removes an ambiguity that otherwise forces the model to infer editorial policy.

Validate before publishing or executing

Validation should match the consequence of a violation. A low-risk drafting assistant might run a language detector over each paragraph, flag unexpected script characters, and ask the author to review. A production workflow can add a deterministic script allowlist, glossary check, sentence-level language identification, quote and named-entity exceptions, and a final human review for high-impact text.

Keep the validation result with the response record. If the system rejects output, store the reason, the text span, the detected script or language, and the context trace required for debugging. Do not silently remove characters, because that can destroy names, quotes, or evidence. Route ambiguous cases to review instead.

Retry with a constrained correction

When a validation failure is genuinely output drift, retry once with a concise correction that includes the failed output and the original requirements: “Rewrite this response in English and Latin script only. Preserve the meaning. Retain only the approved names and quotations listed below.” Reduce sampling variation if your platform exposes a temperature or equivalent control, but do not keep retrying indefinitely. Repeated failure is evidence that the prompt, retrieval set, model, or output policy needs a change.

If your serving stack supports hard output constraints, apply them carefully. A simple Latin-script allowlist may break legitimate accented English, mathematical notation, URLs, names, and quotations. A language test should reject only what the product policy actually forbids, with documented exceptions.

A diagnostic sequence for one incident

1. Preserve the raw model response and the exact rendered version. Compare code points to decide whether the unexpected text existed before display.
2. Mark the exact span and classify it as a borrowed term, named entity, quote, multiword language drift, garbling, or mixed-script confusable.
3. Save the complete request trace: system and user instructions, model and version, decoding settings, tools, retrieved chunks, locale, post-processors, and retry history. Remove or protect sensitive data in the incident record.
4. Search the prompt and retrieved chunks for the same word, language, script, formatting pattern, or an instruction that could have been copied.
5. Reproduce with the same request, then with sampling reduced or disabled where the platform allows it, and then without retrieval. These comparisons separate variation from context influence, but they still cannot reveal proprietary training details.
6. Run language and script validation on the response. Confirm that the validator permits the names, citations, and quotations the task requires.
7. Fix the smallest confirmed cause: clarify the language rule, supply a glossary, remove or relabel a retrieval source, correct a Unicode pipeline, or restrict the workflow. Then add the incident as a regression test.

Limits of this explanation

Outside observers usually cannot inspect the complete corpus, tokenizer, fine-tuning data, decoding parameters, safety layers, caching, model routing, tool calls, or post-processing of a proprietary system. Even if a word appears only once, it may result from context, stochastic sampling, a bug, an unseen document, or a display transformation. A general explanation should therefore name plausible mechanisms and propose tests, not diagnose a vendor's internal cause from a screenshot.

Language validation also has limits. Automatic language identification is less reliable for short spans, proper names, code, transliterated text, closely related languages, and deliberate code-switching. Script validation cannot judge meaning. Use it to catch policy violations and security-relevant confusables, then retain human review where a translation or a single word can materially change the decision.