How this started
While casually messing around with a few chatbots, I ran into cases where simply repeating a single character thousands of times caused GPT and Gemini to behave strangely. No special prompt engineering, no code — just the same character pasted a few thousand times.
- Gemini: On the first attempt it responded normally. On the second attempt, the same kind of input caused it to degrade into a meaningless string of repeated exclamation-like characters.
- GPT: Collapsed immediately on the repeated input, and the output contained what looked like leaked internal scratchpad/reasoning text that should never reach the user.
- In another case, Gemini's collapse produced a long, oddly specific passage that shifted from Japanese into Korean mid-output — reading like an excerpt from someone's personal blog (career change, burnout, working as a data labeler, etc.), rather than anything resembling gibberish.
This felt like more than "funny bug" territory. Digging in, it turns out this belongs to a known family of phenomena that's actively studied.
Prior work: repetition → divergence → training data leakage
Nasr et al. (2023, "Scalable extraction of training data from (production) language models") demonstrated at scale that repetition prompts can cause LLMs to emit memorized fragments of their training data.
A more recent paper, "Retracing the Past: LLMs Emit Training Data When They Get Lost" (arXiv:2511.05518, 2025), digs deeper into the mechanism:
The emission of actual memorized text during divergence is preceded by a sustained, significant spike in the model's token-level prediction entropy.
In other words, repetition itself isn't the point — what matters is whether the model can be driven into a sustained state of high uncertainty about what comes next. The paper builds on this by using gradient-based optimization (GCG) to craft input snippets that deliberately maximize this entropy, calling the approach Confusion-Inducing Attacks (CIA).
Notably, this work is evaluated only on open-weight models (LLAMA family, OLMo). The authors explicitly note that extending this to black-box systems like ChatGPT and Gemini is future work. So the fact that a naive, unoptimized repetition of ordinary characters produced similar behavior on commercial black-box models is, in a sense, an accidental real-world instance of exactly the gap the paper leaves open.
Why might this be easier to trigger in Japanese or Korean? (hypothesis)
Everything from here on is unverified — just a hypothesis based on having built tokenizers myself.
1. Cross-lingual tokenizer disparity
This is a well-established research area. Petrov et al. (2023, "Language Model Tokenizers Introduce Unfairness Between Languages") quantified up to a 15x difference in the number of tokens needed to encode equivalent content across languages. Non-Latin scripts like Japanese and Korean are structurally disadvantaged under frequency-based BPE vocabularies.
More recently (2026), there's emerging work suggesting this disparity isn't just an efficiency/cost issue but may also affect the transfer of safety behavior — "Sycophancy as a Multilingual Alignment Failure" hypothesizes that tokenizer fertility mediates how well RLHF-induced safe behaviors transfer to a given language.
2. Unequal exposure to "meaningless string" as a genre
In English-heavy training corpora, high-entropy, semantically empty strings — API keys, hashes, UUIDs — appear constantly and for legitimate reasons (code, logs, technical docs). This may give models some degree of familiarity with this class of input.
Japanese and Korean have essentially no equivalent genre at this scale. Repeated-character emphasis exists ("あああああ"), but rarely beyond a dozen or so characters, and extended runs of a mora like っ (sokuon/glottal stop marker) have almost no legitimate use case at all. That may mean the model is encountering something it genuinely never saw during training.
3. This mitigation idea breaks down for logographic scripts
The natural fix — deliberately injecting "meaningless repeated strings" into training data for underrepresented languages — works reasonably well for phonetic scripts (kana, Latin alphabet) but doesn't translate cleanly to logographic characters like Kanji/Hanzi. Each character carries independent meaning, so it's hard to construct a string that's unambiguously "meaningless." Chinese in particular has no phonetic-only fallback script, so this whole mitigation strategy may need a different design for logographic languages entirely.
4. Syntactically ambiguous repetition ("buffalo buffalo" territory)
English has the famous "Buffalo buffalo buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo" sentence — grammatically valid despite looking like pure repetition. Japanese has similar constructions (e.g., built around the multiple meanings of "ちゃう" — dialectal "different," the verb-ending "~ちゃう," and the dog breed "チャウチャウ" sharing the same sound).
What makes this class of input tricky is its double nature: it looks like noise on the surface but is actually a legitimate, deeply nested grammatical structure. A model may struggle not just with predicting the next token, but with the more basic judgment of "should I even try to parse this seriously?"
Crucially, "buffalo buffalo" is a famous linguistics reference — it's likely discussed and explained plenty in training data. A regional or niche wordplay construction with the same structural ambiguity, but zero meta-level discussion in the corpus, seems like it would be far riskier. The relevant variable might not be "which language" so much as how well-documented a given linguistic phenomenon is within the training data.
Why this might resist post-training fixes
Tokenizers are fixed before pretraining and aren't changed during post-training (RLHF, SFT, etc.). If the root cause really is tokenizer/data-driven disparity in language exposure, then post-training can teach a model to behave better when it encounters repetition (e.g., refuse or deflect), but it likely can't fix the underlying instability in next-token prediction itself — that's baked in before alignment ever starts.
This seems consistent with the paper's own results: naive repetition attacks (RA) are suppressed to near-0% success on aligned models, but the authors' more sophisticated attack (CIA + mismatched SFT) recovers leakage rates of several percent to over ten percent even against aligned models. That suggests post-training closes off the obvious, well-known attack patterns while leaving the deeper entropy-based mechanism largely intact.
Possible mitigations, by layer
- Data mixture / tokenizer design — the most fundamental fix, but extremely costly (effectively requires redoing pretraining). Realistic only for next-generation models.
- Deliberate noise injection during post-training — similar to training on buggy code to improve bug-fixing ability. Likely only generalizes to patterns explicitly included in training, not to novel wordplay.
- Inference-time entropy monitoring — detect sustained prediction instability during generation and cut it off. Language-agnostic in principle, but running this at full strength on every request may not be cost-effective.
- Cheap input filtering + a specialized lightweight detector — filter the obvious cases cheaply, and route only the ones that slip through to a more expensive, specialized check. Probably the more realistic middle ground.
Summary
- Repetition-induced collapse and memorization leakage is a known research area going back to 2023.
- As of July 2026, I couldn't find work that explicitly connects cross-lingual tokenizer disparity with repetition-based memorization leakage — this specific intersection seems to still be open.
- From a tokenizer-building background, several compounding factors seem plausible: uneven training exposure to "meaningless strings" across languages, tokenizer fertility gaps, the specific difficulty of logographic scripts, and the role of a linguistic phenomenon's "fame" within the training corpus.
- All of this is unverified. Writing it down in case it's useful to someone who wants to actually test it.
References
- Nasr et al., "Scalable extraction of training data from aligned, production language models" (ICLR 2025) — arXiv:2311.17035
- Ko et al., "Retracing the Past: LLMs Emit Training Data When They Get Lost" — arXiv:2511.05518
- Petrov et al., "Language Model Tokenizers Introduce Unfairness Between Languages" (NeurIPS 2023) — arXiv:2305.15425
- Yang et al., "Problematic Tokens: Tokenizer Bias in Large Language Models" — arXiv:2406.11214
- "Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization" — arXiv:2405.17067
- "Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models" — arXiv:2606.08451
This is a personal record of observations and hypotheses, not peer-reviewed research. Corrections welcome.