the1169 ≠ ▁the262

real GPT-2 ids — " the" is so common it earned the smaller number

Prompts are whitespace-sensitive. A stray leading space silently changes the tokens the model sees.

02

1234567 → 12310163452231673134

real GPT-2 split — "1234567" breaks into 123 · 45 · 67

A real reason models fumble arithmetic: the digits never arrive as clean, aligned units.

03

English

~4 B/tok

Hindi / CJK

2–3× tokens

same meaning, more tokens — UTF-8 already costs 3 bytes/char

Non-English users pay more per message and get less context. The gap is baked into the tokenizer.

04

▁SolidGoldMagikarp43453 → untrained embedding

one real GPT-2 token, then almost never seen in training

The vector was created but never learned. Feed it in and the model hallucinates or breaks — a ghost of the merge step.