AIMaks Playgrounds

Tokenizer Arena

Models don't read words — they read tokens, and every tokenizer slices text differently. Paste anything below and compare the slices, the counts, and the bill — then click any token to watch the byte-pair merge cascade that assembled it, from raw UTF-8 bytes up. Runs entirely in your browser.

Text A272 / 10,000
Text B304 / 10,000
Tokenizers

GPT-4o / o-series

o200k_base · 200k vocab · 2024+loading…

GPT-4 Turbo (legacy)

cl100k_base · also GPT-4 / 3.5 · 2023loading…

GPT-2 (2019)

50k vocab · English-centric BPEloading…

Words

whitespace baselineloading…

Characters

code-point baselineloading…
Browse the course libraryAIMaks · Playgrounds