Why character-level jailbreaks work: BPE fragments the safety words

What is this?

Anyone who has watched a chatbot refuse “how to build a bomb” but comply with “how to build a b0mb” has seen the effect. Character-level tricks — random capitalization, leetspeak, inserted spaces, scrambled letters — slip past safety alignment even though a human reads the request without difficulty. It has been an empirical curiosity for a couple of years. A paper posted to arXiv in July 2026 by researchers at Palo Alto Networks, Breaking Safety at the Token Boundary, turns the curiosity into a falsifiable mechanism and, more usefully for defenders, shows why the obvious fixes do not cleanly work.

The claim is structural, not model-specific: byte-pair encoding (BPE), the tokenizer under essentially every modern model, fragments safety-critical words into unfamiliar sub-word pieces, and the alignment data these models were trained on contains no intentionally fragmented examples. The model learned to comprehend messy text during pre-training, but it only ever learned to refuse clean text.

How it works

The authors frame it as a chain that can be broken at any link, and test each link independently across five model families (Qwen-3-4B, Qwen-2.5-7B, Gemma-3-4B, Llama-3.1-8B, Mistral-7B).

Start with tokenization. A single character change can reorganize the whole BPE token sequence. Inserting a space is a clean probe because spaces trigger pre-tokenization splitting before merges are looked up, so they are guaranteed to fragment a token while preserving every original character. On 50 HarmBench prompts, all five families flipped from a refusal-leaning to a compliance-leaning first token on at least 80% of the prompts they had originally refused (84%, 86%, 92%, 80%, and 100% respectively).

The next link is where in the model this matters. Refusal is mediated by a signal in the last roughly 30% of layers — consistent with prior findings that refusal is carried by a single residual-stream direction and that safety behavior is concentrated in the first few generated tokens. Activation patching localizes the disruption there, and targeted mutation isolates the safety words themselves as the point of failure rather than the surrounding prose.

The last link is behavior. An optimization aimed squarely at fragmenting safety tokens flipped the first-token refusal trigger on 80–100% of refused prompts, and 48% of those flips produced genuinely harmful completions (29–65% per model). The trigger flip alone overstates the risk — the authors are careful to separate the signal from the behavior — but roughly half of the time the flip is real. A scan of three public alignment datasets found zero intentionally fragmented prompts among 30,000 examples, which is the missing-distribution explanation for the whole effect.

Why it matters

This reframes a grab-bag of “obfuscation” jailbreaks — leetspeak, spacing, scrambling, Best-of-N sampling over perturbations — as symptoms of one root cause rather than a list of unrelated tricks. That matters for anyone relying on a model’s built-in refusals as a safety layer: the refusal is bound to the token shape of the harmful phrase, not to its meaning, so any transformation that changes the tokenization while preserving human readability tends to travel straight through. It is model-agnostic, requires no weights or gradients for the cheap versions, and applies equally to open and hosted models. The honest limits: the strongest results come from a white-box optimization, the effect is a probabilistic bypass rather than a guaranteed one, and it degrades a built-in guardrail rather than granting new system access on its own.

Defenses

The paper’s most valuable contribution for practitioners is showing that the two intuitive fixes are not free:

• Do not treat retraining as a solved fix. Across a 68-cell grid (55 trained checkpoints), no DPO configuration achieved stable closure of the attack across families. Supervised fine-tuning on fragmented prompts did close it on three of five families — but only by globally raising refusals, including on benign requests. The missing distribution is necessary but not sufficient; naive augmentation buys safety with over-refusal.
• Measure over-refusal explicitly. The authors introduce a paired benign diagnostic (Conv-Benign) precisely to catch fixes that “work” by making the model refuse everything. Any team hardening against this should track benign-refusal rate alongside attack success, not attack success alone.
• Normalize and inspect input tokenization. Because the tell is abnormal fragmentation, an input-side filter that counts characters-per-token can flag obfuscated prompts before they reach the model (see the Broken-Token line of work). Canonicalize spacing, casing, and homoglyphs prior to safety classification.
• Defend deeper than the first token. Since the refusal signal is concentrated in the last layers and first generated tokens, complement it with inference-time defenses (input smoothing, safety-aware decoding, latent adversarial training) and, over the longer term, tokenizer-robust or tokenizer-free architectures.
• Keep guardrails layered. Treat model-native refusals as one control among several — an independent output classifier and least-privilege tool scoping do not share the tokenization blind spot.

Status

Item	Detail
Finding	Character-level jailbreaks trace to BPE fragmentation of safety-critical tokens
Source	Breaking Safety at the Token Boundary, Palo Alto Networks, arXiv July 2026
Tested on	Qwen-3-4B, Qwen-2.5-7B, Gemma-3-4B, Llama-3.1-8B, Mistral-7B (+ Qwen-2.5-72B cross-scale)
Trigger flip	80–100% of refused HarmBench prompts; 48% produce harmful output
Localization	Last ~30% of layers; safety words isolated as disruption locus
Defense result	DPO: no stable closure; SFT-aug: closes 3/5 families but via global over-refusal
Type	Structural alignment weakness (no CVE; research finding)