How matching works
English-only libraries miss mixed-script Maghrebi chat. Anti-Ttyah normalizes first, then looks up tokens and phrases.
raw token
→ strip tashkeel / tatweel
→ fold أ إ آ → ا and ة → ه
→ Franco-arabe digits (3 → ع/a, 7 → ح/h, 9 → ق/q, …)
→ collapse repeats (n*kkk → n*k)
→ drop article prefixes (el / al / ال)
→ fold separators (. * - _ ') when matchSeparators is on
→ lookup against the listed keysThe playground Explain trace shows the keys for each token.
What gets folded
| Step | Example idea |
|---|---|
| Tashkeel / tatweel | Marks on Arabic letters are ignored |
| Alef / teh marbuta | أ/إ/آ → ا, ة → ه |
| Digit-speak | 3 7 9 5 8 2 6 (and Latin 1 0) |
| Repeats | Long runs of the same letter collapse |
| Articles | el / al / ال prefixes |
| Separators | z.e.b.i and ze*bi count as one token (default) |
The full wordlist is not shown here.
Tokens vs phrases
- Terms are single tokens (
[\p{L}\p{N}]+, optionally joined by separators). - Phrases match consecutive tokens (
ولد الق*بة,nique ta mere).
inspect returns spans in the original string, so you can highlight or mask without guessing offsets. Hits from the default list also carry lang and category.
Categories
Default entries are tagged sexual, insult, slur, or mild. Pass categories to createFilter (or --categories on the CLI) to use only some buckets. Extra terms you add yourself are always included.
False positives
Keep the list conservative. If a token is also a product name or given name, prefer allowlist over deleting a useful insult. See Wordlist.