Claude 79f23f76eb
fix(ra2/token_gate): improve token estimation for code and multilingual content
Use ~3.3 chars/token base ratio (more conservative than the previous ~4)
and apply a non-ASCII density penalty that shifts toward ~2.5 chars/token
for CJK, emoji, or symbol-heavy text. Prevents underestimation that
could bypass the token gate on code-heavy or multilingual prompts.

https://claude.ai/code/session_01K7BWJY2gUoJi6dq91Yc7nx
2026-02-19 23:20:23 +00:00
..