CHARS_PER_TOKEN

Constant CHARS_PER_TOKEN 

Source
const CHARS_PER_TOKEN: usize = 4;
Expand description

Approximate characters-per-token ratio.

This is a rough heuristic (4 chars ≈ 1 token) used for Phase 1. Future versions may use a proper tokenizer.