Skip to main content

scale_token_limit_to_source

Function scale_token_limit_to_source 

Source
pub fn scale_token_limit_to_source(source_bytes: usize)
Expand description

Scale the runaway-token backstop to the SOURCE size. The 400M baseline is sized for arXiv-scale inputs; a book-scale source legitimately expands past it (witness: a 131 MB flat-index compilation died at 400M mid-digestion with no loop in sight, at a measured ~3+ tokens/byte and paper-class documents measured up to ~160 tokens/byte). ×200/byte keeps the ceiling finite — a true infinite loop still trips — while never LOWERING the baseline for small documents. An explicit LATEXML_TOKEN_LIMIT wins unchanged (including 0 = disabled).