pub fn scale_token_limit_to_source(source_bytes: usize)Expand description
Scale the runaway-token backstop to the SOURCE size. The 400M baseline is
sized for arXiv-scale inputs; a book-scale source legitimately expands
past it (witness: a 131 MB flat-index compilation died at 400M
mid-digestion with no loop in sight, at a measured ~3+ tokens/byte and
paper-class documents measured up to ~160 tokens/byte). ×200/byte keeps
the ceiling finite — a true infinite loop still trips — while never
LOWERING the baseline for small documents. An explicit
LATEXML_TOKEN_LIMIT wins unchanged (including 0 = disabled).