pub fn decode_input_bytes(raw: &[u8]) -> StringExpand description
Decode raw input bytes as text when no encoding has been declared: valid UTF-8 is taken as-is, anything else is a Latin-1 passthrough (byte → char).
Mirrors Perl Mouth.pm L75-80: when PERL_INPUT_ENCODING is undef Perl
never decodes, so the bytes pass through untouched and the read cannot
fail. The point is that a non-UTF-8 file is never lost — only decoded
conservatively. std::fs::read_to_string gives the opposite behaviour
(hard error on the first stray byte), which silently cost witness
2605.00490 its entire bibliography: a JabRef-written .bib self-declaring
% Encoding: Cp1252. Real bibtex 0.99d is 8-bit clean and reads it fine.
Latin-1 (rather than from_utf8_lossy) is the better fallback here
because it is lossless byte → char: legacy .bib files are overwhelmingly
Latin-1/Cp1252, whose accented names survive intact instead of collapsing
to U+FFFD.
The fallback is applied per line, not per buffer. raw is a single
line when the Mouth calls this, but a whole file when a .bib reader does,
and decoding a whole file as Latin-1 because of one stray byte would
mojibake every correctly-UTF-8-encoded name in it (é → é). Per-line
keeps the damage to the offending line and matches the Mouth’s own
granularity. The all-valid-UTF-8 case (the overwhelming majority) still
costs exactly one from_utf8 SIMD validation of the whole buffer.