Expand description
boundary-safe forward cursor over &str — the shared answer to the
byte-index scanners inherited from Perl’s character-oriented string ops
A forward cursor over &str whose position is always a char boundary.
§Why this exists
&str carries exactly one invariant the compiler enforces — valid UTF-8.
A usize used to index it carries none. So every &s[a..b] is an
unchecked assertion that both ends are char boundaries, and when the
assertion is wrong the program does not return an error, it panics.
That combination is unusually hostile here. This is a Perl port, and Perl
strings are sequences of characters: pos, substr and \G have no
boundary concept at all. Every hand-rolled as_bytes() + i += 1 scanner
translated from Perl therefore introduces an invariant the original never
had, silently, with no type-level trace — and passes every ASCII fixture
forever. Witness 2605.22125: a .bib title containing \“ aborted the whole
document, and the code had been live for months.
§The rule the bug teaches
The panic happens at the slice, but the defect is always in the advance:
| advance | safe? |
|---|---|
| scan to an ASCII delimiter | always — an ASCII byte is never a UTF-8 continuation byte |
by char::len_utf8() | always |
| a fixed count past an unclassified byte | never |
So the fix is not to check slices, it is to remove the ability to advance
wrongly. This cursor exposes no byte-count advance at all, which makes
slice_from infallible and the whole class
unrepresentable in code written against it.
Rust guidelines anti-index-over-iter / perf-iter-over-index: prefer the
iterator std already provides (char_indices) over manual indexing. This is
a thin, self-documenting wrapper over exactly that — not a new abstraction.
§Cost
A Peekable<CharIndices> and the source reference; no allocation, one pass,
the same traversal a byte walker made. char_indices also decodes each
character once instead of re-decoding at every slice.
Structs§
- Char
Cursor - Forward cursor over a
&str, positioned only ever at char boundaries.