Skip to main content

Module char_cursor

Module char_cursor 

Source
Expand description

boundary-safe forward cursor over &str — the shared answer to the byte-index scanners inherited from Perl’s character-oriented string ops A forward cursor over &str whose position is always a char boundary.

§Why this exists

&str carries exactly one invariant the compiler enforces — valid UTF-8. A usize used to index it carries none. So every &s[a..b] is an unchecked assertion that both ends are char boundaries, and when the assertion is wrong the program does not return an error, it panics.

That combination is unusually hostile here. This is a Perl port, and Perl strings are sequences of characters: pos, substr and \G have no boundary concept at all. Every hand-rolled as_bytes() + i += 1 scanner translated from Perl therefore introduces an invariant the original never had, silently, with no type-level trace — and passes every ASCII fixture forever. Witness 2605.22125: a .bib title containing \“ aborted the whole document, and the code had been live for months.

§The rule the bug teaches

The panic happens at the slice, but the defect is always in the advance:

advancesafe?
scan to an ASCII delimiteralways — an ASCII byte is never a UTF-8 continuation byte
by char::len_utf8()always
a fixed count past an unclassified bytenever

So the fix is not to check slices, it is to remove the ability to advance wrongly. This cursor exposes no byte-count advance at all, which makes slice_from infallible and the whole class unrepresentable in code written against it.

Rust guidelines anti-index-over-iter / perf-iter-over-index: prefer the iterator std already provides (char_indices) over manual indexing. This is a thin, self-documenting wrapper over exactly that — not a new abstraction.

§Cost

A Peekable<CharIndices> and the source reference; no allocation, one pass, the same traversal a byte walker made. char_indices also decodes each character once instead of re-decoding at every slice.

Structs§

CharCursor
Forward cursor over a &str, positioned only ever at char boundaries.