Function tokenize_internal 

pub fn tokenize_internal(text: impl Into<TeXString>) -> Tokens

Tokenize a string under the style-file catcode table — Perl Package.pm:TokenizeInternal L1026-1030.

Same swap-and-restore discipline as tokenize, but with @ a letter, so internal control sequences (\@ifnextchar, \@currentlabel, …) tokenize as single names. This is the right choice for a macro body a binding writes itself, and the wrong one for text that came from the document.

Takes an impl Into<TeXString> for the same reason as tokenize.