Skip to main content

Module sxml

Module sxml 

Source
Expand description

Streaming XML substrate for fragmented (bounded-memory) conversion. Streaming XML substrate — the shared representation layer for fragmented (bounded-memory) conversion.

The eager pipeline holds one whole-document DOM from Build to the final write, so peak RSS scales with document size (measured ~1.84 GB per MB of source on the 131 MB witness — docs/performance/STREAMING_CORE_DESIGN_2026-07-29.md). Fragmented mode bounds peak RSS by fragment size instead: closed subtrees are serialized to disk (“spilled”) during Build and re-materialized one at a time for the later phases. This module is the substrate both halves stand on; it knows nothing about TeX or the schema:

  • SegmentStore — the on-disk spill area: numbered segment files under a directory beside the destination (same volume, so crate::watchdog::available_disk_bytes headroom checks measure the right filesystem). Pass 1 writes parseable, _lxfragment-wrapped segments; pass 2 replaces each with its processed, splice-ready output text.
  • FragmentIndex — the document-global facts that must survive a spill as plain strings (never as node handles into a freed DOM — the historical finalize-SIGSEGV class): spilled xml:ids and which segment holds them, label → id, RDFa prefix declarations.
  • FragmentReader — a streaming iterator over a segment (or any XML file): materializes ONE top-level subtree at a time as an owned, mutable [libxml::tree::Document] via xmlTextReaderExpand, while the rest of the file stays unparsed.

Activation policy, spill-eligibility, and the placeholder-splice assembly live with their owners (stomach/document/core_interface for the core half, latexml_post::stream_split for the post half); this module only moves bytes and facts.

Structs§

FragmentIndex
The string-only registry of what spilled content still owes the rest of the conversion.
FragmentReader
Iterates the top-level subtrees of an XML file (the children of its root element), materializing one owned [Document] at a time.
SegmentId
Identifies one spilled segment within its SegmentStore.
SegmentMeta
Per-segment facts recorded at spill time and consumed when the segment is re-materialized in pass 2.
SegmentStore
The on-disk spill area: numbered segment files plus their in-RAM metadata.