Expand description
Main-level wall-clock watchdog that forcibly aborts the process after a deadline.
Complements the cooperative stomach::check_timeout polling for native-code hotspots
(Marpa, libxml2, libxslt) that don’t return to the digestion loop.
Wall-clock watchdog that forcibly aborts the process after a deadline.
The existing stomach::check_timeout() is a cooperative mechanism — it only
fires when the digestion loop polls it. That leaves tight native loops
(Marpa precompute / parse, libxml2 post-processing, FFI calls into libxslt,
…) completely unguarded: a 60-second timeout can easily turn into 10
minutes if control never returns to the digestion loop.
This module provides a main-level Watchdog that spawns a dedicated thread
at construction, wakes after the specified number of seconds, and — if the
watchdog has not yet been cancelled — prints a message and calls
std::process::abort(). That guarantees the process dies within
timeout + poll_interval of the configured deadline, regardless of what
the main thread is doing.
§Design notes
- Uses
Arc<AtomicBool>for cancellation. Polling every 100 ms keeps the cancellation latency low without burning CPU. Dropon theWatchdoghandle cancels the watchdog thread, so RAII usage (let _wd = Watchdog::new(secs)) is sufficient.- We use
std::process::abort()rather thanpanic!because panic may unwind or be caught by a surroundingcatch_unwind, which would defeat the safety guarantee.abort()deliversSIGABRTand always terminates the process. - The existing cooperative
stomach::check_timeout()path is retained: on most conversions it fires before the hard abort, giving callers a niceErr(Fatal)with proper error propagation. The watchdog is a safety net for the pathological cases where cooperative polling doesn’t happen.
§Resource limits
Watchdog::with_limits guards
both a wall-clock deadline and a
resident-memory ceiling — the two defenses any executable that converts
arbitrary input needs. It is the shared guard reused by both
cortex_worker (in-process, one paper per process) and the
latexml_oxide --server LSP (run inside each forked body child, which
self-terminates on breach so the parent reaps it via pipe EOF). The exit
codes are distinct so a supervising parent can tell them apart:
124 = wall-clock timeout, 137 = memory ceiling.
§Portability
The wall-clock guard is portable (std::thread + Instant).
Portable: total_memory_bytes and available_disk_bytes answer on
Linux, macOS and Windows — sysconf(_SC_PHYS_PAGES)/statvfs are POSIX, and
Windows uses GlobalMemoryStatusEx/GetDiskFreeSpaceExW. They back the
machine-derived default ceiling (default_ceiling_mib) and the
spill-headroom check, so those behave identically on every supported OS.
Enforcement, also portable: process_rss_kb — the half that actually
checks live usage against the ceiling — answers on all three: Linux samples
/proc/self/status, macOS asks libproc (proc_pidinfo), and Windows uses
GetProcessMemoryInfo. So the memory ceiling is both computed and checked
on every supported OS; other targets fall back to None (time guard only).
Structs§
Constants§
- EXIT_
OOM - Exit code used when the memory ceiling is exceeded (128 + SIGKILL).
- EXIT_
TIMEOUT - Exit code used when the wall-clock deadline is exceeded (standard
timeout). - FALLBACK_
CEILING_ MIB - Ceiling used when the machine’s RAM cannot be probed — the historical flat default, kept so an unprobeable platform behaves exactly as before rather than losing its guard entirely.
- MAX_
DEFAULT_ CEILING_ MIB - Upper bound on the machine-derived default ceiling (64 GiB in MiB).
- MIN_
DEFAULT_ CEILING_ MIB - Floor for the machine-derived default, so a small container still gets a workable budget rather than a ceiling no conversion can fit under.
Functions§
- available_
disk_ bytes - Free space in bytes on the filesystem holding
path, orNoneif it cannot be determined. Used to check headroom before spilling intermediates to disk. - available_
memory_ bytes - RAM available for new allocations right now, in bytes, or
Nonewhen the platform offers no honest answer. - default_
ceiling_ mib - The default per-conversion memory ceiling in MiB, derived from the machine as it is right now.
- note_
memory_ fatal - Record that a memory-budget Fatal fired, so the end-of-run report knows memory was actually the problem.
- peak_
memory_ report - The end-of-run memory report — emitted ONLY when a memory-budget Fatal
fired during the run (user directive 2026-08-03: alert when needed, stay
quiet on clean runs).
Noneotherwise, or when no peak is measurable. - process_
peak_ rss_ kb - Kernel-tracked PEAK resident set of this process in KiB (
VmHWM), orNoneif it can’t be determined. Unlikeprocess_rss_kb’s point-in-time sample, this is the true high-water mark over the whole process lifetime — no sampling cadence can miss a spike — which makes it the honest basis for the end-of-run “this document needed N” report (peak_memory_report). - process_
rss_ kb - Current resident set size of this process in KiB, or
Noneif it can’t be determined. Linux readsVmRSSfrom/proc/self/status; macOS asks libproc (proc_pidinfo/PROC_PIDTASKINFO) — without it the whole--max-memoryceiling silently did not exist on macOS, and the115_streaming_cliFatal-contract guard caught exactly that on macOS CI (an over-budget run exited 0). Cheap enough to poll a few times a second. - set_
pre_ exit_ hook - total_
memory_ bytes - Total physical RAM on this machine in bytes, or
Noneif it cannot be determined.