Module graphics_cache 

Content-addressed graphics cache.

Every convert_image / convert_image_svg call rasterises a self-contained source file (PDF / EPS / PNG / SVG) with a small set of render-shaping inputs (page, density, target format). Across a canvas of arXiv submissions the same byte content recurs constantly — common journal logos, reused author-affiliation marks, and regenerated runs of the same paper. PERFORMANCE.md §5 records the graphics phase as 36.5% of corpus wall, so a cache that hits even a third of the time shaves ~11% off corpus wall.

Heritage: Perl LaTeXML.cache

The Perl post-processor stores a tied BerkeleyDB hash named LaTeXML.cache per output directory. Keys are built from the processor class, the source path, and the transform string; values are "dest|width|height" strings. The features we mirror are listed below.

• Disable flag (Perl: nocache) → env LATEXML_GRAPHICS_CACHE_OFF.
• Re-use across runs (Perl: one .cache per dest dir) → global XDG cache, shared across all conversions on the host. This is a strict superset of the Perl behaviour: cross-paper sharing, plus reproducible content-keying instead of path-keying.
• Source-staleness check (Perl compared source mtime to cached output mtime) → unnecessary here: the cache key is SHA-256 of the source bytes, so any source edit produces a different key.
• Cached dimensions (Perl: width/height in the cached value) → sidecar <hash>.<ext>.dims file containing width\nheight\n. Lookup returns (success, dims); callers skip the read_image_dimensions syscall on hits.

Beyond Perl: content-hash keying gives stricter staleness; XDG placement gives cross-paper reuse; multi-process safety (below) lets canvas sweeps share the cache without corruption.

Cache key

SHA-256 of: source bytes ‖ page ‖ density ‖ target-extension

The page/density bytes are appended as 8-byte little-endian; the extension is appended as ASCII lower-case bytes. Bytes-only keying makes the cache reproducible across machines and gives near-zero collision risk for our use.

On-disk layout

$XDG_CACHE_HOME/latexml-oxide/graphics/<aa>/<full-hash>.<ext> $XDG_CACHE_HOME/latexml-oxide/graphics/<aa>/<full-hash>.<ext>.dims $XDG_CACHE_HOME/latexml-oxide/graphics/.prune.lock

Sharded by the first two hex characters of the hash to keep any one directory’s entry count bounded (worst case ~1/256th of the total cache size).

$XDG_CACHE_HOME falls back to $HOME/.cache per the XDG spec.

Multi-process safety

The cache is designed to be shared by parallel cortex_worker processes within a canvas sweep. Concurrency guarantees:

1. Concurrent writes to the same key: each writer renders to a private <final>.tmp.<pid>.<nanos> sidecar then issues a single rename(2) into the final path. rename is atomic on the same filesystem; if two writers race, the last rename wins, and because both sources are byte-equivalent the outcome is correct. link_or_copy retries once on EEXIST for the same reason.
2. Concurrent reads + writes: a reader hardlinks the cached file into the destination. POSIX link(2) either succeeds (returning a new directory entry that survives any subsequent unlink of the source) or fails cleanly. If the cache file is unlinked between hash computation and link, the reader gets a miss and falls through — no data corruption.
3. Concurrent prunes: prune holds flock(LOCK_EX | LOCK_NB) on .prune.lock. If another process already holds the lock, this one skips its prune attempt. Only one prune runs at a time per machine. The prune itself walks the dir, sorts by mtime, deletes oldest until under cap — and tolerates ENOENT (another writer could have re-used the entry concurrently).
4. Hardlinked dest ↔ cache: when link_or_copy hardlinks the cache file into the destination, that hardlink is independent of the cache lifecycle. Even if a prune deletes the cache entry afterwards, the destination’s hardlink keeps the data alive on the filesystem.

These guarantees hold on any POSIX filesystem. On Windows (which lacks robust flock), the prune lock is a best-effort OpenOptions create-new sentinel — same correctness, slightly more retry traffic.

Lifecycle

• Hit: hardlink the cache file into the destination (zero-copy on the same filesystem). If hardlink fails (cross-filesystem, EXDEV), fall back to a regular file copy. Read the .dims sidecar if present.
• Miss-then-success: copy the produced destination back into the cache. Write the .dims sidecar alongside.
• LRU prune: each insertion checks the on-disk total against the cap (LATEXML_GRAPHICS_CACHE_MAX_MB, default 2048 = 2 GB). When over cap, holds the prune lock, sorts entries by mtime ascending, deletes until under cap. File access on read also refreshes the mtime so frequently-hit entries survive.

Disable / tune

• LATEXML_GRAPHICS_CACHE_OFF=1 — bypass entirely (read+write both skipped). The wrapper devolves to the bare conversion call.
• LATEXML_GRAPHICS_CACHE_DIR=/path — override the cache directory.
• LATEXML_GRAPHICS_CACHE_MAX_MB=N — cache size cap.

Structs

CacheHit

Result of a cache lookup.

CachedDims

Cached dimensions for an image. Mirrors Perl LaTeXML.cache "dest|width|height" value triple (we already have the dest path at the call site — only the dimensions need round-tripping).

RenderKey

Render-shaping inputs that go into the cache key alongside source bytes. Two calls with the same RenderKey MUST produce byte-equivalent output (modulo metadata variation tools like ImageMagick are known to introduce — see compare_outputs_strict audit in the spawn paths).

Enums

CachePolicy

Whether a given conversion consults the shared graphics cache.

ConvertResult

Three-state result from a cached conversion. Disambiguates the two failure modes that would otherwise collapse to None:

Functions

lookup

Look the cache up. On hit: hardlink/copy into dest, refresh the cache entry’s mtime, return Some(CacheHit{dims}). On miss or any I/O hiccup, return None so the caller falls through to a real conversion.

stats

Return (hits, misses) since process start. Useful for telemetry and the post-run summary log line.

store

Insert dest (and optionally its dimensions) into the cache under (source, key). Idempotent and best-effort: any I/O failure leaves the cache unchanged.

with_cache

Bytes-only cache wrapper. Use when the caller doesn’t need dimensions cached (e.g. SVG path where viewBox dims are cheap to re-read from disk). Returns true on success.

with_cache_dims

Cache-aware wrapper around any (source, dest) -> bool conversion that also wants to round-trip dimensions through the cache.