Struct InFlightSet 

pub struct InFlightSet { /* private fields */ }

The dispatcher’s in-flight (progress) set: the dispatched-but-unfinished tasks, keyed by task id. Phase 4 of the dispatcher rationalization replaces the contended Arc<Mutex<HashMap<i64, TaskProgress>>> with a sharded, lock-free [DashMap] — so the ventilator’s lease (Self::lease), the sink’s return (Self::claim_result), and the reaper sweep (Self::take_expired) no longer serialise on one global lock — plus an AtomicUsize size counter so the per-request backpressure check (in_flight_saturated) reads the size in O(1) without locking or scanning the map.

It also holds, for late_result_window seconds, each task the reaper released (re-queued or dead-lettered), with the worker of every attempt, so the sink can accept a result whose lease expired unless a result that must stand already landed (KNOWN_ISSUES D-26). Those entries are not in flight: Self::len, Self::is_empty and Self::ids count only leased tasks. Every transition of one task id happens under its map shard’s lock, so a lease, a result and a reap of the same task cannot interleave.

The counters are maintained in lock-step with the map here (the only place any is mutated), so they converge to exactly the map’s state; a momentary ±1 skew between a map mutation and its atomic update is harmless for backpressure (self-correcting, never a leak). Shared as Arc<InFlightSet> across the ventilator / sink / reaper threads.

Implementations

impl InFlightSet

pub fn new() -> Self

A new, empty in-flight set with the default late-result window.

pub fn with_late_result_window(seconds: i64) -> Self

A new, empty in-flight set that keeps reaped tasks for late results for seconds (<= 0: no late results are accepted).

pub fn len(&self) -> usize

Current number of in-flight tasks — an O(1) atomic load (the backpressure hot read), not a map scan.

pub fn is_empty(&self) -> bool

Whether the in-flight set is empty.

pub fn awaiting_replies(&self) -> usize

Number of in-flight tasks still awaiting a reply: Self::len less those whose current attempt answered with an unusable result or a retryable cause, which only the reaper moves on. A bounded run’s sink stops once dispatching is done and this is zero.

pub fn settling_len(&self) -> usize

Number of tasks held out of flight for a late result.

pub fn insert(&self, progress_task: TaskProgress)

Record a dispatched task as in-flight (keyed by task.id), unconditionally. Re-inserting the same id overwrites without double-counting. The ventilator leases through Self::lease.

pub fn lease(&self, progress_task: TaskProgress) -> bool

Lease a task: record it in flight, unless it is a re-queued retry that a late result already finished while it waited in the dispatch queue (KNOWN_ISSUES D-26) — then false, and the caller drops it instead of converting it again. A first attempt (retries == 0: a fresh fetch, e.g. an operator’s rerun of the task) is always leased; a held task’s earlier attempts become its older attempts. A retry takes the last retryable cause reported while it was queued.

pub fn remove(&self, taskid: i64) -> Option<TaskProgress>

Remove and return the in-flight task with taskid. Negative (mock-reply) ids are never tracked, so they short-circuit to None; removing an already-gone task is a no-op that never underflows the counter, and a task held for a late result is left alone.

pub fn task(&self, taskid: i64) -> Option<Task>

The task a result for taskid would be reported against — in flight or held for a late result — without claiming it. The sink checks the result’s service and output path against it first, so an unplaceable result leaves the task exactly where it was (the reaper’s).

pub fn note_received(&self, taskid: i64)

A result for in-flight task taskid was received and handed to its archive writer: the reaper leaves the task alone until the writer decides the result (Self::claim_result, Self::defer_retryable, Self::note_unusable_reply), so a result that arrives at the deadline is not raced by a re-lease.

pub fn note_unusable_reply(&self, taskid: i64, worker: &str, received: bool)

The worker worker replied for in-flight task taskid with a result that could not be used (empty, unreadable, unwritable, unplaceable — D-18): the task stays in flight for the reaper (retry, then dead-letter). If the reply is the current attempt’s, the task awaits no reply. received says whether the reply was recorded by Self::note_received (an unplaceable reply never was).

pub fn defer_retryable( &self, taskid: i64, worker: &str, cause: Vec<NewTaskMessage>, ) -> RetryDecision

The worker worker reported a retryable cause for task taskid (the harness’s Fatal:timeout:wallclock, Fatal:oom:shed, Fatal:crash:unrecorded, tagged Info:cortex:retryable): not a result to record, but the task’s retry budget’s business. For the attempt in flight with budget left, the cause becomes the task’s last cause and its lease is due now: the next reap re-queues it (RetryDecision::Deferred). On the task’s last attempt the cause is its final record (RetryDecision::Final): the caller publishes it as a stand-in, held like an accepted failure. For a task re-queued meanwhile, the newest attempt’s cause rides along to its retry. Otherwise — an older attempt’s cause, a task whose result already landed, or one already dead-lettered — it is dropped.

pub fn claim_result( &self, taskid: i64, worker: &str, status: TaskStatus, ) -> ResultClaim

Decide a result with status status for taskid from the worker worker (KNOWN_ISSUES D-26): accept it unless a result that must stand already landed — a newer attempt’s, or a usable one that this failure (Fatal/Invalid) must not replace. An accepted result takes the task out of flight; if it belongs to an older attempt than the one in flight, the task is held so the newer attempt’s result can still replace it, and a result for a re-queued task keeps the queued retry from being leased (Self::lease).

pub fn note_publish_failed( &self, taskid: i64, replaced: Option<(usize, TaskStatus)>, )

The accepted result for held task taskid could not be published: restore the result it replaced (replaced, from ResultClaim::Accept), or, when it replaced none, leave the task open for any later result — a publish failure never costs the task a good result.

pub fn take_expired(&self) -> Vec<TaskProgress>

Take every in-flight task past its visibility-timeout deadline (the reaper sweep), except one whose received result is still with its writer. Each one stays in the set, out of flight, for the late-result window (unless the window is off or SETTLING_HARD_LIMIT is reached): a result of any of its attempts may still land there.

pub fn forget_settled(&self) -> usize

Forget the tasks whose late-result window has passed. Returns how many. Run on the reaper’s cadence.

pub fn queued_retries(&self) -> Vec<TaskProgress>

The retries the reaper queued in the ventilator that no lease or late result has taken yet, rebuilt for a restarted ventilator (its queue died with it) to queue again rather than reset to TODO: a reset task would be re-fetched as a new run, and its earlier attempts’ late results dropped as stale (D-28). Due at once (created_at 0).

pub fn worked_earlier_run(&self, taskid: i64, worker: &str) -> bool

Whether worker worked an attempt of taskid that the set still holds — so a fresh fetch leased to it would start a new run whose results could not be told from the old run’s.

pub fn kept_on_restart(&self) -> Vec<i64>

The task ids a restarted ventilator must not reset to TODO: every task in flight (its result may still come) and every task held for late results — a queued retry (queued again, Self::queued_retries), or one whose report (a late result, a stand-in) may not be written yet: a reset row is no longer at the report’s lease mark, so finalize would drop it (D-28).

pub fn ids(&self) -> Vec<i64>

Snapshot of the currently in-flight task ids — used on a ventilator restart to exclude these from the Queued → TODO crash-recovery reset (clear_limbo_tasks_except), so tasks the sink is still processing are not re-leased mid-flight (KNOWN_ISSUES D-4).

Trait Implementations

impl Default for InFlightSet

fn default() -> Self

Returns the “default value” for a type.