Module corpora 

Corpus-management capability: list/inspect/import/delete corpora as screens + API.

Follows the symmetry contract — one shared CorpusDto renders as JSON for agents and (later) as HTML for humans. Handlers live here; the app is assembled in crate::frontend::server. This is the first capability drained out of the binary’s legacy routes; more land per increment.

Structs

ActivateForm

Fields of the human “Activate a service” form on the corpus screen.

CorpusDetailDto

A corpus with its activated services and their status counts.

CorpusDto

A corpus as exposed over the API/UI. name is the stable external handle used by every route.

DeactivateForm

Fields of the human per-service “Deactivate” form: the service echoed as confirmation.

DeleteForm

Fields of the human “Delete corpus” form: the name echoed as confirmation.

ExportForm

Fields of the human “Export dataset” form (the web twin of export_dataset).

ExportRequest

Request body for exporting a corpus/service’s converted HTML into ZIP archives (export_dataset). Mirrors the cortex export-dataset CLI flags.

ImportForm

Fields of the human “Add a corpus” form on the admin dashboard.

ImportRequest

Request body for registering and importing a corpus.

SandboxForm

Fields of the human “Create a sandbox” form on the corpus page.

SandboxProvenanceDto

Provenance of a sandbox corpus: the parent it was carved from and the carve predicate. Only present on sandbox corpora (null for ordinary corpora).

SandboxRequest

Request body for carving a sandbox corpus out of a parent by a filter (Arm 5). Task-status and message-severity are independent, intersecting dimensions (Model C); category/what narrow the message filter, mirroring the report drill-down.

ServiceStatusDto

Per-service status counts within a corpus (mirrors the progress report).

SnapshotAckDto

Acknowledgement for a save-snapshot: the (corpus, service) frozen and how many per-task status rows were appended to historical_tasks.

Functions

activate_service

Activates a registered service on a corpus: creates a TODO task per imported document so the workers begin converting it. Token-gated via the Actor guard (the run is attributed to the authenticated actor); the work runs as a background job — poll GET /api/jobs/<uuid> for the pending/done status. 401 without a valid token, 404 on an unknown corpus/service, 409 if the service is already registered on the corpus (registration is idempotent-neutral — no re-activation wipes existing results; use extend/rerun instead), 202 with the job handle on success.

activate_service_human

The human twin of activate_service: the corpus screen’s “Activate a service” form. Gated by the signed-in AdminSession cookie (anonymous → sign-in); spawns the activation job and redirects to /jobs. 404 on an unknown corpus/service.

api_corpora

Lists all registered corpora (the agent twin of the overview screen).

api_corpus

Inspects a single corpus: its activated services and per-service status counts.

corpus_page

The corpus screen (HTML twin of api_corpus): the services registered on a corpus. 404 if the corpus is unknown, 503 if the pool is exhausted.

create_sandbox_corpus

Carves a sandbox corpus from <parent> by a message-condition filter and starts the job that populates it; returns 202 Accepted + the job handle to poll. Token-gated via the Actor guard; 401 without a valid token, 404 if the parent is unknown, 409 if the sandbox name is taken. The sandbox is a first-class corpus an agent can then run/rerun to iterate a campaign.

create_sandbox_human

The human twin of create_sandbox_corpus: the corpus page’s “Create a sandbox” form. Gated by the signed-in AdminSession cookie; carves the sandbox off the request path and redirects to /jobs. 404 unknown parent, 409 name taken.

deactivate_service

Deactivates (retires) a service from a corpus: deletes that pair’s tasks + log messages (the service definition and its work on other corpora are untouched — the symmetric counterpart of activate_service). Token-gated via the Actor guard and confirmation-gated (?confirm=<service>, echoing the service name). Returns 204 on success, 400 if the confirmation doesn’t match, 404 if the corpus or service is unknown.

deactivate_service_human

The human twin of deactivate_service: the corpus screen’s per-service “Deactivate” form. Gated by the signed-in AdminSession cookie (anonymous → sign-in) and confirmation-gated (echoes the service name), then redirects back to the corpus page. 400 if the confirmation doesn’t match, 404 if unknown.

delete_corpus

Deletes a corpus and all of its tasks and log messages. Token-gated via the Actor guard (an unauthenticated wipe of a corpus must not be possible — 401 without a valid token) and double-guarded: the caller must also echo the corpus name via ?confirm=<name> to proceed (prevents accidental wipes; the UI confirms the same way). Returns 204 on success, 400 if the confirmation does not match, 404 if unknown. Every sandbox carved from the corpus (the corpora whose parent chain leads to it in GET /api/corpora) is deleted with it — FK cascade.

delete_corpus_human

The human twin of delete_corpus: the corpus screen’s “Delete corpus” form. Gated by the signed-in AdminSession cookie (anonymous → sign-in) and confirmation-gated (the form echoes the corpus name), then redirects to the overview. 400 if the confirmation doesn’t match, 404 if unknown.

export_dataset

Exports a corpus/service’s already-converted HTML into ZIP archives off the shared filesystem as an in-process background job (no conversion is run); returns 202 Accepted + the job handle, which agents and humans poll via GET /api/jobs/<uuid>. The agent twin of cortex export-dataset (and the future web form), over the same export_html_dataset core. Token-gated via the Actor guard (it reads /data and writes archives server-side); 401 without a valid token, 404 if the corpus or service is unknown, 422 for an invalid group_by or severity key (pre-flighted so a doomed export never starts).

export_dataset_human

The human twin of export_dataset: the “Export dataset” form post. Gated by the signed-in AdminSession cookie (anonymous → sign-in); spawns the background export via the shared [start_export] core and redirects to the job’s live-progress page. A failed submit re-renders the form with a friendly message + the values preserved (404 unknown corpus/service, 422 bad grouping / no severity).

export_dataset_page

The “Export dataset” screen (GET /export/<c>/<s>): the human form that drives the same export as cortex export-dataset and the agent export_dataset. Signed-in admins only (anonymous → sign-in). Pre-fills a default output path + the CLI’s default severity set. A sibling top-level path (like /runs/<c>/<s>, /history/<c>/<s>) so it never collides with the report ladder’s /corpus/<c>/<s>/<severity> rung.

extend_corpus

Extends an existing corpus with newly-arrived entries; starts an in-process job and returns 202 Accepted + the job handle. Token-gated via the Actor guard; 401 without a valid token, 404 if the corpus is unknown, 409 if it is a sandbox (a frozen snapshot — extend its parent), 422 if its source path is unreadable.

extend_corpus_human

The human twin of extend_corpus: the corpus screen’s “Re-scan for new entries” button. Gated by the signed-in AdminSession cookie (anonymous → sign-in); spawns the extend job and redirects to /jobs. 404 if the corpus is unknown, 409 if it is a sandbox.

import_corpus

Registers a corpus and starts an in-process import job; returns 202 Accepted + the job handle. Agents and humans poll GET /api/jobs/<uuid> (or the progress page) for completion. Token-gated via the Actor guard (creating a corpus + a filesystem import job is a consequential write); 401 without a valid token, 409 if the corpus name already exists or a root corpus already owns the path, 422 if the path is not a readable directory on the server.

import_corpus_human

The human twin of import_corpus: the admin dashboard’s “Add a corpus” form. Gated by the signed-in AdminSession cookie (no token typed in the form — an anonymous browser is redirected to sign-in); registers + imports the corpus off the request path and redirects to /jobs. 409 if the name or the root path is taken.

new_corpus_page

The “Add a corpus” form (GET /corpora/new). Signed-in admins only (anonymous → sign-in).

overview_page

The overview screen (HTML twin of api_corpora): the table of registered corpora — the admin landing page. 503 if the pool is exhausted.

routes

The route set for the corpus-management capability (API + human screens).

snapshot_tasks

Freezes the current per-task statuses of a (corpus, service) into historical_tasks — the agent twin of the report screen’s “save snapshot” action (POST /savetasks/...), so an agent can capture a baseline before a rerun campaign and later diff against it (GET /api/runs/.../tasks). Token-gated via the Actor guard; the snapshot is append-only (history stays immutable over the API — there is deliberately no snapshot delete/modify endpoint; pruning is a human-admin operation, see crate::frontend::retention). 401 without a valid token, 404 on an unknown corpus/service, 202 with the appended-row count on success. Uses a fresh connection (not the request pool) since the snapshot is a single bulk INSERT … SELECT over every task and shouldn’t pin a pooled slot.