Crate cortex 

A general purpose processing framework for corpora of scientific documents

Modules

backend

All aggregate operations over the CorTeX PostgresQL store are accessed through the connection of a Backend object.

bootstrap

Self-install and diagnostics: the library logic behind cortex init and cortex doctor.

concerns

Backend models concerns and traits

config

Layered, runtime configuration for CorTeX.

dispatcher

A ZMQ-based job dispatcher, interfacing between the task Backend and an open set of remote workers

frontend

CorTeX Frontend capabilities - reports, rendering, auth and more

helpers

Helper structures and methods for Task

importer

Import a new corpus into the framework

jobs

Background jobs: one persisted row per long-running administrative operation, run on an in-process thread with progress persisted to the database. The shared mechanism behind corpus import/extend, service activation, runs, and dataset export. See docs/archive/JOB_MODEL.md.

migrations

Embedded database migrations.

models

Backend models and traits for the CorTeX “Task store”

observability

Observability bootstrap (Arm 8): one tracing subscriber for the binaries.

reports

Virtual tables/ORM for reports produced by the CorTeX backend

schema

Auto-generated diesel schema for the backend DB

worker

Worker for performing corpus imports, when served as “init” tasks by the CorTeX dispatcher