Struct cortex::dispatcher::sink::Sink

pub struct Sink {
    pub port: usize,
    pub queue_size: usize,
    pub message_size: usize,
    pub backend_address: String,
}

Specifies the binding and operation parameters for a ZMQ sink component

Fields

port: usize

port to listen on

queue_size: usize

the size of the dispatch queue (also the batch size for Task store queue requests)

message_size: usize

size of an individual message chunk sent via zeromq (keep this small to avoid large RAM use, increase to reduce network bandwidth)

backend_address: String

address for the Task store postgres endpoint

Implementations

impl Sink

pub fn start(
    &self,
    services_arc: &Arc<Mutex<HashMap<String, Option<Service>>>>,
    progress_queue_arc: &Arc<Mutex<HashMap<i64, TaskProgress>>>,
    done_queue_arc: &Arc<Mutex<Vec<TaskReport>>>,
    job_limit: Option<usize>
) -> Result<(), Box<dyn Error>>

Starts a receiver/sink Server (ZMQ Pull), to accept processing responses. The sink shares state with other manager threads via queues for tasks in progress, as well as a queue for completed tasks pending persisting to disk. A job limit can be provided as a termination condition for the sink server.