Composable primitives for asynchronous data flows.
A pipe is written as nested applications, one stage per step, over the three symmetric contracts declared here: a
Feed opens the data and accepts the steps advancing it, a Task moves it on by transforming,
filtering or reshaping the items, and a Sink closes it with the final result.
A feed is opened from a value or a data source, supplied as it is or as a promise, then composed with any number of
tasks and an optional sink, bracketed by pipe. Closed by a sink, the pipe resolves to the final result;
left open, it ends with a feed, iterated with for await to draw the items it carries.
Every operation the companion modules provide states where it stands on four axes, so that a pipe can be assembled
knowing what it completes on and what it costs: bounded or infinite, for how far a feed goes; incremental
or exhaustive, for how much of a feed a task or sink draws before emitting or resolving; streaming or
materialising, for what is held in memory; stateless or stateful, for whether the outcome depends on the
items drawn before it.
Pipes compose all the way down, with no privileged core: the companion feed, task and sink modules are written
against the very contracts declared here, and so is anything you add. A custom feed opens a pipe over a source of
your own, a custom task extends one and a custom sink closes it, each chaining with the built-in steps in any order.
Every step is handed a feed in turn, so a custom one either draws the items itself or delegates the whole job, or
part of it, to steps already available.
Composable primitives for asynchronous data flows.
A pipe is written as nested applications, one stage per step, over the three symmetric contracts declared here: a Feed opens the data and accepts the steps advancing it, a Task moves it on by transforming, filtering or reshaping the items, and a Sink closes it with the final result.
A feed is opened from a value or a data source, supplied as it is or as a promise, then composed with any number of tasks and an optional sink, bracketed by pipe. Closed by a sink, the pipe resolves to the final result; left open, it ends with a feed, iterated with
for awaitto draw the items it carries.Every operation the companion modules provide states where it stands on four axes, so that a pipe can be assembled knowing what it completes on and what it costs: bounded or infinite, for how far a feed goes; incremental or exhaustive, for how much of a feed a task or sink draws before emitting or resolving; streaming or materialising, for what is held in memory; stateless or stateful, for whether the outcome depends on the items drawn before it.
Pipes compose all the way down, with no privileged core: the companion feed, task and sink modules are written against the very contracts declared here, and so is anything you add. A custom feed opens a pipe over a source of your own, a custom task extends one and a custom sink closes it, each chaining with the built-in steps in any order. Every step is handed a feed in turn, so a custom one either draws the items itself or delegates the whole job, or part of it, to steps already available.