Ready-made tasks for retrieving and persisting data from external sources.
@metreeca/pipe brings ready-made @metreeca/flow tasks for moving content between a pipeline and the systems it works with: drawing records out of web pages, object stores and databases, and writing results back to them. The tasks run under the @metreeca/gear job executor, which supplies the shared services they draw on.
Pipelines are server-side workloads targeting Node.js 22 or later, relying on facilities such
as the filesystem, the process environment and fetch. The packages are not intended for the browser.
npm install @metreeca/gear # job executor and shared services
npm install @metreeca/pipe-<source> # task package, one per source family
TypeScript consumers must use "moduleResolution": "nodenext"/"node16"/"bundler" in tsconfig.json.
The legacy "node" resolver is not supported.
Add a task package for each source family the pipeline reaches. Source packages are self-contained leaves, each pulling in only the drivers its own family needs. The job executor comes from @metreeca/gear, which the task packages pull in transitively; install it directly to set up and run a job.
| Package | Description |
|---|---|
| @metreeca/pipe-url | URL processing tasks |
Each package documents its own API in its README and API reference; for complete coverage, see the API reference.
This project is licensed under the Apache 2.0 License – see LICENSE file for details.