Function retire

Creates a task that selects the resources to retire.

The incoming feed is a scan of the source: it lists the identifiers of every resource the source still provides. The existing feed lists the identifiers currently in the store. The task returns the existing identifiers that are missing from the scan, each at most once. It returns nothing until the scan is complete, so a scan still in progress never leads to the retirement of a resource that the rest of the scan would have listed.

A scan that returns too few identifiers usually means a broken source rather than a source that was really emptied. A scan is therefore refused if it lists fewer identifiers than the survival fraction of the existing ones. The check counts every identifier in the scan, not only those already in the store. A source that renames all its resources at once still passes the check, while a truncated or unreachable source is still caught.

By default, a refused scan skips only the retirements and lets the other writes of the run go through. Once the source works again, its next scan retires the same resources. With atomic, a refused scan fails the whole run instead. This suits sources that cannot tolerate a partially applied run of the pipeline.

A survival of 0 disables the check, so even an empty scan retires everything. A survival of 1 refuses any scan that lists fewer identifiers than the store currently holds for the source.