hazimp.pipeline

The purpose of this module is to provide objects to process a series of Job functions in a sequential order. The order is determined by the queue of Job.

Typically, a PipeLine is created through pre-defined Templates that have been built into HazImp already. These templates ensure the correct set of jobs are executed, in the correct order, for given use cases.

It’s also possible to build a PipeLine manually, using the PipeLine.add_job() method to add more jobs.

Warning

The order of jobs in a PipeLine is important. The existing templates ensure correct order of the jobs. If creating a PipeLine manually, the user will be responsible for ensuring the correct order of jobs. A RuntimeError is raised if the order is incorrect, and we recommend any new job pipelines are adapted from the existing templates.

Note

Currently we include the SaveProvenance job in the templates, so a manually defined PipeLine will have to explicitly include that at the end to ensure provenance information is captured.

Module Contents

Classes

PipeLine

PipeLine allows to create a queue of jobs and execute them in order.

Attributes

log

hazimp.pipeline.log

class hazimp.pipeline.PipeLine(jobs_list=None)

Bases: object

PipeLine allows to create a queue of jobs and execute them in order.

__eq__(self, other)

Equal operator for pipeline

__ne__(self, other)

Not equal operator for pipeline

add_job(self, a_job)

Append a new job the to queue

run(self, context)

Run all the jobs in queue, where each job take input data and write the results of calculation in context.

Parameters:

context – Context object holding the i/o data for the pipelines.