`hazimp.jobs.jobs`¶

A collection of jobs to put into the pipeline. Jobs know about the context instance. The initial jobs will be for setting up the calculations, such as loading the exposure data.

And key, value pairs that are in the config file are passed to the jobs function. The function name is used to determine what to pass in.

Special named parameters:

file_name - The attribute used to describe files to load. If the file is not present Error out. This is checked in the validate job.

file_list - The attribute used to describe a list of files. If any file is not present Error out.

Module Contents¶

Classes¶

`Job`	Abstract Jobs class. Should use abc then.
`ConstTest`	Simple test class. Moving a config value to the context.
`Const`	Given a key and a constant value, insert a vector of the value.
`RandomConst`	Given a key and a dictionary of values with an associated probability
`Add`	Add two columns together, put the answer in a new column.
`Mult`	Multiply two columns together, put the answer in a new column.
`MultipleDimensionMult`	Multiply two arrays together, put the answer in a new array.
`LoadCsvExposure`	Read a csv exposure file into the context object.
`LoadXmlVulnerability`	Read the vulnerability sets into the context object.
`SimpleLinker`	Link a list of vulnerability functions to each asset, given the
`SelectVulnFunction`	Produce vulnerability curves for each asset, given the
`LookUp`	Do a lookup on all the vulnerability curves, returning the
`PermutateExposure`	Iterate through the exposure attributes, randomly permutating
`LoadRaster`	Load one or more files and get the value for all the
`AggregateLoss`	Aggregate loss attributes based on the `groupby` attribute
`SaveExposure`	Save all of the exposure information in the context.
`SaveAggregation`	Abstract Jobs class. Should use abc then.
`Aggregate`	Abstract Jobs class. Should use abc then.
`Tabulate`	Abstract Jobs class. Should use abc then.
`Categorise`	Abstract Jobs class. Should use abc then.
`SaveProvenance`	Abstract Jobs class. Should use abc then.

Attributes¶

`ADD`
`MULT`
`MDMULT`
`CONSTANT`
`LOADCSVEXPOSURE`
`LOADRASTER`
`LOADXMLVULNERABILITY`
`SIMPLELINKER`
`SELECTVULNFUNCTION`
`LOOKUP`
`SAVEALL`
`SAVEAGG`
`VALIDATECONFIG`
`CELLJOIN`
`RANDOM_CONSTANT`
`PERMUTATE_EXPOSURE`
`AGGREGATE_LOSS`
`AGGREGATE`
`TABULATE`
`CATEGORISE`
`SAVEPROVENANCE`
`DATEFMT`
`JOBS`

hazimp.jobs.jobs.ADD = add¶

hazimp.jobs.jobs.MULT = mult¶

hazimp.jobs.jobs.MDMULT = MultipleDimensionMult¶

hazimp.jobs.jobs.CONSTANT = constant¶

hazimp.jobs.jobs.LOADCSVEXPOSURE = load_exposure¶

hazimp.jobs.jobs.LOADRASTER = load_raster¶

hazimp.jobs.jobs.LOADXMLVULNERABILITY = load_xml_vulnerability¶

hazimp.jobs.jobs.SIMPLELINKER = simple_linker¶

hazimp.jobs.jobs.SELECTVULNFUNCTION = select_vulnerability_functions¶

hazimp.jobs.jobs.LOOKUP = look_up¶

hazimp.jobs.jobs.SAVEALL = save_all¶

hazimp.jobs.jobs.SAVEAGG = save_agg¶

hazimp.jobs.jobs.VALIDATECONFIG = validate_config¶

hazimp.jobs.jobs.CELLJOIN = cell_join¶

hazimp.jobs.jobs.RANDOM_CONSTANT = random_constant¶

hazimp.jobs.jobs.PERMUTATE_EXPOSURE = permutate_exposure¶

hazimp.jobs.jobs.AGGREGATE_LOSS = aggregate_loss¶

hazimp.jobs.jobs.AGGREGATE = aggregate¶

hazimp.jobs.jobs.TABULATE = tabulate¶

hazimp.jobs.jobs.CATEGORISE = categorise¶

hazimp.jobs.jobs.SAVEPROVENANCE = saveprovenance¶

hazimp.jobs.jobs.DATEFMT = %Y-%m-%d %H:%M:%S¶

class hazimp.jobs.jobs.Job¶

Bases: object

Abstract Jobs class. Should use abc then.

get_call_funct(self)¶: Return the ‘user’ name for the function

get_required_args_no_context(self)¶

Get the arguments and default arguments of the job function.

Any context parameter will be ignored.

Returns:: args - the arguments of the job function.
Returns:: defaults - the default arguments of the job function.

class hazimp.jobs.jobs.ConstTest¶

Bases: Job

Simple test class. Moving a config value to the context.

__call__(self, context, c_test=None)¶

A dummy job for testing.

Parameters:

context – The context instance, used to move data around.
c_test – Variable to add to context.

class hazimp.jobs.jobs.Const¶

Bases: Job

Given a key and a constant value, insert a vector of the value.

__call__(self, context, var, value)¶

Given a key and a constant value, insert a vector of the value.

Parameters:

context – The context instance, used to move data around.
var – Variable to add to context.
value – Value of the variable added.

class hazimp.jobs.jobs.RandomConst¶

Bases: Job

Given a key and a dictionary of values with an associated probability probabilistically assign a value to each element in the array.

__call__(self, context, var, values, forced_random=None)¶

A dummy job for testing.

Parameters:

context – The context instance, used to move data around.
var – Variable to add to context.
values – Value of the variable added.
forced_random – Used for testing. A vector or value to be used as the random numbers.

class hazimp.jobs.jobs.Add¶

Bases: Job

Add two columns together, put the answer in a new column.

__call__(self, context, var1, var2, var_out)¶

Add two columns together, put the answer in a new column.

Parameters:

context – The context instance, used to move data around.
var1 – The values in this column are added.
var2 – The values in this column are added.
var_out – The new column name, with the values of var1 + var2.

class hazimp.jobs.jobs.Mult¶

Bases: Job

Multiply two columns together, put the answer in a new column.

__call__(self, context, var1, var2, var_out)¶

Multiply two arrays together, put the answer in a new array.

Parameters:

context – The context instance, used to move data around.
var1 – The values in this column are Multiplied.
var2 – The values in this column are Multiplied.
var_out – The new column name, with the values of var1 * var2.

class hazimp.jobs.jobs.MultipleDimensionMult¶

Bases: Job

Multiply two arrays together, put the answer in a new array.

Var1 has assets in the 0 dimension, and can have other dimensions. Var2 has the asserts in the 0 dimension and has only this dimension.

__call__(self, context, var1, var2, var_out)¶

Multiply two columns together, put the answer in a new column.

Parameters:

context – The context instance, used to move data around.
var1 – Variable name of data. Usually intensity. Assets is the 0 dimension, and can have other dimensions.
var2 – The values in this column are Multiplied. Asserts is the 0 dimension and has only this dimension.
var_out – The new variable name, with the values of var1 * var2.

class hazimp.jobs.jobs.LoadCsvExposure¶

Bases: Job

Read a csv exposure file into the context object.

__call__(self, context, file_name, exposure_latitude=None, exposure_longitude=None, use_parallel=True)¶

Read a csv exposure file into the context object.

Parameters:

context – The context instance, used to move data around.
file_name – The csv file to load.
exposure_latitude – the title string of the latitude column.
exposure_longitude – the title string of the longitude column.

Content return:: exposure_att: Add the file values into this dictionary. key: column titles value: column values, except the title

class hazimp.jobs.jobs.LoadXmlVulnerability¶

Bases: Job

Read the vulnerability sets into the context object.

__call__(self, context, file_name: Union[str, list])¶

Read XML vulnerability files into the context object.

Parameters:

context – The context instance, used to move data around.
file_name – The xml files to load.

class hazimp.jobs.jobs.SimpleLinker¶

Bases: Job

Link a list of vulnerability functions to each asset, given the vulnerability_sets and exposure columns that represents the vulnerability function id.

__call__(self, context, vul_functions_in_exposure)¶

Link a list of vulnerability functions to each asset, given the vulnerability_sets and exposure columns that represents the vulnerability function id.

Parameters:

context – The context instance, used to move data around.
vul_functions_in_exposure – A dictionary with keys being vulnerability_set_ids and values being the exposure title that holds vulnerability function ID’s.

Content return:: vul_function_titles: Add’s the exposure_titles

class hazimp.jobs.jobs.SelectVulnFunction¶

Bases: Job

Produce vulnerability curves for each asset, given the vulnerability_sets and exposure columns that represents the vulnerability function id.

From the vulnerability set and a function id you get the vulnerability function. Then, using the variability_method e.g. ‘mean’ you get the vulnerability curve.

__call__(self, context, variability_method=None)¶

Specifies what vulnerability sets to use. Links vulnerability curves to assets. Assumes the necessary vulnerability_sets have been loaded and there is an exposure column that represents the vulnerability function id.

NOTE:: This is where the vulnerability function is selected,: As well as sampled.

Parameters:

context – The context instance, used to move data around.
variability_method – The vulnerability sets that will be looked up and the sampling method used for each set. A dictionary with keys being vulnerability_set_ids and values being the sampling method to generate a vulnerability curve from a vulnerability function. e.g. {‘EQ_contents’: ‘mean’, ‘EQ_building’: ‘mean’} Limitation: A vulnerability set can only be used once, since it needs a unique name.

Returns:

A dictionary of realised vulnerability curves, associated with the exposure data. key - intensity measure; value - realised vulnerability curve instance per asset

class hazimp.jobs.jobs.LookUp¶

Bases: Job

Do a lookup on all the vulnerability curves, returning the associated loss.

__call__(self, context)¶

Does a look up on all the vulnerability curves, returning the associated loss.

Parameters:: context – The context instance, used to move data around.
Returns:: exposure_vuln_curves: A dictionary of realised vulnerability curves, associated with the exposure data. Keys are intensity measure, values are the realised vulnerability curve instance per asset

class hazimp.jobs.jobs.PermutateExposure¶

Bases: Job

Iterate through the exposure attributes, randomly permutating fields each time.

__call__(self, context, groupby=None, iterations=1000, quantile=[0.05, 0.95])¶

Calculates the loss for the given vulnerability set, randomly permutating the exposure attributes to arrive at a distribution of loss outcomes. We do not take the absolute maximum loss, rather we use an upper quantile of the accumulated loss to define “minimum” (or “best-case”) and “maximum” (or “worst-case”) loss.

The result is that the “structural_min” and “structural_max” is the loss associated with the permutation that gives the lower and upper percentile of total loss for the analysis area. The “structural” value is the average loss across all permutations.

Parameters:

context – The context instance, used to move data around.
groupby (str) – The name of the exposure attribute to group exposure assets by before randomly permutating the corresponding vulnerability curves.
iterations (int) – Number of iterations to perform
quantile (list) – Represents the “minimum” and “maximum” event loss in the range [0, 1], default=[0.05, 0.95]

class hazimp.jobs.jobs.LoadRaster¶

Bases: Job

Load one or more files and get the value for all the points. Primarily this will be used to load hazard data.

There may be NAN values in this data

__call__(self, context, attribute_label, file_list, clip_exposure2all_hazards=False, file_format=None, variable=None, no_data_value=None)¶

Load one or more files and get the value for all the exposure points. All files have to be of the same attribute. Alternatively a numeric array of the raster data can be passed in.

Parameters:

context – The context instance, used to move data around.
attribute_label – The string to be associated with this data.
clip_exposure2all_hazards – True if the exposure data is clippped to the hazard data, so no hazard values are ignored.
file_list – A list of files or a single file to be loaded.
no_data_value – Values in the raster that represent no data.

Context return:

exposure_att: Add the file values into this dictionary.: key: column titles value: column values, except the title

class hazimp.jobs.jobs.AggregateLoss¶

Bases: Job

Aggregate loss attributes based on the groupby attribute used in the permutation of the vulnerability curves.

__call__(self, context, groupby=None, kwargs=None)¶

Aggregate using pandas.GroupBy objects

Parameters:

context – The context instance, used to move data around.
groupby – The name of the exposure attribute to group exposure assets by before performing aggregations (sum, mean, etc.).

class hazimp.jobs.jobs.SaveExposure¶

Bases: Job

Save all of the exposure information in the context.

__call__(self, context, file_name=None, use_parallel=True)¶

Save all of the exposure information in the context.

Parameters:: context – The context instance, used to move data around.
Params file_name:: The file where the expsoure data will go.

class hazimp.jobs.jobs.SaveAggregation¶

Bases: Job

Abstract Jobs class. Should use abc then.

__call__(self, context, file_name=None, use_parallel=True)¶

Save all of the aggregated exposure information in the context.

Parameters:: context – The context instance, used to move data around.
Params file_name:: The file where the expsoure data will go.

class hazimp.jobs.jobs.Aggregate¶

Bases: Job

Abstract Jobs class. Should use abc then.

__call__(self, context, filename=None, boundaries=None, impactcode=None, boundarycode=None, categories=True, fields=None, categorise=None, use_parallel=True)¶

Aggregate the data by geographic areas

This calls context.save_aggregation() to do the work.

Parameters:

filename (str) – Path to the output file
boundaries (str) – Path to a geospatial data file that contains polygon features by which the data will be aggregated.
impactcode (str) – An attribute in the exposure file that contains a unique code for each geographic region to aggregate by.
boundarycode (str) – An attribute in the boundaries file that contains the same unique code for each geographic region. Prferably the impactcode and boundarycode will be of the same type. These values are used to join the two datasets.
fields (dict) –
A dict with keys of valid column names (from the DataFrame) and values being lists of aggregation functions to apply to the columns. For example:
```
fields=dict('structural': ['mean', 'max'])
```

categorise (dict) –

A dict containing key/value pairs of bins and labels used to classify loss data. For example:

categorise = dict('bins': [0, 0.02, 0.1, 0.2, 0.5, 1.0],
                  'labels': ['Negligible', 'Slight',
                             'Moderate', 'Extensive',
                             'Complete'])

class hazimp.jobs.jobs.Tabulate¶

Bases: Job

Abstract Jobs class. Should use abc then.

__call__(self, context, file_name=None, index=None, columns=None, aggfunc=None, use_parallel=True)¶

class hazimp.jobs.jobs.Categorise¶

Bases: Job

Abstract Jobs class. Should use abc then.

__call__(self, context, bins=None, labels=None, field_name=None)¶: Calls the categorise() method on the Context object. All args are passed through to the method without modification.

class hazimp.jobs.jobs.SaveProvenance¶

Bases: Job

Abstract Jobs class. Should use abc then.

__call__(self, context, file_name=None)¶

Save provenance information. By default we save to xml format. This will also generate an image of the provenance graph.

Parameters:

context – Context instance to move data around, including provenance information
file_name (str) – Destination for the provenance file.

hazimp.jobs.jobs.JOBS¶

`hazimp.jobs.jobs`¶

Module Contents¶

Classes¶

Attributes¶

Table of Contents

This Page

hazimp.jobs.jobs¶

Module Contents¶

Classes¶

Attributes¶

`hazimp.jobs.jobs`¶