:py:mod:`hazimp.parallel` ========================= .. py:module:: hazimp.parallel .. autoapi-nested-parse:: Functions that haven't found a proper module. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: hazimp.parallel.Parallel Functions ~~~~~~~~~ .. autoapisummary:: hazimp.parallel.scatter_dict hazimp.parallel.gather_dict hazimp.parallel.csv2dict Attributes ~~~~~~~~~~ .. autoapisummary:: hazimp.parallel.STATE .. py:class:: Parallel Bases: :py:obj:`object` Parallelise to run on a cluster. :param rank: What is the id of this node in the cluster. :param size: How many processors are there in the cluster. :param node: name of the cluster node. :param is_parallel: True if parallel is operational :param file_tag: A string that can be added to files to identify who wrote the file. .. py:method:: _not_parallel(self) Set the attributes if there is only one node. .. py:data:: STATE .. py:function:: scatter_dict(whole) Broadcast and recieve a dictionary where the values are 1d arrays and the arrays are chunked for the workers. Only rank 0 needs the whole dictionary. :param whole: The dictionary of 1d arrays to subdict. :returns: (chunk of dictionary of 1d arrays, indexes of whole array) .. py:function:: gather_dict(subdict, indexes) Recieve a dictionary from the children where the values are 1d arrays and the arrays are chunks of the whole dictionary. :param indexes: The indexes into the whole array. :param subdict: The dictionary of 1d arrays to subset. :returns: whole array .. py:function:: csv2dict(filename, use_parallel=True) Read a csv file in and return the information as a dictionary where the key is the column names and the values are column arrays. This dictionary will be chunked and sent to all processors. :param filename: The csv file path string. :returns: subsection of the array