Module: processing

The processing module supports multiple execution of EXUDYN models. It includes parameter variation and (genetic) optimization functionality.

Author: Johannes Gerstmayr, Stefan Holzinger
Date: 2020-11-17 (2022-02-04 modified by Stefan Holzinger)
Notes: Parallel processing, which requires multiprocessing library, can lead to considerable speedup (measured speedup factor > 50 on 80 core machine). The progess bar during multiprocessing requires the library tqdm.

Function: GetVersionPlatformString

GetVersionPlatformString()

function description:

internal function to return Exudyn version string, which allows to identify how results have been obtained

writes something like ‘Exudyn version = 1.2.33.dev1; Python3.9.11; Windows AVX2 FLOAT64; Windows10 V10.0.19044; AMD64; Intel64 Family 6 Model 142 Stepping 10, GenuineIntel’
notes:

If exudyn C++ module is not available, it outputs the Python version

Function: ProcessParameterList

ProcessParameterList(parameterFunction, parameterList, useMultiProcessing, clusterHostNames = [], **kwargs)

function description:

processes parameterFunction for given parameters in parameterList, see ParameterVariation
input:

parameterFunction: function, which takes the form parameterFunction(parameterDict) and which returns any values that can be stored in a list (e.g., a floating point number)

parameterList: list of parameter sets (as dictionaries) which are fed into the parameter variation, see example

useMultiProcessing: if True, the multiprocessing lib is used for parallelized computation; WARNING: be aware that the function does not check if your function runs independently; DO NOT use GRAPHICS and DO NOT write to same output files, etc.!

numberOfThreads: default: same as number of cpus (threads); used for multiprocessing lib;

resultsFile: if provided, output is immediately written to resultsFile during processing

clusterHostNames: list of hostnames, e.g. clusterHostNames=[‘123.124.125.126’,’123.124.125.127’] providing a list of strings with IP addresses or host names, see dispy documentation. If list is non-empty and useMultiProcessing==True and dispy is installed, cluster computation is used; NOTE that cluster computation speedup factors shown are not fully true, as they include a significant overhead; thus, only for computations which take longer than 1-5 seconds and for sufficient network bandwith, the speedup is roughly true

useDispyWebMonitor: if given in **kwargs, a web browser is startet in case of cluster computation to manage the cluster during computation

useMPI: if given in **kwargs and set True, and if Python package mpi4py is installed, mpi parallelization is used; for hints see parameterVariationExample.py
output:

returns values containing the results according to parameterList
notes:

options are passed from Parametervariation
example:

def PF(parameterSet):
    #in reality, value will be result of a complex exudyn simulation:
    value = sin(parameterSet['mass']) * parameterSet['stiffness']
    return value
values=ProcessParameterList(parameterFunction=PF,
                            parameterList=[{'m':1, 's':100},
                                          {'m':2, 's':100},
                                          {'m':3, 's':100},
                                          {'m':1, 's':200},
                                          {'m':2, 's':250},
                                          {'m':3, 's':300},
                                          ], useMultiProcessing=False )

Function: ParameterVariation

ParameterVariation(parameterFunction, parameters, useLogSpace = False, debugMode = False, addComputationIndex = False, useMultiProcessing = False, showProgress = True, parameterFunctionData = {}, clusterHostNames = [], numberOfThreads = None, resultsFile = '', **kwargs)

function description:

calls successively the function parameterFunction(parameterDict) with variation of parameters in given range; parameterDict is a dictionary, containing the current values of parameters,

e.g., parameterDict=[‘mass’:13, ‘stiffness’:12000] to be computed and returns a value or a list of values which is then stored for each parameter
input:

parameterFunction: function, which takes the form parameterFunction(parameterDict) and which returns any values that can be stored in a list (e.g., a floating point number)

parameters: given as a dictionary, consist of name and tuple of (begin, end, numberOfValues) same as in np.linspace(…), e.g. ‘mass’:(10,50,10), for a mass varied from 10 to 50, using 10 steps OR a list of values [v0, v1, v2, …], e.g. ‘mass’:[10,15,25,50]

useLogSpace: (optional) if True, the parameters are varied at a logarithmic scale, e.g., [1, 10, 100] instead linear [1, 50.5, 100]

debugMode: if True, additional print out is done

addComputationIndex: if True, key ‘computationIndex’ is added to every parameterDict in the call to parameterFunction(), which allows to generate independent output files for every parameter, etc.

useMultiProcessing: if True, the multiprocessing lib is used for parallelized computation; WARNING: be aware that the function does not check if your function runs independently; DO NOT use GRAPHICS and DO NOT write to same output files, etc.!

showProgress: if True, shows for every iteration the progress bar (requires tqdm library)

resultsFile: if provided, output is immediately written to resultsFile during processing

numberOfThreads: default(None): same as number of cpus (threads); used for multiprocessing lib;

parameterFunctionData: dictionary containing additional data passed to the parameterFunction inside the parameters with dict key ‘functionData’; use this e.g. for passing solver parameters or other settings

clusterHostNames: list of hostnames, e.g. clusterHostNames=[‘123.124.125.126’,’123.124.125.127’] providing a list of strings with IP addresses or host names, see dispy documentation. If list is non-empty and useMultiProcessing==True and dispy is installed, cluster computation is used; NOTE that cluster computation speedup factors shown are not fully true, as they include a significant overhead; thus, only for computations which take longer than 1-5 seconds and for sufficient network bandwith, the speedup is roughly true

useDispyWebMonitor: if given in **kwargs, a web browser is started in case of cluster computation to manage the cluster during computation

useMPI: if given in **kwargs and set True, and if Python package mpi4py is installed, mpi parallelization is used; for hints see parameterVariationExample.py
output:

returns [parameterList, values], containing, e.g., parameterList={‘mass’:[1,1,1,2,2,2,3,3,3], ‘stiffness’:[4,5,6, 4,5,6, 4,5,6]} and the result values of the parameter variation accoring to the parameterList,

values=[7,8,9 ,3,4,5, 6,7,8] (depends on solution of problem …, can also contain tuples, etc.)
example:

if __name__ == '__main__':
    ParameterVariation(parameterFunction=Test,
                       parameters={'mass':(1,10,10), 'stiffness':(1000,10000,10)},
                       useMultiProcessing=True)