Analysis pipelines¶
One of the ixpeobssim design goals since the very beginning of the code development was to allow the user to develop simulation and analysis pipelines with the minimum possible effort.
The basic technology for developing pipelines is the
ixpeobssim.core.pipeline
module.
Application wrappers¶
Each ixpeobssim application, see the Application reference page, is wrapped in the
ixpeobssim.core.pipeline
module so that it can be called from within
a generic Python script with the exact same arguments that one would
pass from command line.
Therefore the shell command
xpobssim --configfile config/toy_point_source.py --duration 10000
should in principle be exactly equivalent to the Python snippet
import ixpeobssim.core.pipeline as pipeline
file_list = pipeline.xpobssim(configfile='config/toy_point_source.py', duration=10000)
By design all the command-line switches that can be passed when any of the
ixpeobssim applications is invoked in the shell can be passed as keyword
arguments to the corresponding Python wrapper in
ixpeobssim.core.pipeline
. Easy, isn’t it?
Chaining application calls¶
ixpeobssim Python wrappers typically return the list of all the files that the function call has created (or, more precisely, a list of strings, each one representing the path to a specific output file). This makes it easy to chain applications one after the other, which is a typical use case:
import ixpeobssim.core.pipeline as pipeline
evt_file_list = pipeline.xpobssim(configfile='config/toy_disk.py', duration=10000)
cmap_file_list = pipeline.xpbin(*evt_file_list, algorithm='CMAP')
count_map = xBinnedMap.from_file_list(cmap_file_list)
count_map.plot()
While this works as advertised, when dealing with more complex examples it is often the case that one might want to run specific pieces of the pipeline independently from the others. In this case passing around file lists with the mechanism shown in the example above doesn’t really make sense, as each step relies on the fact that all the previous ones did run through.
An alternative possibility that the ixpeobssim pipeline framework offers is based on the fact that file paths are typically constructed by adding a suffix to a base name, which in turn coincides with the name of the source model being simluated and anlyzed.
import ixpeobssim.core.pipeline as pipeline
pipeline.setup(model='toy_disk')
pipeline.xpobssim(configfile='config/toy_disk.py', duration=10000)
file_list = pipeline.file_list()
pipeline.xpbin(*file_list, algorithm='CMAP')
file_list = pipeline.file_list('cmap')
xBinnedMap.from_file_list(file_list)
count_map.plot()
The reader is referred to the [github]/ixpeobssim/examples/toy_periodic_source.py example for a somewhat advanced illustration of the file-list mechanism implemented in the ixpeobssim pipeline.
Pipeline rc parameters¶
The ixpeobssim pipeline is implemented as a series of stand-alone methods, and its state is controlled by a look-up dictionary of global parameters that can be effectively used to exchange informations.
Run-commands parameters are set with the setup()
method and retrieved via
the param()
method
import ixpeobssim.core.pipeline as pipeline
pipeline.setup(model=toy_disk)
print(pipeline.param('model'))
The model
rc param plays a peculiar role, in that under normal conditions
it can be used as a helper to resolve and create file paths. Once the model
parameter is specied one can run, e.g., ixpeobssim without specifying the path
to the configuration file, assuming you want to use the one in the default
folder. The example above can therefore be recasted as
import ixpeobssim.core.pipeline as pipeline
pipeline.setup(model='toy_disk')
pipeline.xpobssim(duration=10000)
file_list = pipeline.file_list()
pipeline.xpbin(*file_list, algorithm='CMAP')
file_list = pipeline.file_list('cmap')
xBinnedMap.from_file_list(file_list)
count_map.plot()
Pipelines in action¶
The ixpeobssim.core.pipeline
module provides a bootstrap function
that should be the preferred way, in practice, to create a simulation and
analysis pipeline. A minimal pipeline example will typically look like
import ixpeobssim.core.pipeline as pipeline
def run():
"""Do something useful.
"""
# Put implementation here
pass
if __name__ == '__main__':
pipeline.bootstrap_pipeline('toy_model')
In this case the boostrap function set the model name for the pipeline, create a custom option parser that allows to control from command line the relevant options, and parse the command-line options.
You can run any of the pipelines in the [github]/ixpeobssim/examples folder
with the --help
option to see what the boostrap function makes available,
but in a nutshell this will allow you to
execute a specific method in your pipeline definition;
save the plots to file;
run in batch.
Note that the bootstrap function includes call to the routines showing the plots and saving them—i.e., you should not call the plt.save() or plt.show() methods explicitely.