Filtering event lists¶
ixpeobssim
provides facilities to filter event lists in different flavors,
selecting in time, phase, energy and position in the sky. Or work-horse
application in this respect is xpeselect.
...
usage: xpselect.py [-h] [--suffix SUFFIX] [--tmin TMIN] [--tmax TMAX]
[--tinvert {True,False}] [--phasemin PHASEMIN]
[--phasemax PHASEMAX] [--phaseinvert {True,False}]
[--emin EMIN] [--emax EMAX] [--einvert {True,False}]
[--ra RA] [--dec DEC] [--rad RAD] [--innerrad INNERRAD]
[--regfile REGFILE] [--reginvert {True,False}]
[--mask MASK] [--mcsrcid MCSRCID] [--mc {True,False}]
[--ltimeupdate {True,False}] [--ltimealg {LTSUM,LTSCALE}]
[--overwrite {True,False}]
filelist [filelist ...]
Basic data selection interface. This utility can process IXPE event files
(i.e., photon lists) and apply a generic selection on sky-position, time,
phase, energy, etc., producing smaller event files. In conjunction with
xpbin this is handy to, e.g., energy- or phase-resolved polarization
measurements.
xpselect supports spatial selections from a ds9 region file thorough the
--regfile command-line switch.
positional arguments:
filelist path(s) to the input file(s)
optional arguments:
-h, --help show this help message and exit
--suffix SUFFIX suffix for the output files (default: None)
--tmin TMIN minimum time in s (default: None)
--tmax TMAX maximum tims in s (default: None)
--tinvert {True,False}
invert the time selection (default: False)
--phasemin PHASEMIN minimum phase for periodic sources (default: None)
--phasemax PHASEMAX maximum phase for periodic sources (default: None)
--phaseinvert {True,False}
invert the phase selection (default: False)
--emin EMIN minimum energy in keV (default: None)
--emax EMAX maximum energy in keV (default: None)
--einvert {True,False}
invert the energy selection (default: False)
--ra RA RA of acceptance cone in decimal degrees (default:
None)
--dec DEC Dec of acceptance cone in decimal degrees (default:
None)
--rad RAD ROI radius in arcminutes (default: None)
--innerrad INNERRAD ROI inner radius in arcminutes (for selecting annuli)
(default: None)
--regfile REGFILE path to a ds9 region file (default: None)
--reginvert {True,False}
invert the ds9 region file selection (default: False)
--mask MASK path to .npy file with a saved boolean event mask
(default: None)
--mcsrcid MCSRCID the Monte Carlo source ID to select (default: [])
--mc {True,False} use Monte Carlo information (default: False)
--ltimeupdate {True,False}
update the livetime-related keywords in the output
files (default: False)
--ltimealg {LTSUM,LTSCALE}
algorithm to be used to update the livetime (default:
LTSCALE)
--overwrite {True,False}
overwrite existing output files (default: True)
Note
New in version 22.0.0.
xpeselect was largely refactored in ixpeobssim version
22.0.0, mainly in response to
issue #169.
If you are using xpselect
and lagging behind the ixpeobssim
version,
you should definitely consider upgrading.
At the time of writing, the XSELECT FTOOL is being updated to support IXPE, with a new HEASOFT release expected in February, 2022, and it is conceivable that xpeselect will be modified in the future to avoid overlap in functionality with the tools officially supported by HEASARC. In a nutshell: we shall try and strip off xpselect anything that will be provided out off the shelf by XSELECT.
The basic idea is that filtered event lists should be functionally identical to their parents level-2 files and they should, at least in principle, inter-operate with all the analysis tools in exactly the same fashion. If that is not the case, it is likely a bug :-)
Selecting in time or phase¶
Selections in time or phase are achieved by means of the --tmin/max
and
--phasemin/max
command-line switches. For the sake of making it easier to
keep track of the livetime and time-related header keywords when selecting in
time and phase, xpselect
performs some minimal validation on the input
values of the command line switches, and will exit with an error message if
any of the conditions below is not met:
TSTART <= TMIN <= TSTOP
TSTART <= TMAX <= TSTOP
TMAX > TMIN
0. <= PHASEMIN <= 1.
0. <= PHASEMAX <= 1.
Warning
xpeselect does not currently support concurrent selections on time and phase, and there is no specific plan to add this functionality, unless proved to be important in practice. In other words, you can select an arbitrary time or phase interval, but not at the same time.
Chaining multiple xpeselect calls in time or phase is also discouraged as, depending on the exact setup, this makes keeping track of the livetime extremely difficult. This should not be a limitation for any real analysis.
It goes without saying that, in order to be able to select in phase, you must
run the xpphase tool first, in order to have the PHASE
column added to the event list.
Propagating the livetime¶
Note
New in version 22.0.0.
As of version xpeselect provides optional command-line switches to propagate the relevant time-related header keywords to the filtered event lists. We are fully aware that the treatment is approximate, at the limit of naiveness, and we fully anticipate it will undergo modifications as we gain experience with flight data.
Warning
The new infrastructure for the livetime correction is only suitable to experiment with simulated data, and does not yet inter-operate with filtered level-2 flight data. This is the reason why the correction is disabled by default.
xpeselect tries to do a decent job in keeping track of the livetime when selecting in time or phase. Unfortunately the matter is not completely trivial, and a short discussion is in order, here.
IXPE event lists provide a LIVETIME
column encapsulating
the livetime from the previous event in microseconds, or the total time
in which the trigger was enabled from the end of the readout of the previous
event from the trigger time of the current event. Under normal conditions the
sum of the LIVETIME
column should be equal to the values of the LIVETIME
and EXPOSURE
header keywords up to numerical rounding errors.
The default mean for xpeselect to re-calculate the livetime, when
time or phase selections are applied is to simply sum the LIVETIME
column
over the events surviving the selections. This is what happens when the
--ltimealg
command-line switch is set to LTSUM
, and the thing works as
advertised when the selected events are contiguous, e.g., when a simple time
selections is applied.
When selecting in phase the thing gets more complicated, as in some circumstances
(i.e., when the average time distance between subsequent events is larger than
the period of the ephemeris we use for folding) the LIVETIME
value for any
particular event will in general be referred to an event in a different phase bin.
In order to cope (approximately) with this situation, xpeselect
provide an alternative mean of propagating the livetime, that is enabled by
using the --ltimealg LTSCALE
command-line switch. In a nutshell, we first
calculate the average dead time per event for the original event list by simply
dividing the difference between the ONTIME
and LIVETIME
header keywords
by the total number of events. (This is typically in the ms ball-park.) This
average dead time per event can in turn be used to calculate a total scaled livetime
within the selection, by simply multiplying it by the number of events within
the selection.
Warning
The livetime scaled we just described is only valid in the limit that the average dead time per event in the original photon list is representative of the average dead time per event within the selection. Since the dead time per event depends on the size of the region of interest of the event, which in turns depends on the energy, this assumption might not be accurate in the presence of strong spectral variations as a function of the pulse phase. (In practice, all the dependences are mild, and this should not be an issue in most practical situations.)
In many conditions, and particularly for selections in time, the default
LTSUM
mechanism will work fine. As a rule of thumb, you can use LTSUM
when selecting in time and LTSCALE
when selecting in phase, but be advised
that there might be cases when one need to examine more closely the situation and
opt for an ad-hoc solution.
For completeness, when we select in time, the header keywords in the releavant extensions are update as follows
TSTART -> max(TSTART, TMIN)
TSTOP -> min(TSTOP, TMAX)
ONTIME -> TSTOP - TSTART
and when we select in phase we have instead.
TSTART -> TSTART (unchanged)
TSTOP -> TSTOP (unchanged)
ONTIME -> `ONTIME * selected_phase_fraction
In all cases, the livetime-related columns are updated according to:
EXPOSURE -> modified livetime (LTSUM or LTSCALE)
LIVETIME -> same as EXPOSURE
DEADC -> EXPOSURE / ONTIME