run data

class module0_flow.resources.run_data.RunData(**params)

Bases: h5flow.core.H5FlowResource

Provides access to run-level data:

charge_data_file: charge raw data file source name

light_data_file: light raw data file source name

e_field: TPC electric field in kV/mm

light_nsamples: light system number of samples

charge_threshold: charge system global thresholds (either high or medm)

is_mc: boolean flag, True if file was produced by simulation

Parameters:

path: str, path to run data within file
runlist_file: str, path to runlist file containing run meta data
defaults: dict, key value pairs of data_name: data_value to use if lookup fails

To access data, use the corresponding RunData property, e.g.:

resources['RunData'].e_field

A runlist file is required the first time the resource is included in a workflow. For subsequent workflows, data is stored and loaded directly from the hdf5 file.

Example config:

resources:
    - classname: RunData
      params:
        path: 'run_info'
        runlist_file: 'runlist.txt'

Run list file specification:

Whitespace-delimited text file

First line of the text file contains the column names: e_field, charge_filename, light_filename, charge_thresholds, light_samples, in any order.

The remainder of the file consists of whitespace separated data corresponding to the column names.

e_field run list file units are V/cm.

property charge_filename: Base string for run file with charge data

property charge_thresholds: Charge threshold setting, either 'high' or 'medm'

class_version = 0.1.1

property crs_ticks: Charge readout system clock cycle (us)

default_path = run_info

default_runlist_file = runlist.txt

property e_field: TPC electric field in kV/mm

init(source_name)

property is_mc: Simulation flag, True if file comes from simulation

property light_filename: Base string for run file with light data

property light_samples: Number of light waveform samples per trigger

property lrs_ticks: Light readout system clock cycle (us)

required_attr = ('charge_filename', 'light_filename', 'e_field', 'light_samples', 'charge_thresholds', 'is_mc', 'crs_ticks', 'lrs_ticks')

source_filename_columns = ('charge_filename', 'light_filename')

module0_flow.resources.run_data.assert_compat_version(version, other_version)

Raises an AssertionError if the requested version (version) is not compatible with other_version. Compatibility requires:

major versions are eqal

minor version of version is less than or equal to other_version

Parameters:

version – version str formatted 'major.minor.subminor'
other_version – version str formatted 'major.minor.subminor'

Returns:

None

LAr data

class module0_flow.resources.lar_data.LArData(**params)

Bases: h5flow.core.H5FlowResource

Provides helper functions for calculating properties of liquid argon. Values will be saved and/or loaded from metadata within the output file.

Requires RunData resource within workflow.

Parameters:

path: str, path to stored lar data within file
electron_mobility_params: list, electron mobility calculation parameters, see LArData.electron_mobility

Provides:

mode: choice of vdrift model; 1. LArSoft, 2. BNL mobility measurement
temp: LAr temperature in K
v_drift: electron drift velocity in mm/us
ionization_w: ionization W-value
density: LAr density
ionization_recombination(dedx): helper function for calculating recombination factor
electron_lifetime(unix_ts): helper function for looking up electron lifetime at a given timestamp
A: atomic mass number for atmospheric Argon
Z: atomic number for Argon
radiation_length: radiation length in argon

Example usage:

from h5flow.core import resources

resources['LArData'].v_drift

Example config:

resources:
    - classname: LArData
      params:
        path: 'lar_info'

property A: Fixed value of 39.948

property Z: Fixed value of 18

class_version = 0.3.0

default_electron_lifetime = 2200.0

default_electron_lifetime_file

default_electron_mobility_params

default_mode = 1

default_path = lar_info

default_temp

property density: Liquid argon density in g/mm^3. Fixed value of 0.00138

drift_speed_helper(params, e, t=87.17): Help function for drift speed calculation w.r.t LAr temperature and electric field.

electron_lifetime(unix_ts)

Convert the run unix timestamp into an electron lifetime value

Returns:: lifetime, (lifetime_lower_bound, lifetime_upper_bound)

electron_lifetime_data_dtype

electron_mobility(e, t=87.17)

Calculation of the electron mobility w.r.t temperature and electric field.

References:

https://lar.bnl.gov/properties/trans.html (summary)
https://doi.org/10.1016/j.nima.2016.01.073 (parameterization)

Parameters:

e – electric field in kV/mm
t – temperature in K

Returns:

electron mobility in mm^2/kV/us

init(source_name)

ionization_recombination(dedx)

Calculate charge recombination factor using Birks Model with parameters:

A = 0.8

K = 0.0486 (units = g/(MeV cm^2) kV/cm)

Defined by:

R_i = dQ * W_i / dE

property ionization_w: Ionization W-value in LAr in keV/e-. Fixed value of 0.0236

property mode: choices for vdrift models

property radiation_length: 19.55 g cm^-2 / density

scintillation_recombination(dedx)

Calculate photon recombination factor using Birks Model (see ionization_recombination()). Defined by:

R_s = dQ * W_s / dE

property scintillation_w: Scintillation W-value in LAr in keV/photon. Fixed value of 0.0195

property temp: LAr temperature in K

property v_drift

Electron drift velocity in mm/us

mode: 1. LArSoft (commonly used, a lot hard codded numbers here)

Ref: https://internal.dunescience.org/doxygen/lardataalg_2lardataalg_2DetectorInfo_2DetectorPropertiesStandard_8cxx_source.html

BNL mobility measurement (see function electron_mobility())

module0_flow.resources.lar_data.assert_compat_version(version, other_version)

Raises an AssertionError if the requested version (version) is not compatible with other_version. Compatibility requires:

major versions are eqal

minor version of version is less than or equal to other_version

Parameters:

version – version str formatted 'major.minor.subminor'
other_version – version str formatted 'major.minor.subminor'

Returns:

None

geometry

class module0_flow.resources.geometry.Geometry(**params)

Bases: h5flow.core.H5FlowResource

Provides helper functions for looking up geometric properties

Parameters:

path: str, path to stored geometry data within file
network_agnostic: bool, optional, ignore the (io_channel % 4), useful if the io_channel mapping changes run-to-run (True == ignore)
n_io_channels_per_tile: int, optional, only used with network_agnostic == True, sets the number of io channels to have same geometry info
crs_geometry_file: str, path to yaml file describing charge readout system geometry
lrs_geometry_file: str, path to yaml file describing light readout system

Provides (for charge geometry):

pixel_pitch: pixel pitch in mm
pixel_xy: lookup table for pixel (x,y) coordinates
tile_id: lookup table for io channel tile ids
anode_z: lookup table for tile z coordinate
drift_dir: lookup table for tile drift direction (±z)
regions: drift regions minimum and maximum corners of TPC drift regions
in_fid(): helper function for defining fiducial volumes
get_z_coordinate(): helper function for converting drift time to z coordinate

Provides (for light geometry):

tpc_id: lookup table for TPC number for light detectors
det_id: lookup table for detector number from adc, channel id
det_bounds: lookup table for detector minimum and maximum corners light detectors
solid_angle(): helper function for determining the solid angle of a given detector

Example usage:

from h5flow.core import resources

resources['Geometry'].pixel_pitch

Example config:

resources:
    - classname: Geometry
      params:
        path: 'geometry_info'
        crs_geometry_file: 'data/module0_flow/multi_tile_layout-2.2.16.yaml'
        lrs_geometry_file: 'data/module0_flow/light_module_desc-0.0.0.yaml'

property anode_z

Lookup table for anode z coordinate, usage:

resource['Geometry'].anode_z[(tile_id,)]

class_version = 0.2.0

default_crs_geometry_file = -

default_lrs_geometry_file = -

default_n_io_channels_per_tile = 4

default_network_agnostic = False

default_path = geometry_info

property det_bounds

Lookup table for detector min and max xyz coordinate, usage:

resource['Geometry'].det_bounds[(tpc_id, det_id)]

property det_id

Lookup table for detector id within a TPC, usage:

resource['Geometry'].det_id[(adc_index, channel_index)]

property drift_dir

Lookup table for drift direction, usage:

resource['Geometry'].drift_dir[(tile_id,)]

get_z_coordinate(io_group, io_channel, drift)

Convert a drift distance on a set of (io group, io channel) to a z-coordinate.

Parameters:

io_group – io group to calculate z coordinate, shape: (N,)
io_channel – io channel to calculate z coordinate, shape: (N,)
drift – drift distance [mm], shape: (N,)

Returns:

z coordinate [mm], shape: (N,)

in_fid(xyz, cathode_fid=0.0, field_cage_fid=0.0, anode_fid=0.0)

Check if xyz point is contained in the specified fiducial volume

Parameters:

xyz – point to check, array shape: (N,3)
cathode_fid – fiducial boundary for cathode and anode, float, optional
field_cage_fid – fiducial boundary for field cage walls, float, optional

Returns:

boolean array, shape: (N,), True indicates point is within fiducial volume

init(source_name)

load_geometry()

property pixel_pitch: Pixel pitch in mm

property pixel_xy

Lookup table for pixel xy coordinate, usage:

resource['Geometry'].pixel_xy[(io_group,io_channel,chip_id,channel_id)]

property regions: List of active volume extent for each TPC, each shape: (2,3) representing the minimum xyz coordinate and the maximum xyz coordinate

solid_angle(xyz, tpc_id, det_id)

Calculate the solid angle of a rectangular detector det_id in TPC tpc_id as seen from the point xyz, under the assumption that the detector is oriented along the drift direction

Note: this method does not consider cathode / field cage visibilty.

Parameters:

xyz – array shape: (N,3)
tpc_id – array shape: (M,)
det_id – array shape: (M,)

Returns:

array shape: (N, M)

property tile_id

Lookup table for tile id, usage:

resource['Geometry'].tile_id[(io_group,io_channel)]

property tpc_id

Lookup table for TPC id, usage:

resource['Geometry'].tpc_id[(adc_index, channel_index)]

class module0_flow.resources.geometry.LUT(dtype, *min_max_keys, default=None, shape=None)

Bases: object

Creates a lookup table that can be used to quickly access data based on tuples of integers. Works best if keys are contiguous within each position of the tuple. E.g.:

key0 = [0,1,2]
key1 = [30,31,32]

is 10x more memory efficient than:

key0 = [10,20,30]
key1 = [300,310,320]

Initialize with tuples of min and max values for each of the used key values:

key0 = [0,1,2,3]
key1 = [5,6,7,8]
shape = (2,)
dtype = 'f8'
lut = LUT(dtype, (min(key0), max(key0)), (min(key1), max(key1)), shape=shape)

Data can then be stored in the table using a tuple of key arrays:

lut[(key0,key1)] = np.array([[0,0],[1,1],[2,2],[3,3]])

and accessed:

lut[(key0,key1)] # np.array([[0,0],[1,1],[2,2],[3,3]])

A default value should be set for keys that are not found in the table:

lut.default = np.array([-1,-1])
lut[([0],[0])] # np.array([-1,-1])

clear(*keys)

Remove stored value for specified keys

Parameters:

*keys –

arrays of key values, shape: (N,)

compress(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: compressed array of entry data that has been filled

property default: Default value to return if key not found in table. Datatype is same as lookup table

static from_array(meta_arr, data_arr)

Convert an array-based representation of a lookup table (as returned by LUT.to_array()) to a LUT object.

Parameters:

meta_arr – array containing meta data
data_arr – array containing lookup table data

Returns:

LUT object

hash(*keys)

Generate a hash index from key value arrays

Parameters:

*keys –

arrays of each key value, shape: (N,)

Returns:

array of hash index, shape: (N,)

keys()

Return existing keys

Returns:: tuple of arrays, each shape: (N,)

max(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: maximum value of compressed LUT data

min(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: minimum value of compressed LUT data

property nbytes

Returns:: number of bytes used by underlying arrays

to_array()

Generate an array-based representation of a LUT object. Returns two arrays. The first has a datatype:

dtype([('min_max_keys', 'i8', ({nkeys}, 2)), ('default', {data_dtype}, {data_shape})])

and shape: (1,). This contains meta-data needed to reconstruct the LUT hashing function and the default value. The second has a datatype:

dtype([('data', {dtype}, {data_shape}), ('filled', bool, {data_shape})])

and shape: (N,). This represents the data stored in the lookup table.

Returns:: tuple of meta-array and data-array as described above

module0_flow.resources.geometry.assert_compat_version(version, other_version)

Raises an AssertionError if the requested version (version) is not compatible with other_version. Compatibility requires:

major versions are eqal

minor version of version is less than or equal to other_version

Parameters:

version – version str formatted 'major.minor.subminor'
other_version – version str formatted 'major.minor.subminor'

Returns:

None

module0_flow.resources.geometry.read_lut(data_manager, path, name=None)

module0_flow.resources.geometry.write_lut(data_manager, path, lut, name=None)

Light geometry description file

The Geometry resource uses a YAML config file to describe the existing light detector and their locations within the module. This YAML file must have the following keys:

format_version: version number (major.minor.fix) of the YAML file formatting

geometry_version: version number (major.minor.fix) of the geometry

geom: dictionary describing the shape of each detector module (described in more detail below)

det_center: dictionary describing the position of each detector module (described in more detail below)

tpc_center: dictionary for the center coordinate of each TPC (<TPC #>: [x,y,z])

det_geom: dictionary for detector shape lookup (<DET #>: <GEOM #>), assumed to be the same for each TPC

det_adc: dictionary for the ADC index used for each detector (<TPC #>: <DET #>: <ADC INDEX>)

det_chan: dictionary for the channel indices used for each detector (<TPC #>: <DET #>: [<CHANNEL #s>])

Generally each detector module is assigned two numbers: a TPC # and a DET #. The first refers to the drift region (2 per module) that the detector module sees. The second refers the position along the field cage wall the detector inhabits - with a convention of the (-x, -y) corner being DET # == 0 and increasing with with y position. Each detector module can consist of more than one SiPM, as reflected in the det_chan, and will be treated as though they have the same position and acceptance.

The geom key describes each different light detector geometry, assuming they are rectangular. The format of this dictionary is:

<DET GEOM #>: {
    min: [x,y,z],
    max: [x,y,z]
}

The det_center key describes the position and geometry of each light detector. The format of this dictionary is:

<DET #>: {
    geom: <DET GEOM #>,
    center: [x,y,z]
}

particle data

class module0_flow.resources.particle_data.ParticleData(**params)

Bases: h5flow.core.H5FlowResource

Provides helper functions for calculating and accessing particle properties. Range tables will be saved and/or loaded to/from metadata within the output file.

Requires LArData resource within workflow.

Parameters:

path: str, path to stored particle data within file
muon_range_table_path: str, path to PDG text file of muon range in LAr
proton_range_table_path: str, path to NIST text file of proton range in LAr

Provides:

muon_range_table: Range, kinetic energy, and <dE/dx> for muons in LAr
proton_range_table: Range, kinetic energy, and <dE/dx> for protons in LAr
landau_width: 1-sigma width of Landau dE/dx distribution in LAr
landau_peak: MPV of Landau dE/dx distribution in LAr
{}_mass: for proton (p), neutron (n), muon (mu), electron (e), pion (pi), pi0 (pi0)

Example usage:

from h5flow.core import resources

resources['ParticleData'].muon_range_table['range']

Example config:

resources:
    - classname: ParticleData
      params:
        path: 'particle_info'

class_version = 0.0.0

default_muon_range_table_path = PDG_muon_range_table_Ar.txt

default_path = particle_info

default_proton_range_table_path = NIST_proton_range_table_Ar.txt

e_mass

init(source_name)

landau_peak(t, mass, dx): Moyal peak location for Landau dE/dx distribution in LAr

landau_width(t, mass, dx): Moyal scale factor for Landau dE/dx width in LAr

static load_nist_range_table(path)

Loads particle range, kinetic energy, and dE/dx from a NIST text file [https://physics.nist.gov/PhysRefData/Star/Text/PSTAR-t.html].

Parameters:: path – path to range table file
Returns:: dict with keys range, t, dedx

static load_pdg_range_table(path)

Loads particle range, kinetic energy, and dE/dx from a PDG text file [https://pdg.lbl.gov/2021/AtomicNuclearProperties/].

Parameters:: path – path to range table file
Returns:: dict with keys range, t, dedx

mcs_angle(t, mass, dx): Multiple coulomb scattering characteristic angle

mu_mass

property muon_range_table: Range v. kinetic energy v. dE/dx for a muon in LAr. dict with keys: range, t, and dedx

n_mass

p_mass

pi0_mass

pi_mass

property proton_range_table: Range v. kinetic energy v. dE/dx for a proton in LAr. dict with keys: range, t, and dedx

module0_flow.resources.particle_data.assert_compat_version(version, other_version)

Raises an AssertionError if the requested version (version) is not compatible with other_version. Compatibility requires:

major versions are eqal

minor version of version is less than or equal to other_version

Parameters:

version – version str formatted 'major.minor.subminor'
other_version – version str formatted 'major.minor.subminor'

Returns:

None

disabled channels

class module0_flow.resources.disabled_channels.DisabledChannels(**params)

Bases: h5flow.core.H5FlowResource

Provides helper functions for identifying the positions of disabled channels.

Parameters:

path: str, path to stored geometry data within file
disabled_channels_list: str, path to file specifying channels that are disabled
missing_asic_list: str, path to file specifying coordinates disabled coordinates not in geometry file

Provides:

disabled_xy: x,y coordinates of all disabled channels
disabled_channel_lut: lookup table to find if a pixel x,y coordinate is disabled
is_active(): helper function for determining if a 3D point in in an active region

Example usage:

from h5flow.core import resources

resources['DisabledChannels'].disabled_channel_lut[(io_group,x,y)]

Example config:

resources:
    - classname: DisabledChannels
      params:
        path: 'disabled_channels'
        disabled_channels_list: 'module0-run1-selftrigger-disabled-list.json'
        missing_asic_list: 'module0-network-absent-ASICs.json'

class_version = 0.0.0

default_path = disabled_channels

property disabled_channel_lut

property disabled_xy

init(source_name)

is_active(xyz)

Lookup a specific position to determine if it would fall onto an active pixel

Parameters:: xyz – 3D position shape: (..., 3)
Returns:: boolean array with True == active, shape: (...,)

static load_disabled_channels_lut(disabled_channels_list=None, missing_asic_list=None)

Loads a disabled channels lookup-table from the json formatted filenames:

disabled_channels_list
missing_asic_list

disabled_channels_list contains chip-key: [channel_id] pairs of disabled channels that are defined within the geometry, but should be considered as disabled. The Geometry resource is used to find the xy locations of these pixels.

missing_asic_list contains io_group: [[x,y], ...] pixel positions that should be considered as disabled regions.

Creates a boolean lookup table with keys of (io_group, int(pixel_x), int(pixel_y)) to determine if a given pixel position falls onto a disabled channel.

Returns:: tuple of boolean module0_flow.util.lut.LUT and list of xy coordinates for each disabled channel

class module0_flow.resources.disabled_channels.LUT(dtype, *min_max_keys, default=None, shape=None)

Bases: object

Creates a lookup table that can be used to quickly access data based on tuples of integers. Works best if keys are contiguous within each position of the tuple. E.g.:

key0 = [0,1,2]
key1 = [30,31,32]

is 10x more memory efficient than:

key0 = [10,20,30]
key1 = [300,310,320]

Initialize with tuples of min and max values for each of the used key values:

key0 = [0,1,2,3]
key1 = [5,6,7,8]
shape = (2,)
dtype = 'f8'
lut = LUT(dtype, (min(key0), max(key0)), (min(key1), max(key1)), shape=shape)

Data can then be stored in the table using a tuple of key arrays:

lut[(key0,key1)] = np.array([[0,0],[1,1],[2,2],[3,3]])

and accessed:

lut[(key0,key1)] # np.array([[0,0],[1,1],[2,2],[3,3]])

A default value should be set for keys that are not found in the table:

lut.default = np.array([-1,-1])
lut[([0],[0])] # np.array([-1,-1])

clear(*keys)

Remove stored value for specified keys

Parameters:

*keys –

arrays of key values, shape: (N,)

compress(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: compressed array of entry data that has been filled

property default: Default value to return if key not found in table. Datatype is same as lookup table

static from_array(meta_arr, data_arr)

Convert an array-based representation of a lookup table (as returned by LUT.to_array()) to a LUT object.

Parameters:

meta_arr – array containing meta data
data_arr – array containing lookup table data

Returns:

LUT object

hash(*keys)

Generate a hash index from key value arrays

Parameters:

*keys –

arrays of each key value, shape: (N,)

Returns:

array of hash index, shape: (N,)

keys()

Return existing keys

Returns:: tuple of arrays, each shape: (N,)

max(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: maximum value of compressed LUT data

min(sel=tuple())

Parameters:: sel – optional, for multi-dimensional LUT data apply this selection to the returned data
Returns:: minimum value of compressed LUT data

property nbytes

Returns:: number of bytes used by underlying arrays

to_array()

Generate an array-based representation of a LUT object. Returns two arrays. The first has a datatype:

dtype([('min_max_keys', 'i8', ({nkeys}, 2)), ('default', {data_dtype}, {data_shape})])

and shape: (1,). This contains meta-data needed to reconstruct the LUT hashing function and the default value. The second has a datatype:

dtype([('data', {dtype}, {data_shape}), ('filled', bool, {data_shape})])

and shape: (N,). This represents the data stored in the lookup table.

Returns:: tuple of meta-array and data-array as described above

module0_flow.resources.disabled_channels.assert_compat_version(version, other_version)

Raises an AssertionError if the requested version (version) is not compatible with other_version. Compatibility requires:

major versions are eqal

minor version of version is less than or equal to other_version

Parameters:

version – version str formatted 'major.minor.subminor'
other_version – version str formatted 'major.minor.subminor'

Returns:

None

module0_flow.resources.disabled_channels.read_lut(data_manager, path, name=None)

module0_flow.resources.disabled_channels.write_lut(data_manager, path, lut, name=None)