Behavior#

Audio#

class AudioInterface(file_paths: list[typing.Annotated[pathlib.Path, PathType(path_type='file')]], verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for writing .wav audio recordings to an NWB file.

Data interface for writing acoustic recordings to an NWB file.

Writes acoustic recordings as an AcousticWaveformSeries from the ndx_sound extension.

Parameters

file_paths (list of FilePaths) – The file paths to the audio recordings in sorted, consecutive order. We recommend using natsort to ensure the files are in consecutive order:
```
from natsort import natsorted
natsorted(file_paths)
```
verbose (bool, default: False)

display_name: Union[str, None] = 'Wav Audio'#

keywords: tuple[str] = ('sound', 'microphone')#

associated_suffixes: tuple[str] = ('.wav',)#

info: Union[str, None] = 'Interface for writing audio recordings to an NWB file.'#

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns: The JSON schema defining the metadata structure.
Return type: dict

get_metadata() → dict[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

get_timestamps() → Optional[ndarray][source]#

Retrieve the timestamps for the data in this interface.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: list[numpy.ndarray])[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float)[source]#

Align all starting times for all audio files in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_starting_time (float) – The common starting time for all temporal data in this interface. Applies to all segments if there are multiple file paths used by the interface.

set_aligned_segment_starting_times(aligned_segment_starting_times: list[float])[source]#

Align the individual starting time for each audio file in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_segment_starting_times (list of floats) – The relative starting times of each audio file (segment).

align_by_interpolation(unaligned_timestamps: ndarray, aligned_timestamps: ndarray)[source]#

Interpolate the timestamps of this interface using a mapping from some unaligned time basis to its aligned one.

Use this method if the unaligned timestamps of the data in this interface are not directly tracked by a primary system, but are known to occur between timestamps that are tracked, then align the timestamps of this interface by interpolating between the two.

An example could be a metronomic TTL pulse (e.g., every second) from a secondary data stream to the primary timing system; if the time references of this interface are recorded within the relative time of the secondary data stream, then their exact time in the primary system is inferred given the pulse times.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

unaligned_timestamps (numpy.ndarray) – The timestamps of the unaligned secondary time basis.
aligned_timestamps (numpy.ndarray) – The timestamps aligned to the primary time basis.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, stub_test: bool = False, stub_frames: int = 1000, write_as: Literal['stimulus', 'acquisition'] = 'stimulus', iterator_options: Optional[dict] = None)[source]#

Parameters

nwbfile (NWBFile) – Append to this NWBFile object
metadata (dict, optional)
stub_test (bool, default: False)
stub_frames (int, default: 1000)
write_as ({‘stimulus’, ‘acquisition’}) – The acoustic waveform series can be added to the NWB file either as “stimulus” or as “acquisition”.
iterator_options (dict, optional) – Dictionary of options for the SliceableDataChunkIterator.

Return type

NWBFile

DeepLabCut#

class DeepLabCutInterface(file_path: Annotated[pathlib.Path, PathType(path_type='file')], config_file_path: Optional[Annotated[pathlib.Path, PathType(path_type='file')]] = None, subject_name: str = 'ind1', verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for DeepLabCut datasets.

Interface for writing DLC’s output files to nwb using dlc2nwb.

Parameters

file_path (FilePath) – Path to the file output by dlc (.h5 or .csv).
config_file_path (FilePath, optional) – Path to .yml config file
subject_name (str, default: “ind1”) – The name of the subject for which the NWBFile is to be created.
verbose (bool, default: True) – Controls verbosity.

display_name: Optional[str] = 'DeepLabCut'#

keywords: tuple[str] = ('DLC', 'DeepLabCut', 'pose estimation', 'behavior')#

associated_suffixes: tuple[str] = ('.h5', '.csv')#

info: Optional[str] = 'Interface for handling data from DeepLabCut.'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns: The JSON schema for the source_data.
Return type: dict

get_metadata()[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: Union[list, ndarray])[source]#

Set aligned timestamps vector for DLC data with user defined timestamps

Parameters: aligned_timestamps (list, np.ndarray) – A timestamps vector.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, container_name: str = 'PoseEstimationDeepLabCut')[source]#

Conversion from DLC output files to nwb. Derived from dlc2nwb library.

Parameters

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict) – metadata info for constructing the nwb file (optional).
container_name (str, default: “PoseEstimationDeepLabCut”) – name of the PoseEstimation container in the nwb

FicTrac#

class FicTracDataInterface(file_path: Annotated[pathlib.Path, PathType(path_type='file')], radius: Optional[float] = None, configuration_file_path: Optional[Annotated[pathlib.Path, PathType(path_type='file')]] = None, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for FicTrac datasets.

Interface for writing FicTrac files to nwb.

Parameters

file_path (FilePath) – Path to the .dat file (the output of fictrac)
radius (float, optional) – The radius of the ball in meters. If provided the radius is stored as a conversion factor and the units are set to meters. If not provided the units are set to radians.
configuration_file_path (FilePath, optional) – Path to the .txt file with the configuration metadata. Usually called config.txt
verbose (bool, default: False) – controls verbosity. True by default.

display_name: Optional[str] = 'FicTrac'#

keywords: tuple[str] = ('fictrack', 'visual tracking', 'fictive path', 'spherical treadmill', 'visual fixation')#

associated_suffixes: tuple[str] = ('.dat',)#

info: Optional[str] = 'Interface for FicTrac .dat files.'#

timestamps_column = 21#

columns_in_dat_file = ['frame_counter', 'rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam', 'rotation_delta_error', 'rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab', 'rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam', 'rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab', 'x_pos_radians_lab', 'y_pos_radians_lab', 'animal_heading', 'movement_direction', 'movement_speed', 'forward_motion_lab', 'side_motion_lab', 'timestamp', 'sequence_counter', 'delta_timestamp', 'alt_timestamp']#

column_to_nwb_mapping = {'animal_heading': {'column_in_dat_file': ['animal_heading'], 'description': "Animal's heading direction in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesAnimalHeading'}, 'integrated_motion': {'column_in_dat_file': ['forward_motion_lab', 'side_motion_lab'], 'description': 'Integrated x/y position of the sphere in laboratory coordinates, neglecting heading.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesIntegratedMotion'}, 'movement_direction': {'column_in_dat_file': ['movement_direction'], 'description': "Instantaneous running direction of the animal in the lab coordinates. Direction inferred by the ball's rotation (roll and pitch)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementDirection'}, 'movement_speed': {'column_in_dat_file': ['movement_speed'], 'description': 'Instantaneous running speed of the animal in radians per frame.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementSpeed'}, 'position_lab': {'column_in_dat_file': ['x_pos_radians_lab', 'y_pos_radians_lab'], 'description': 'x and y positions in the lab frame in radians, inferred by integrating the rotation over time.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesPosition'}, 'rotation_cam': {'column_in_dat_file': ['rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam'], 'description': "Orientation in radians from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationCameraFrame'}, 'rotation_delta_cam': {'column_in_dat_file': ['rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam'], 'description': "Change in orientation since last frame from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationDeltaCameraFrame'}, 'rotation_delta_error': {'column_in_dat_file': ['rotation_delta_error'], 'description': "Error in rotation delta in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaError'}, 'rotation_delta_lab': {'column_in_dat_file': ['rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab'], 'description': "Change in orientation since last frame from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaLabFrame'}, 'rotation_lab': {'column_in_dat_file': ['rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab'], 'description': "Orientation in radians from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationLabFrame'}}#

spatial_series_descriptions = {'animal_heading': {'column_in_dat_file': ['animal_heading'], 'description': "Animal's heading direction in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesAnimalHeading'}, 'integrated_motion': {'column_in_dat_file': ['forward_motion_lab', 'side_motion_lab'], 'description': 'Integrated x/y position of the sphere in laboratory coordinates, neglecting heading.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesIntegratedMotion'}, 'movement_direction': {'column_in_dat_file': ['movement_direction'], 'description': "Instantaneous running direction of the animal in the lab coordinates. Direction inferred by the ball's rotation (roll and pitch)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementDirection'}, 'movement_speed': {'column_in_dat_file': ['movement_speed'], 'description': 'Instantaneous running speed of the animal in radians per frame.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementSpeed'}, 'position_lab': {'column_in_dat_file': ['x_pos_radians_lab', 'y_pos_radians_lab'], 'description': 'x and y positions in the lab frame in radians, inferred by integrating the rotation over time.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesPosition'}, 'rotation_cam': {'column_in_dat_file': ['rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam'], 'description': "Orientation in radians from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationCameraFrame'}, 'rotation_delta_cam': {'column_in_dat_file': ['rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam'], 'description': "Change in orientation since last frame from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationDeltaCameraFrame'}, 'rotation_delta_error': {'column_in_dat_file': ['rotation_delta_error'], 'description': "Error in rotation delta in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaError'}, 'rotation_delta_lab': {'column_in_dat_file': ['rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab'], 'description': "Change in orientation since last frame from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaLabFrame'}, 'rotation_lab': {'column_in_dat_file': ['rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab'], 'description': "Orientation in radians from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationLabFrame'}}#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns: The JSON schema for the source_data.
Return type: dict

get_metadata()[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None)[source]#

Parameters

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict, optional) – metadata info for constructing the nwb file.

get_original_timestamps()[source]#

Retrieve and correct timestamps from a FicTrac data file.

This function addresses two specific issues with timestamps in FicTrac data:

Resetting Initial Timestamp In some instances, FicTrac replaces the initial timestamp (0) with the system time. This commonly occurs when the data source is a video file, and OpenCV reports the first timestamp as 0. Since OpenCV also uses 0 as a marker for invalid values, FicTrac defaults to system time in that case. This leads to inconsistent timestamps like [system_time, t1, t2, t3, …]. The function corrects this by resetting the first timestamp back to 0 when a negative difference is detected between the first two timestamps.
Re-centering Unix Epoch Time If timestamps are in Unix epoch time format (time since 1970-01-01 00:00:00 UTC), this function re-centers the time series by subtracting the first timestamp. This adjustment ensures that timestamps represent the elapsed time since the start of the experiment rather than the Unix epoch. This case appears when one of the sources of data in FicTrac (such as PGR or Basler) lacks a timestamp extraction method. FicTrac then falls back to using the system time, which is in Unix epoch format.

Returns: An array of corrected timestamps, in seconds.
Return type: np.ndarray

Notes

The issue of the initial timestamp replacement appears in FicTrac 2.1.1 and earlier versions.
Re-centering is essential for timestamps in Unix epoch format as timestamps in an NWB file must be relative

to the start of the session. The heuristic here is to check if the first timestamp is larger than the length of a 10-year experiment in seconds. If so, it’s assumed that the timestamps are in Unix epoch format.

References

Issue discussion on FicTrac’s timestamp inconsistencies: rjdmoore/fictrac#29

get_timestamps()[source]#

Retrieve the timestamps for the data in this interface.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

set_aligned_timestamps(aligned_timestamps)[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time)[source]#

Align the starting time for this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_starting_time (float) – The starting time for all temporal data in this interface.

extract_session_start_time(file_path: Path, configuration_file_path: Optional[Path] = None) → Optional[datetime][source]#

Extract the session start time from a FicTrac data file or its configuration file.

The session start time is determined from the data file if the timestamps are in Unix epoch format. If not, the function defaults to extracting the date from the configuration file and assuming that the start time is midnight. If neither of these methods works, the function returns None.

The session start time, has two different meanings depending on the source of the FicTrac data: - For video file sources (.avi, .mp4, etc.), the session start time corresponds to the time when the FicTrac analysis commenced. That is, the session start time reflects the analysis time rather than the actual start of the experiment. - For camera sources (such as PGR or Basler), the session start time is either the time reported by the camera or the system time if the camera’s SDK does not provide timestamps to Fictrac. In both cases, this time is the experiment start time, barring synchronization issues.

Parameters

file_path (FilePath) – Path to the FicTrac data file.
configuration_file_path (FilePath, optional) – Path to the FicTrac configuration file. If omitted, the function defaults to searching for “fictrac_config.txt” in the same directory as the data file.

Returns

The session start time of in UTC as a datetime object. None if the session start time cannot be extracted.

Return type

datetime | None

parse_fictrac_config(file_path: Path) → dict[source]#

Parse a FicTrac configuration file and return a dictionary of its parameters.

Parameters: file_path (FilePath) – Path to the configuration file in txt format.
Returns: A dictionary where the keys are the parameter names and the values are the parameter values.
Return type: dict

LightningPose#

class LightningPoseConverter(file_path: Annotated[pathlib.Path, PathType(path_type='file')], original_video_file_path: Annotated[pathlib.Path, PathType(path_type='file')], labeled_video_file_path: Optional[Annotated[pathlib.Path, PathType(path_type='file')]] = None, image_series_original_video_name: Optional[str] = None, image_series_labeled_video_name: Optional[str] = None, verbose: bool = False)[source]#

Bases: NWBConverter

Primary conversion class for handling Lightning Pose data streams.

The converter for Lightning Pose format to convert the pose estimation data along with the original and the optional labeled video added as ImageSeries to NWB.

Parameters

file_path (FilePath) – Path to the .csv file that contains the predictions from Lightning Pose.
original_video_file_path (FilePath) – Path to the original video file (.mp4).
labeled_video_file_path (FilePath, optional) – Path to the labeled video file (.mp4).
image_series_original_video_name (string, optional) – The name of the ImageSeries to add for the original video.
image_series_labeled_video_name (string, optional) – The name of the ImageSeries to add for the labeled video.
verbose (bool, default: False) – controls verbosity. True by default.

display_name: Union[str, None] = 'Lightning Pose Converter'#

keywords: tuple[str] = ('pose estimation', 'video')#

associated_suffixes: tuple[str] = ('.csv', '.mp4')#

info: Union[str, None] = 'Interface for handling multiple streams of lightning pose data.'#

classmethod get_source_schema()[source]#

Compile input schemas from each of the data interface classes.

Returns: The compiled source schema from all data interface classes.
Return type: dict

get_conversion_options_schema() → dict[source]#

Compile conversion option schemas from each of the data interface classes.

Returns: The compiled conversion options schema from all data interface classes.
Return type: dict

get_metadata() → DeepDict[source]#

Auto-fill as much of the metadata as possible. Must comply with metadata schema.

Returns: The metadata dictionary containing auto-filled metadata from all interfaces.
Return type: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict, reference_frame: Optional[str] = None, confidence_definition: Optional[str] = None, external_mode: bool = True, starting_frames_original_videos: Optional[list[int]] = None, starting_frames_labeled_videos: Optional[list[int]] = None, stub_test: bool = False)[source]#

Add behavior and pose estimation data, including original and labeled videos, to the specified NWBFile.

Parameters

nwbfile (NWBFile) – The NWBFile object to which the data will be added.
metadata (dict) – Metadata dictionary containing information about the behavior and videos.
reference_frame (str, optional) – Description of the reference frame for pose estimation, by default None.
confidence_definition (str, optional) – Definition for the confidence levels in pose estimation, by default None.
external_mode (bool, optional) – If True, the videos will be referenced externally rather than embedded within the NWB file, by default True.
starting_frames_original_videos (list of int, optional) – List of starting frames for the original videos, by default None.
starting_frames_labeled_videos (list of int, optional) – List of starting frames for the labeled videos, by default None.
stub_test (bool, optional) – If True, only a subset of the data will be added for testing purposes, by default False.

run_conversion(nwbfile_path: Optional[Path] = None, nwbfile: Optional[NWBFile] = None, metadata: Optional[dict] = None, overwrite: bool = False, reference_frame: Optional[str] = None, confidence_definition: Optional[str] = None, external_mode: bool = True, starting_frames_original_videos: Optional[list] = None, starting_frames_labeled_videos: Optional[list] = None, stub_test: bool = False) → None[source]#

Run the full conversion process, adding behavior, video, and pose estimation data to an NWB file.

Parameters

nwbfile_path (FilePath, optional) – The file path where the NWB file will be saved. If None, the file is handled in memory.
nwbfile (NWBFile, optional) – An in-memory NWBFile object. If None, a new NWBFile object will be created.
metadata (dict, optional) – Metadata dictionary for describing the NWB file contents. If None, it is auto-generated.
overwrite (bool, optional) – If True, overwrites the NWB file at nwbfile_path if it exists. If False, appends to the file, by default False.
reference_frame (str, optional) – Description of the reference frame for pose estimation, by default None.
confidence_definition (str, optional) – Definition for confidence levels in pose estimation, by default None.
external_mode (bool, optional) – If True, the videos will be referenced externally rather than embedded within the NWB file, by default True.
starting_frames_original_videos (list of int, optional) – List of starting frames for the original videos, by default None.
starting_frames_labeled_videos (list of int, optional) – List of starting frames for the labeled videos, by default None.
stub_test (bool, optional) – If True, only a subset of the data will be added for testing purposes, by default False.

Neuralynx NVT#

class NeuralynxNvtInterface(file_path: Annotated[pathlib.Path, PathType(path_type='file')], verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for Neuralynx NVT files. NVT files store position tracking information.

Interface for writing Neuralynx .nvt files to nwb.

Parameters

file_path (FilePath) – Path to the .nvt file
verbose (bool, default: Falsee) – controls verbosity.

display_name: Union[str, None] = 'Neuralynx NVT'#

keywords: tuple[str] = ('position tracking',)#

associated_suffixes: tuple[str] = ('.nvt',)#

info: Union[str, None] = 'Interface for writing Neuralynx position tracking .nvt files to NWB.'#

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray) → None[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns: The JSON schema defining the metadata structure.
Return type: dict

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, add_position: bool = True, add_angle: Optional[bool] = None)[source]#

Add NVT data to a given in-memory NWB file

Parameters

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict, optional) – metadata info for constructing the nwb file.
add_position (bool, default=True)
add_angle (bool, optional) – If None, write angle as long as it is not all 0s

SLEAP#

class SLEAPInterface(file_path: Annotated[pathlib.Path, PathType(path_type='file')], video_file_path: Optional[Annotated[pathlib.Path, PathType(path_type='file')]] = None, verbose: bool = False, frames_per_second: Optional[float] = None)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for SLEAP datasets.

Interface for writing sleap .slp files to nwb using the sleap-io library.

Parameters

file_path (FilePath) – Path to the .slp file (the output of sleap)
verbose (bool, default: Falsee) – controls verbosity. True by default.
video_file_path (FilePath, optional) – The file path of the video for extracting timestamps.
frames_per_second (float, optional) – The frames per second (fps) or sampling rate of the video.

display_name: Union[str, None] = 'SLEAP'#

keywords: tuple[str] = ('pose estimation', 'tracking', 'video')#

associated_suffixes: tuple[str] = ('.slp', '.mp4')#

info: Union[str, None] = 'Interface for SLEAP pose estimation datasets.'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns: The JSON schema for the source_data.
Return type: dict

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns: timestamps – The timestamps for the data stream.
Return type: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray)[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None)[source]#

Conversion from DLC output files to nwb. Derived from sleap-io library.

Parameters

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict) – metadata info for constructing the nwb file (optional).

Video#

class VideoInterface(file_paths: list[typing.Annotated[pathlib.Path, PathType(path_type='file')]], verbose: bool = False, *, metadata_key_name: str = 'Videos')[source]#

Bases: BaseDataInterface

Data interface for writing videos as ImageSeries.

Create the interface for writing videos as ImageSeries.

Parameters

file_paths (list of FilePaths) – Many video storage formats segment a sequence of videos over the course of the experiment. Pass the file paths for this videos as a list in sorted, consecutive order.
metadata_key_name (str, optional) – The key used to identify this video data within the overall experiment metadata. Defaults to “Videos”.

This key is essential when multiple video streams are present in a single experiment. The associated metadata should be a list of dictionaries, with each dictionary containing metadata for a specific video segment:

``` metadata[“Behavior”][metadata_key_name] = [

{video1_metadata}, {video2_metadata}, …

If other video interfaces exist, they would follow a similar structure:

``` metadata[“Behavior”][“other_video_key_name”] = [

{other_video1_metadata}, {other_video2_metadata}, …

]

display_name: Union[str, None] = 'Video'#

keywords: tuple[str] = ('movie', 'natural behavior', 'tracking')#

associated_suffixes: tuple[str] = ('.mp4', '.avi', '.wmv', '.mov', '.flx', '.mkv')#

info: Union[str, None] = 'Interface for handling standard video file formats.'#

get_metadata_schema()[source]#

Retrieve JSON schema for metadata.

Returns: The JSON schema defining the metadata structure.
Return type: dict

get_metadata()[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

get_original_timestamps(stub_test: bool = False) → list[numpy.ndarray][source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns

timestamps (numpy.ndarray) – The timestamps for the data stream.
stub_test (bool, default: False) – This method scans through each video; a process which can take some time to complete.

To limit that scan to a small number of frames, set stub_test=True.

get_timing_type() → Literal['starting_time and rate', 'timestamps'][source]#

Determine the type of timing used by this interface.

Returns

The type of timing that has been set explicitly according to alignment.

If only timestamps have been set, then only those will be used. If only starting times have been set, then only those will be used.

If timestamps were set, and then starting times were set, the timestamps will take precedence as they will then be shifted by the corresponding starting times.

If neither has been set, and there is only one video in the file_paths, it is assumed the video is regularly sampled and pre-aligned with a starting_time of 0.0 relative to the session start time.

Return type

Literal[“starting_time and rate”, “timestamps”]

get_timestamps(stub_test: bool = False) → list[numpy.ndarray][source]#

Retrieve the timestamps for the data in this interface.

Returns

timestamps (numpy.ndarray) – The timestamps for the data stream.
stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

To limit that scan to a small number of frames, set stub_test=True.

set_aligned_timestamps(aligned_timestamps: list[numpy.ndarray])[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps (list of numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float, stub_test: bool = False)[source]#

Align all starting times for all videos in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_starting_time (float) – The common starting time for all segments of temporal data in this interface.
stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

To limit that scan to a small number of frames, set stub_test=True.

set_aligned_segment_starting_times(aligned_segment_starting_times: list[float], stub_test: bool = False)[source]#

Align the individual starting time for each video (segment) in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_segment_starting_times (list of floats) – The relative starting times of each video.
stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

To limit that scan to a small number of frames, set stub_test=True.

align_by_interpolation(unaligned_timestamps: ndarray, aligned_timestamps: ndarray)[source]#

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, stub_test: bool = False, external_mode: bool = True, starting_frames: Optional[list[int]] = None, chunk_data: bool = True, module_name: Optional[str] = None, module_description: Optional[str] = None)[source]#

Convert the video data files to ImageSeries and write them in the NWBFile. Data is written in the ImageSeries container as RGB. [times, x, y, 3-RGB].

Parameters

nwbfile (NWBFile, optional) – nwb file to which the recording information is to be added
metadata (dict, optional) – Dictionary of metadata information such as names and description of each video. Metadata should be passed for each video file passed in the file_paths. If storing as ‘external mode’, then provide duplicate metadata for video files that go in the same ImageSeries container. Should be organized as follows:
```
metadata = dict(
    Behavior=dict(
        Videos=[
            dict(name="Video1", description="This is the first video.."),
            dict(name="SecondVideo", description="Video #2 details..."),
            ...
        ]
    )
)
```
and may contain most keywords normally accepted by an ImageSeries (https://pynwb.readthedocs.io/en/stable/pynwb.image.html#pynwb.image.ImageSeries). The list for the ‘Videos’ key should correspond one to the video files in the file_paths list. If multiple videos need to be in the same ImageSeries, then supply the same value for “name” key. Storing multiple videos in the same ImageSeries is only supported if ‘external_mode’=True.
stub_test (bool, default: False) – If True, truncates the write operation for fast testing.
external_mode (bool, default: True) – ImageSeries may contain either video data or file paths to external video files. If True, this utilizes the more efficient method of writing the relative path to the video files (recommended).
starting_frames (list, optional) – List of start frames for each video written using external mode. Required if more than one path is specified per ImageSeries in external mode.
chunk_data (bool, default: True) – If True, uses a DataChunkIterator to read and write the video, reducing overhead RAM usage at the cost of reduced conversion speed (compared to loading video entirely into RAM as an array). This will also force to True, even if manually set to False, whenever the video file size exceeds available system RAM by a factor of 70 (from compression experiments). Based on experiments for a ~30 FPS system of ~400 x ~600 color frames, the equivalent uncompressed RAM usage is around 2GB per minute of video. The default is True.
module_name (str, optional) – Name of the processing module to add the ImageSeries object to. Default behavior is to add as acquisition.
module_description (str, optional) – If the processing module specified by module_name does not exist, it will be created with this description. The default description is the same as used by the conversion_tools.get_module function.

MedPC#

class MedPCInterface(file_path: Annotated[pathlib.Path, PathType(path_type='file')], session_conditions: dict, start_variable: str, metadata_medpc_name_to_info_dict: dict, aligned_timestamp_names: Optional[list[str]] = None, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data Interface for MedPC output files.

The output files from MedPC are raw text files that contain behavioral data from the operant box sessions such as lever presses, reward port entries, nose pokes, etc. The output text files format this data into a series of colon-separated variables that are either single-line (for metadata) or multi-line (for arrays). The multi-line variables keep a colon-separated index of the array every 5 elements. For example, a single variable might look like:

Start Date: 11/09/18

while a multi-line variable might look like:

A:
     175.150      270.750      762.050      762.900     1042.600
    1567.800     1774.950     2448.450     2454.050     2552.800
    2620.550     2726.250

Different sessions are usually separated by a blank line or two.

This data is parsed by the MedPCInterface and added to the NWBFile as Events and IntervalSeries objects in the behavior module.

Initialize MedpcInterface.

Parameters

file_path (FilePath) – Path to the MedPC file.
session_conditions (dict) – The conditions that define the session. The keys are the names of the single-line variables (ex. ‘Start Date’) and the values are the values of those variables for the desired session (ex. ‘11/09/18’).
start_variable (str) – The name of the variable that starts the session (ex. ‘Start Date’).
metadata_medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“Start Date”: {“name”: “start_date”, “is_array”: False}}
aligned_timestamp_names (list, optional) – The names of the variables that are externally aligned timestamps, which should be retrieved from self.timestamps_dict instead of the MedPC output file.
verbose (bool, optional) – Whether to print verbose output, by default True

keywords: tuple[str] = ('behavior',)#

display_name: Union[str, None] = 'MedPC'#

info: Union[str, None] = 'Interface for handling MedPC output files.'#

associated_suffixes: tuple[str] = ('.txt',)#

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns: The metadata dictionary containing basic NWBFile metadata.
Return type: DeepDict

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns: The JSON schema defining the metadata structure.
Return type: dict

get_original_timestamps(medpc_name_to_info_dict: dict) → dict[str, numpy.ndarray][source]#

Retrieve the original unaltered timestamps dictionary for the data in this interface.

This function retrieves the data on-demand by re-reading the medpc file.

Parameters: medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“A”: {“name”: “left_nose_poke_times”, “is_array”: True}}
Returns: timestamps_dict – A dictionary mapping the names of the variables to the original medpc timestamps.
Return type: dict

get_timestamps() → dict[str, numpy.ndarray][source]#

Retrieve the timestamps dictionary for the data in this interface.

Returns: timestamps_dict – A dictionary mapping the names of the variables to the timestamps.
Return type: dict

set_aligned_timestamps(aligned_timestamps_dict: dict[str, numpy.ndarray]) → None[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters: aligned_timestamps_dict (dict) – A dictionary mapping the names of the variables to the synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float, medpc_name_to_info_dict: dict) → None[source]#

Align the starting time for this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_starting_time (float) – The starting time for all temporal data in this interface.
medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“A”: {“name”: “left_nose_poke_times”, “is_array”: True}}

add_to_nwbfile(nwbfile: NWBFile, metadata: dict) → None[source]#

Define a protocol for mapping the data from this interface to NWB neurodata objects.

These neurodata objects should also be added to the in-memory pynwb.NWBFile object in this step.

Parameters

nwbfile (pynwb.NWBFile) – The in-memory object to add the data to.
metadata (dict) – Metadata dictionary with information used to create the NWBFile.
**conversion_options – Additional keyword arguments to pass to the .add_to_nwbfile method.