MCS HDF5 Format Definitions¶
Definition of the HDF5 format for raw data¶
MCS-HDF5 Protocol Type: RawData (Raw-Data protocol)
Protocol Version: 3 based on the definitions of RawDataFileIO in version 10.
All strings are only ASCII-encoded
Changelog¶
Version 1:
- Initial draft
Version 2:
- New Root-Folder attributes added to detect name and version of the creating application and library
Version 3:
- Data structures for DataSubType::Average of StreamType::Segment added
Hierarchy¶
Root-Folder “/”¶
Contains all information for one experiment - measured data (inside the folder Data) and a description (possibly in the future) inside the folder Experiment/Description/…
Attributes:
Name | Description | Data Type | MCS-HDF5 Protocol Version |
---|---|---|---|
McsHdf5ProtocolType | Type of the used MCS-HDF5 protocol definition
(e.g. RawData for the raw data MCS-HDF5 definitions)
|
[String,Scalar] | 1 ≤ |
McsHdf5ProtocolVersion | Version number of the used MCS-HDF5 protocol | [Integer,Scalar] | 1 ≤ |
GeneratingApplicationName | Name of the application that generated this HDF5 file | [String,Scalar] | 2 ≤ |
GeneratingApplicationVersion | Version of the application that generated this HDF5 file | [String,Scalar] | 2 ≤ |
McsDataToolsVersion | Version of the McsDataTools library that was used
by the application to create the HDF5 file
|
[String,Scalar] | 2 ≤ |
Datasets:
- none
Folder “Data”¶
Navigation: /Data
Contains all recordings for this experiment.
Attributes:
Name | Description | Data Type |
---|---|---|
ProgramName | Name of the recording program | [String,Scalar] |
ProgramVersion | Version number of the recording program | [String,Scalar] |
MeaName | Name of the recorded MEA | [String,Scalar] |
MeaLayout | Layout descriptor | [String,Scalar] |
MeaSN | Serial number of the MEA | [String,Scalar] |
Date | Date of the recording | [String,Scalar] |
DateInTicks | Date of the recording in .NET ticks (100 ns) | [Long(64-bit Integer),Scalar] |
FileGUID | GUID of the converted raw data file | [String,Scalar] |
Comment | Comment | [String,Scalar] |
Datasets:
- none
Folder “Recording_x”¶
Navigation: /Data/Recording_x
Contains all recorded streams for recording x.
Attributes:
Name | Description | Data Type |
---|---|---|
RecordingID | Recording ID | [Integer(32-bit Integer),Scalar] |
RecordingType | Recording type | [String,Scalar] |
TimeStamp | Start time of the recording in microseconds | [Long(64-bit Integer),Scalar] |
Duration | Total recording duration in microseconds
(This duration can differ from the actual
duration of the recorded data!!!)
|
[Long(64-bit Integer),Scalar] |
Label | Label | [String,Scalar] |
Comment | Comment | [String,Scalar] |
Datasets:
- none
Folder “AnalogStream”¶
Navigation: /Data/Recording_x/AnalogStream
(Organisational) folder for all channel-based streams of this recording
Attributes:
- none
Datasets:
- none
Sub-folder “Stream_x” of “AnalogStream”¶
Navigation: /Data/Recording_x/AnalogStream/Stream_x
Container for an analog stream
Attributes:
Name | Description | Data Type | StreamInfoVersion |
---|---|---|---|
StreamInfoVersion | Version number of the meta information structure | [Int(32-bit Integer),Scalar] | 1 ≤ |
Label | Label | [String,Scalar] | 1 ≤ |
SourceStreamGUID | GUID of the source streams | [String,Scalar] | 1 ≤ |
StreamGUID | GUID | [String,Scalar] | 1 ≤ |
StreamType | Type of the stream, e.g. Electrode | [String,Scalar] | 1 ≤ |
DataSubType | Sub-type of the analog stream (e.g. Analog) | [String,Scalar] | 1 ≤ |
Datasets:
- Matrix InfoChannel → n × 16 matrix of describing information vectors
for the n channels:
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
Name | Description | Data Type | InfoVersion |
---|---|---|---|
ChannelID | ID of the channel as given by the recording software | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
RowIndex | Row number of this channel inside the ChannelData matrix where
the data of this channel is stored
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
GroupID | ID of the group that this channel belongs to | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Label | Label of the channel | [String,Array] | 1 ≤ |
RawDataType | Type of the raw data | [String,Array] | 1 ≤ |
Unit | Physical unit of the measured sensor value | [String,Array] | 1 ≤ |
Exponent | Exponent n ⇒ 1En resp. 10n in which the channel values
magnitude is measured (e.g. k,m,µ,…)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ADZero | ADC-Step that represents the 0-point of the measuring range of the ADC | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Tick | Sample tick Δ between two sample points of a channel in µs
⇒ sampling frequency = 1000000 / Δ
|
[Long(64-bit Integer),Array(Size 1)] | 1 ≤ |
ConversionFactor | Conversion factor for the mapping ADC-Step ⇒ measured value | [Long(64-bit Integer),Array(Size 1)] | 1 ≤ |
ADCBits | Number of bits used by the AD-Converter | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
HighPassFilterType | Type of the high-pass filter (empty string if not available) | [String,Scalar] | 1 ≤ |
HighPassFilterCutOffFrequency | Cut-off frequency of the high-pass filter (‘-1’-String if not available) | [String,Scalar] | 1 ≤ |
HighPassFilterOrder | Order of the high-pass filter (-1 if not available) | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
LowPassFilterType | Type of the low-pass filter (empty string if not available) | [String,Scalar] | 1 ≤ |
LowPassFilterCutOffFrequency | Cut-off frequency of the low-pass filter (‘-1’-String if not available) | [String,Scalar] | 1 ≤ |
LowPassFilterOrder | Order of the low-pass filter (-1 if not available) | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
- 2-dimensional Data-Matrix ChannelData → Data for sampled channels
organized as n × m matrix ⇒ one row per channel and one column per
sample time point
- reconstruct the value of the measured signal: \(y(\text{channel},t_{ind}) = (\text{ChannelData}[\text{InfoChannel}[\text{channel}].\text{RowIndex},t_{ind}] - \text{ADZero}) * \text{InfoChannel}[\text{channel}].\text{ConversionFactor} * 10^{\text{InfoChannel}[\text{channel}].\text{Exponent}}\) in \(\text{InfoChannel}[\text{channel}].\text{Unit}\)
- reconstruct the sample time point: \(t = t_{ind} * \text{InfoChannel}[\text{channel}].\text{Tick}\) in \(\mu s\)
- Matrix ChannelDataTimeStamps → k × 3 matrix of segments where the
rows are one segment and the columns are:
- first column → time stamp of the first sample point of the segment
- second column → first index (column) of the segment in ChannelData
- third column → last index (column) of the segment in ChannelData
Folder “FrameStream”¶
Navigation: /Data/Recording_x/FrameStream
(Organisational) folder for all frame-based streams of this recording
Attributes:
- none
Datasets:
- none
Subfolder “Stream_x” of “FrameStream”¶
Navigation: /Data/Recording_x/FrameStream/Stream_x
Folder that contains all Frame-Entities of one Frame-Stream:
Attributes:
Name | Description | Data Type | StreamInfoVersion |
---|---|---|---|
StreamInfoVersion | Version number of the meta information structure | [Int(32-bit Integer),Scalar] | 1 ≤ |
Label | Label | [String,Scalar] | 1 ≤ |
SourceStreamGUID | GUID of the source stream | [String,Scalar] | 1 ≤ |
StreamGUID | GUID | [String,Scalar] | 1 ≤ |
StreamType | Type of the stream Frame | [String,Scalar] | 1 ≤ |
DataSubType | Sub-type of the event stream (e.g. SpikeTimeStamp) | [String,Scalar] | 1 ≤ |
Datasets:
- Matrix InfoFrame → n × 24 matrix of describing information vectors
for the n Frame-Entities:
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
Name | Description | Data Type | InfoVersion |
---|---|---|---|
FrameID | ID of the frame entity as given by the recording software | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
FrameDataID | ID of the frame entity inside the stream folder that maps this information
vector to the entity folder (FrameDataID → subfolder FrameDataEntity_FrameDataID)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
GroupID | ID of the group that this frame entity belongs to | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Label | Label of the entity | [String,Array] | 1 ≤ |
RawDataType | Type of the raw data | [String,Array] | 1 ≤ |
Unit | Physical unit of the measured sensor value | [String,Array] | 1 ≤ |
Exponent | Exponent n ⇒ 1En resp. 10n in which the sensor values magnitude
is measured (e.g. k,m,µ,…)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ADZero | ADC-Step that represents the 0-point of the measuring range of the ADC | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ADCBits | Number of bits used by the AD-Converter | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Tick | Sample tick Δ between two frames in µs
⇒ sampling frequency = 1000000 / Δ
|
[Long(64-bit Integer),Array(Size 1)] | 1 ≤ |
HighPassFilterType | Type of the high-pass filter (empty string if not available) | [String,Scalar] | 1 ≤ |
HighPassFilterCutOffFrequency | Cut-off frequency of the high-pass filter (‘-1’-String if not available) | [String,Scalar] | 1 ≤ |
HighPassFilterOrder | Order of the high-pass filter (-1 if not available) | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
LowPassFilterType | Type of the low-pass filter (empty string if not available) | [String,Scalar] | 1 ≤ |
LowPassFilterCutOffFrequency | Cut-off frequency of the low-pass filter (‘-1’-String if not available) | [String,Scalar] | 1 ≤ |
LowPassFilterOrder | Order of the low-pass filter (-1 if not available) | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
SensorSpacing | Distance between adjacent sensors in µm | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
FrameLeft | Sensor count of the left edge of the entity frame based on the reference frame | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
FrameTop | Sensor count of the top edge of the entity frame based on the reference frame | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
FrameRight | Sensor count of the right edge of the entity frame based on the reference frame | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
FrameBottom | Sensor count of the bottom edge of the entity frame based on the reference frame | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ReferenceFrameLeft | Sensor count of the left edge of the reference frame
(defined by the used sensor array)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ReferenceFrameTop | Sensor count of the left edge of the reference frame
(defined by the used sensor array)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ReferenceFrameRight | Sensor count of the left edge of the reference frame
(defined by the used sensor array)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
ReferenceFrameBottom | Sensor count of the left edge of the reference frame
(defined by the used sensor array)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Subfolder “FrameDataEntity_x”¶
Navigation: /Data/Recording_x/FrameStream/Stream_x/FrameDataEntity_x
Contains all datasets of the Frame-Entity x
Datasets:
- Matrix ConversionFactors → n × m matrix of conversion factors for the sensor array
- 3-dimensional Data-Cube FrameData → cube of the frame data organized
as one frame to one sample time point (n × m matrix of sampled signal
values per sensor) × sample time points
- reconstruct the value of the measured signal: y = (FrameData[x,y,t] - ADZero) * ConversionFactors[x,y]
- reconstruct the sample time point:
- Matrix FrameDataTimeStamps → k × 3 matrix of segments where the rows
are one segment and the columns are:
- first column → time stamp of the first sample point of the segment
- second column → first index (z-axis) of the segment in FrameData
- third column → last index (z-axis) of the segment in FrameData
Datasets:
- none
Folder “EventStream”¶
Navigation: /Data/Recording_x/EventStream
(Organisational) folder for all event-based streams of this recording
Attributes:
- none
Datasets:
- none
Subfolder “Stream_x” of “EventStream”¶
Navigation: /Data/Recording_x/EventStream/Stream_x
Folder that contains all Event-Entities of one Event-Stream:
Attributes:
Name | Description | Data Type | StreamInfoVersion |
---|---|---|---|
StreamInfoVersion | Version number of the meta information structure | [Int(32-bit Integer),Scalar] | 1 ≤ |
Label | Label | [String,Scalar] | 1 ≤ |
SourceStreamGUID | GUID of the source stream | [String,Scalar] | 1 ≤ |
StreamGUID | GUID of the current stream | [String,Scalar] | 1 ≤ |
StreamType | Type of the stream Event | [String,Scalar] | 1 ≤ |
DataSubType | Sub-type of the event stream (e.g. StgSideband, UserInput, DigitalPort) | [String,Scalar] | 1 ≤ |
Sub-type Description:
- StgSideband → The event is associated to a STG sideband change.
- UserInput → The event is associated with an user input.
- DigitalPort → The event is associated with a digital port change.
Datasets:
- Matrix InfoEvent → n × 7 matrix of describing information vectors for
the n Event-Entities:
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
Name | Description | Data Type | InfoVersion |
---|---|---|---|
EventID | ID of the event entity | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
GroupID | ID of the group that the entity belongs to | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Label | Label of the entity | [String,Array] | 1 ≤ |
RawDataType | Type of the raw data | [String,Array] | 1 ≤ |
RawDataBytes | Number of bytes of the raw data type | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
SourceChannelIDs | Comma separated list of ID’s of (source) channel that were
involved in the generation of this event
|
[String,Array] | 1 ≤ |
SourceChannelLabels | Comma separated list of labels of the source channels | [String,Scalar] | 1 ≤ |
- 2-dimensional matrix EventEntity_x → 2 × n matrix ⇒ n events with
describing vector (time stamp of event, duration of event)
- Attributes: Short description of content
- \(t_\text{event i} = \text{EventEntity}\_x[0,i]\) in \(\mu s\)
- \(\Delta_\text{event i} = \text{EventEntity}\_x[1,i]\) in \(\mu s\)
Folder “SegmentStream”¶
Navigation: /Data/Recording_x/SegmentStream
(Organisational) folder for all segment-based streams of this recording. A segment is a cutout of parts of the sampled signal relative to an event, defined by a pre- and post interval.
Attributes:
- none
Datasets:
- none
Subfolder “Stream_x” of “SegmentStream”¶
Navigation: /Data/Recording_x/SegmentStream/Stream_x
Folder that contains all Segment-Entities of one Segment-Stream:
Attributes:
Name | Description | Data Type | StreamInfoVersion |
---|---|---|---|
StreamInfoVersion | Version number of the meta information structure | [Int(32-bit Integer),Scalar] | 1 ≤ |
Label | Label | [String,Scalar] | 1 ≤ |
SourceStreamGUID | GUID of the source stream | [String,Scalar] | 1 ≤ |
StreamGUID | GUID of the current stream | [String,Scalar] | 1 ≤ |
StreamType | Type of the stream Segment | [String,Scalar] | 1 ≤ |
DataSubType | Sub-type of the segment stream (e.g. Spike) | [String,Scalar] | 1 ≤ |
Datasets:
- Matrix InfoSegment → n × 7 matrix of describing information vectors
for the n Segment-Entities:
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
Name | Description | Data Type | InfoVersion |
---|---|---|---|
SegmentID | ID of the segment entity | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
GroupID | ID of the group that the segment entity belongs to | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Label | Label of the entity | [String,Array] | 1 ≤ |
PreInterval | Time interval in µs before the segment defining event occured
- definition of the beginning of the segment
|
[Int(64-bit Integer),Array(Size 1)] | 1 ≤ |
PostInterval | Time interval in µs after the segment defining event occured
- definition of the end of the segment
\(\text{length of the segment} = \text{PreInterval} + \text{PostInterval}\) in µs
|
[Int(64-bit Integer),Array(Size 1)] | 1 ≤ |
SegmentType | Type of the segment (e.g. SpikeCutout) | [String,Array] | 1 ≤ |
SourceChannelIDs | Comma separated list of ID’s of (source) channels that the segements are taken from
→ Link to the SourceChannelInfo matrix
|
[String,Array] | 1 ≤ |
- 2-dimensional matrix SourceChannelInfo → n × 15 matrix ⇒ n of
describing vectors for the n source channels, the structure is the
same as in ChannelInfo used in section Sub-folder “Stream_x” of
“AnalogStream”
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
- Vector SegmentData_ts_x → n time stamps in µs of the event triggering the segment, one for each of the n segments contained by segment entity x
- 2-dimensional matrix or 3-dimensional cube SegmentData_x → k × n
matrix (k sample points for one segment, n number of sampled
segments) or k × m × n cube (k sample points for one segment, m
number of segments for one time point/for one multi-segment, n number
of sampled multi-segments) of segment data:
- Attributes: SourceChannelID → Comma separated list of ID’s of (source) channels that the segements are taken from [String,Scalar] (the same as in InfoSegment, repeated for clarification)
- reconstruct the value of the measured segment signal (only one
segment \(id_\text{segment}\) → 2-dimensional matrix
M[row,col]):
- \(t_{ind}[row,col] = \text{SegmentData ts x}[\text{col}] + (\text{row} - 1) * \text{tick}_\text{source-channel} - \text{PreInterval}\) in µs
- \(y(id_\text{segment},t_{ind}(row,col)) = (\text{SegmentData x}[row, col] - \text{ADZero}_\text{source-channel}) * \text{ConversionFactor}_\text{source-channel} * 10^{\text{Exponent}_\text{source-channel}}\) in \(\text{InfoChannel}[\text{source-channel}].\text{Unit}\)
- reconstruct the value of the measured segment signal (m segments →
multi-segments → 3-dimensional cube M[row,col,z]):
- col → \(id_\text{segment}\) → source-channel
- \(t_{ind}[row,col,z] = \text{SegmentData ts x}[\text{z}] + (\text{row} - 1) * \text{tick}_{\text{source-channel}[col]}\) in µs
- \(y(id_\text{segment},t_{ind}(row, z)) = (\text{SegmentData x}[row, \text{col}, z] - \text{ADZero}_{\text{source-channel}[\text{col}]}) * \text{ConversionFactor}_{\text{source-channel}[\text{col}]} * 10^{\text{Exponent}_{\text{source-channel}[\text{col}]}}\) in \(\text{InfoChannel}[\text{source-channel}[\text{col}]].\text{Unit}\)
DataSubType-Average: Subfolder “Stream_x” of “SegmentStream”¶
Navigation: /Data/Recording_x/SegmentStream/Stream_x
Folder that contains all Segment-Entities of one Segment-Stream with DataSybType == Average:
Attributes: no difference to the standard case above
Datasets:
- Matrix InfoSegment: no difference to the standard case above
- Matrix SourceChannelInfo: no difference to the standard case above
- (3 × n) matrix AverageData_Range_x → (start, end,
count) per segment average × count of segment averages contained
by segment entity x. start and end denote the start and end
timestamp in µs of the interval that contains all averaged segments.
count is the number of averaged segments.
- Attributes: description of the content
- (2 × k × n) cube AverageData_x → (mean and standard deviation) × k
sample points of the segment × n number of segment averages
- Attributes:: description of the content
- reconstruct the value of the mean and standard deviation of the
average segment (n average segments → 3-dimensional cube
M[row,col,z]):
- row: mean → row = 0; StdDev → row = 1
- col: \(t_{ind}(col) = (\text{col} - 1) * \text{tick}_{\text{source-channel}}\) → time range \((0, PreInterval[\text{SegmentID}] + PreInterval[\text{SegmentID}])\) in µs
- z: z = \(id_\text{average}\) (number of average segment)
- \(Mean(id_\text{average},t_{ind}(col)) = (\text{AverageData x}[0, \text{col}, id_\text{average}] - \text{ADZero}_{\text{source-channel}}) * \text{ConversionFactor}_{\text{source-channel}} * 10^{\text{Exponent}_{\text{source-channel}}}\) in \(\text{InfoChannel}_\text{source-channel}.\text{Unit}\)
- \(StdDev(id_\text{average},t_{ind}(col)) = \text{AverageData x}[1, \text{col}, id_\text{average}] * \text{ConversionFactor}_{\text{source-channel}} * 10^{\text{Exponent}_{\text{source-channel}}}\) in \(\text{InfoChannel}_\text{source-channel}.\text{Unit}\)
Folder “TimeStampStream”¶
Navigation: /Data/Recording_x/TimeStampStream
(Organisational) folder for all TimeStamp-based streams of this recording
Attributes:
- none
Datasets:
- none
Subfolder “Stream_x” of “TimeStampStream”¶
Navigation: /Data/Recording_x/TimeStampStream/Stream_x
Folder that contains all TimeStamp-Entities of one TimeStamp-Stream:
Attributes:
Name | Description | Data Type | StreamInfoVersion |
---|---|---|---|
StreamInfoVersion | Version number of the meta information structure | [Int(32-bit Integer),Scalar] | 1 ≤ |
Label | Label | [String,Scalar] | 1 ≤ |
SourceStreamGUID | GUID of the source stream | [String,Scalar] | 1 ≤ |
StreamGUID | GUID of the current stream | [String,Scalar] | 1 ≤ |
StreamType | Type of the stream TimeStamp | [String,Scalar] | 1 ≤ |
DataSubType | Sub-type of the TimeStamp stream (e.g. NeuralSpike) | [String,Scalar] | 1 ≤ |
Sub-type Description:
- NeuralSpike → The entity contains time stamps of neural spikes
Datasets:
- Matrix InfoTimeStamp → n × 7 matrix of describing information vectors
for the n Event-Entities:
- Attributes: InfoVersion → Version number of the Info-Objects [Int(32-bit Integer),Scalar]
Name | Description | Data Type | InfoVersion |
---|---|---|---|
TimeStampEntityID | ID of the event entity | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
GroupID | ID of the group that the entity belongs to | [Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
Label | Label of the entity | [String,Array] | 1 ≤ |
Unit | Physical unit of the measured sensor value | [String,Array] | 1 ≤ |
Exponent | Exponent n ⇒ 1En resp. 10n in which the channel values
magnitude is measured (e.g. k,m,µ,…)
|
[Int(32-bit Integer),Array(Size 1)] | 1 ≤ |
SourceChannelIDs | Comma separated list of ID’s of (source) channel that were
involved in the generation of this event
|
[String,Array] | 1 ≤ |
SourceChannelLabels | Comma separated list of labels of the source channels | [String,Scalar] | 1 ≤ |
- Vector TimeStampEntity_x → n time stamps in \(\mu s\)
Comment¶
All time-related information except dates (100ns ticks) are given in \(\mu s\) ticks!!
Category:Software