HDF5 Experiment Hierarchy — raw, processed, metadata (.h5)
A three-group HDF5 file laid out the way a real bench run is — raw arrays, a processed derivative that names its source in a derived_from attribute, and a metadata group — with units on every dataset. It is the walk-the-tree fixture: a reader has to recurse rather than assume a flat namespace.
| Path | Shape | dtype | units |
|---|---|---|---|
| /raw/detector_counts | (32, 8) | float64 | counts |
| /raw/timestamp_s | (32,) | float64 | s |
| /processed/normalised | (32, 8) | float64 | 1 |
| /metadata/channel_index | (8,) | int32 | 1 |
Specifications
- Groups
- 3
- Datasets
- 4
- Root Attributes
- 3
- Shape
- 32 x 8 detector counts
- Units On Every Dataset
- true
- Seed
- 20260818
- Provenance Attribute
- derived_from
Testing contract
Expected to pass- Scenario
- Recursively walk the file and collect every dataset path together with its units attribute.
- Expected result
- You find exactly four datasets under three groups, each carries a units attribute, and /processed/normalised equals /raw/detector_counts divided by 1000 to six decimals.
What is a .h5 file?
HDF5 (.h5) is a binary container format for large, heterogeneous scientific data. It stores multidimensional arrays (datasets) in a hierarchical group structure with attributes and chunked, compressed storage, and is standard in ML, physics, and geoscience.
How to use this file
Use an example .h5 file to test HDF5 readers (h5py, PyTables), group and dataset traversal, and attribute extraction.
How to use this file for testing
“HDF5 Experiment Hierarchy — raw, processed, metadata (.h5)” is a deterministic Novus Examples fixture for Scientific data, Serialization testing. Citation catalogs (BibTeX, RIS), chemistry structures (MDL Molfile, PDB), and gridded binary data (NetCDF, FITS) — for testing reference managers, molecule viewers, and scientific-data loaders.
Documented properties for this file: seed 20260818. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Scientific fixtures are small, valid, and fully synthetic — no real organism, patient, sample, or observation. Point your parser or loader at the file and check it reads the documented records, variables, or headers; binary formats ship a readable twin or metadata listing for comparison.
Related files
- h5HDF5 Chunked + gzip + shuffle Filters (.h5)One array stored twice in the same file: chunked with the shuffle and gzip filters applied, and contiguous with no filters at all. Comparing the two proves a reader ran the HDF5 filter pipeline rather than reading raw chunk bytes.
- h5HDF5 Compound Record dtype — Mixed Field Types (.h5)A compound-dtype dataset mixing an integer key, two float measurements, a fixed-width byte string and a boolean flag in a single record. It is the fixture for readers that must map an HDF5 record type onto a native struct without reordering or repadding the fields.
- h5HDF5 Degenerate Shapes — Scalar, Zero-Length and Null (.h5)Five degenerate but entirely legal datasets — a rank-0 scalar, a zero-length vector, a (0, 5) array, a single-element vector and a NULL dataspace — plus an empty group. None of them is corrupt, and a reader that reports them as errors is the thing being tested.
- h5HDF5 Hard, Soft and Dangling Links (.h5)One array reachable under four names: itself, a hard link sharing its object address, a soft link resolved at access time, and a soft link pointing nowhere. A walker that counts names instead of object addresses reports four arrays and then crashes on the dangling one.
- h5HDF5 Numeric dtype Zoo — Every Width at Its Limits (.h5)Twelve datasets, one per numeric HDF5 type, each holding the extreme values of that type including the uint64 and int64 limits that do not survive a trip through a double. It is the fixture that exposes a reader which widens everything to float64 on the way in.
- h5HDF5 Strings — Fixed, Variable-Length and UTF-8 (.h5)Fixed-width byte strings, variable-length UTF-8 strings and UTF-8 attribute text in one file, including Greek, CJK and combining diacritics. HDF5 hands fixed-width strings back as padded bytes and variable-length ones as decoded text, and conflating the two is where string handling usually breaks.
Generated by generation/scientific.py. Free for any use, no attribution required — license.