HDF5 Strings — Fixed, Variable-Length and UTF-8 (.h5)
Fixed-width byte strings, variable-length UTF-8 strings and UTF-8 attribute text in one file, including Greek, CJK and combining diacritics. HDF5 hands fixed-width strings back as padded bytes and variable-length ones as decoded text, and conflating the two is where string handling usually breaks.
| Dataset | Kind | First value |
|---|---|---|
| /fixed_ascii | fixed 8-byte, NUL-padded | alpha |
| /variable_utf8 | variable-length UTF-8 | ångström |
| /unit_symbols | variable-length UTF-8 | K |
| root attr 'note' | UTF-8 attribute string | Attribute strings are UTF-8 too: 5 µm ± 0.2 µm |
Specifications
- Fixed Width Dataset
- S8 (3 items)
- Variable Length Datasets
- 2
- Encoding
- UTF-8
- Non Ascii Items
- 5
- Scripts
- Latin with diacritics, Greek, CJK, arrows
Testing contract
Expected to pass- Scenario
- Read all three string datasets and the root note attribute, decoding each according to its declared HDF5 string type.
- Expected result
- fixed_ascii yields NUL-padded bytes of width 8, variable_utf8 yields five decoded strings starting with 'ångström', and no value arrives mojibaked or truncated at a multi-byte boundary.
What is a .h5 file?
HDF5 (.h5) is a binary container format for large, heterogeneous scientific data. It stores multidimensional arrays (datasets) in a hierarchical group structure with attributes and chunked, compressed storage, and is standard in ML, physics, and geoscience.
How to use this file
Use an example .h5 file to test HDF5 readers (h5py, PyTables), group and dataset traversal, and attribute extraction.
How to use this file for testing
“HDF5 Strings — Fixed, Variable-Length and UTF-8 (.h5)” is a deterministic Novus Examples fixture for Scientific data, Serialization testing, Encoding detection. Citation catalogs (BibTeX, RIS), chemistry structures (MDL Molfile, PDB), and gridded binary data (NetCDF, FITS) — for testing reference managers, molecule viewers, and scientific-data loaders.
Documented properties for this file: UTF-8. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Scientific fixtures are small, valid, and fully synthetic — no real organism, patient, sample, or observation. Point your parser or loader at the file and check it reads the documented records, variables, or headers; binary formats ship a readable twin or metadata listing for comparison.
Related files
- npyNumPy .npy Format Version 3.0 — UTF-8 Field Names (.npy)A structured array whose field names use Greek and CJK characters, which is the only reason format version 3.0 exists: its header is UTF-8 where 1.0 and 2.0 are latin-1. A parser that decodes the header as latin-1 mangles all three names.
- h5HDF5 Chunked + gzip + shuffle Filters (.h5)One array stored twice in the same file: chunked with the shuffle and gzip filters applied, and contiguous with no filters at all. Comparing the two proves a reader ran the HDF5 filter pipeline rather than reading raw chunk bytes.
- h5HDF5 Deeply Nested Group Tree — Eight Levels (.h5)Eight levels of nested groups, each holding a dataset whose values equal its own depth. Any tree walk that caps recursion depth or flattens names silently loses the lower levels, and the depth-valued arrays make that immediately visible.
- h5HDF5 Dimension Scales Attached to a 3-D Array (.h5)The CF NetCDF grid expressed with HDF5's own dimension-scale mechanism, so each axis of the 3-D array points at a real coordinate dataset through DIMENSION_LIST references. It lets a reader be tested on scale resolution instead of on filename conventions.
- h5HDF5 Experiment Hierarchy — raw, processed, metadata (.h5)A three-group HDF5 file laid out the way a real bench run is — raw arrays, a processed derivative that names its source in a derived_from attribute, and a metadata group — with units on every dataset. It is the walk-the-tree fixture: a reader has to recurse rather than assume a flat namespace.
- h5HDF5 Hard, Soft and Dangling Links (.h5)One array reachable under four names: itself, a hard link sharing its object address, a soft link resolved at access time, and a soft link pointing nowhere. A walker that counts names instead of object addresses reports four arrays and then crashes on the dangling one.
Generated by generation/scientific.py. Free for any use, no attribution required — license.