float64 to float32 Precision Loss — Per-Value Error (.csv)
Twelve doubles narrowed to single precision with the absolute, relative and ULP error of each, plus whether the value survived unchanged. It covers the three distinct ways narrowing fails: rounding, overflow to infinity at 1e39, and demotion to a subnormal at 1e-40.
value_float64_repr,value_float32_repr,absolute_error,relative_error,ulp_error_float32,round_trips_exactly,float32_class
0.1,0.10000000149011612,1.4901161138336505e-09,1.490116e-08,0.2000,false,normal
0.3333333333333333,0.3333333432674408,9.9341074810688212e-09,2.980232e-08,0.3333,false,normal
3.141592653589793,3.1415927410125732,8.7422780126189537e-08,2.782754e-08,0.3667,false,normal
16777217.0,16777216.0,1,5.960464e-08,0.5000,false,normal
123456789.12345679,123456792.0,2.8765432089567184,2.330000e-08,0.3596,false,normal
1e-40,9.99994610111476e-41,5.3898885239711403e-46,5.389889e-06,0.3846,false,normal
3.4e+38,3.3999999521443642e+38,4.7855635712161142e+30,1.407519e-08,0.2359,false,normal
1e+39,inf,inf,inf,inf,false,infinity
2.2250738585072014e-308,0.0,2.2250738585072014e-308,1.000000e+00,inf,false,positive zero
6.02214076e+23,6.022140643549849e+23,11645015092625408,1.933700e-08,0.3232,false,normal
1.602176634e-19,1.602176597458587e-19,3.6541412966041288e-27,2.280736e-08,0.2827,false,normal
0.30000000000000004,0.30000001192092896,1.1920928910669204e-08,3.973643e-08,0.4000,false,normal
Specifications
- Rows
- 12
- Columns
- 7
- Error Measures
- absolute, relative and ULP
- Overflow Case
- 1e39 becomes +Inf in float32
- Underflow Case
- 1e-40 becomes a float32 subnormal
- Exact Cases
- 0
Testing contract
Reference control- Scenario
- Narrow each value_float64_repr to float32 in your own pipeline and compare the results against value_float32_repr.
- Expected result
- Every value matches, 1e39 becomes +Inf rather than a large finite number, 1e-40 arrives as a float32 subnormal, and 16777217.0 lands on 16777216.0 — a whole integer of error, half a float32 ULP at that magnitude.
What is a .csv file?
CSV (Comma-Separated Values) is a plain-text tabular format where rows are lines and fields are separated by commas, with quoting rules for values that contain delimiters, quotes, or newlines. It has no formal type system and depends on encoding and dialect conventions. It is the most portable format for tabular data exchange.
How to use this file
Use an example CSV to test parsers against quoting and embedded-delimiter edge cases, header handling, encoding detection, and import pipelines into databases or spreadsheets.
How to use this file for testing
“float64 to float32 Precision Loss — Per-Value Error (.csv)” is a deterministic Novus Examples fixture for Scientific data, Editor testing, Conversion testing. Citation catalogs (BibTeX, RIS), chemistry structures (MDL Molfile, PDB), and gridded binary data (NetCDF, FITS) — for testing reference managers, molecule viewers, and scientific-data loaders.
Documented properties for this file: 12 rows · 7 columns. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Scientific fixtures are small, valid, and fully synthetic — no real organism, patient, sample, or observation. Point your parser or loader at the file and check it reads the documented records, variables, or headers; binary formats ship a readable twin or metadata listing for comparison.
Code examples
import pandas as pd
df = pd.read_csv("float64-to-float32-precision-loss.csv")
print(df.head())
print(df.dtypes)Related files
- csvfloat64 Subnormal Ladder — Reference Table (.csv)Each rung of the subnormal ladder with its bit pattern, decimal value, C99 hex float and remaining significant bits, plus the value a flush-to-zero pipeline would produce instead. It makes gradual underflow a table you can diff rather than a behaviour you have to infer.
- csvIEEE-754 float32 Boundary Values — Bit-Level CSV (.csv)The float32 boundary values split into sign, 8-bit exponent and 23-bit mantissa alongside their decimal forms. Use it to check a single-precision encoder field by field rather than guessing which end of the number went wrong.
- csvIEEE-754 float64 Boundary Values — Bit-Level CSV (.csv)Every value in the float64 boundary array decomposed into its shortest decimal representation, its C99 hexadecimal float form, and its raw sign, exponent and mantissa fields. It turns 'the numbers came back wrong' into a diff you can point at a single bit.
- csvNaN Payloads and Signalling NaNs — Bit Reference (.csv)The six NaN bit patterns broken into sign bit, quiet bit and payload so the difference between a quiet and a signalling NaN is a column rather than folklore. It is the reference for testing whether a serializer preserves NaN payloads or normalises them all to one canonical pattern.
- csvRaw float64 Endian Pair — Byte-Level Reference (.csv)Every value in the raw endian pair with its exact byte sequence in both orders, so a decoder can be checked offset by offset instead of value by value. Negative zero and the subnormal entries are the rows where a byte-order mistake stops looking like a mistake.
- csvChemistry Fixture Molecule Index (.csv)One row per molecule used across the chemistry fixtures, with formula, molecular weight, SMILES and the atom and bond counts each file should yield. It is the oracle a cheminformatics toolkit can be scored against without needing a second toolkit to generate the answers.
Generated by generation/scientific.py. Free for any use, no attribution required — license.