Censored Values and Detection Limits — Non-Numeric Results (.csv)
Nine laboratory results where only four are plain numbers: two are below the detection limit, one is above range, two are missing in different spellings, and one is a legitimate small negative near the blank. Coercing the censored strings to numbers or to NaN both bias the summary, and the file distinguishes every case explicitly.
sample_id,analyte,result_mg_L,detection_limit_mg_L,censoring,note
S-001,nitrate,0.421,0.050,none,a plain numeric result
S-002,nitrate,<0.050,0.050,left,below the detection limit
S-003,nitrate,ND,0.050,left,"not detected, reported as text"
S-004,nitrate,>1000,0.050,right,above the calibrated range
S-005,nitrate,,0.050,missing,"empty field, no measurement attempted"
S-006,nitrate,n/a,0.050,missing,a different spelling of the same thing
S-007,nitrate,0.0,0.050,none,"a real zero, not a censored value"
S-008,nitrate,-0.003,0.050,none,a legitimate negative near the blank
S-009,nitrate,0.500,0.050,none,a plain numeric result
Specifications
- Rows
- 9
- Columns
- 6
- Numeric Rows
- 4
- Left Censored
- 2
- Right Censored
- 1
- Missing Rows
- 2
- Detection Limit
- 0.05
- Trap
- coercing '<0.050' to 0.05 or to NaN both change the statistics
Testing contract
Expected to recover- Scenario
- Parse the result column and classify each row as numeric, left-censored, right-censored or missing without coercing text to numbers.
- Expected result
- Exactly four rows are numeric including the legitimate -0.003 and the real 0.0, two are left-censored, one is right-censored and two are missing — matching the censoring column in every row.
What is a .csv file?
CSV (Comma-Separated Values) is a plain-text tabular format where rows are lines and fields are separated by commas, with quoting rules for values that contain delimiters, quotes, or newlines. It has no formal type system and depends on encoding and dialect conventions. It is the most portable format for tabular data exchange.
How to use this file
Use an example CSV to test parsers against quoting and embedded-delimiter edge cases, header handling, encoding detection, and import pipelines into databases or spreadsheets.
How to use this file for testing
“Censored Values and Detection Limits — Non-Numeric Results (.csv)” is a deterministic Novus Examples fixture for Scientific data, CSV parsing, Editor testing, Error handling. Citation catalogs (BibTeX, RIS), chemistry structures (MDL Molfile, PDB), and gridded binary data (NetCDF, FITS) — for testing reference managers, molecule viewers, and scientific-data loaders.
Documented properties for this file: 9 rows · 6 columns. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Scientific fixtures are small, valid, and fully synthetic — no real organism, patient, sample, or observation. Point your parser or loader at the file and check it reads the documented records, variables, or headers; binary formats ship a readable twin or metadata listing for comparison.
Code examples
import pandas as pd
df = pd.read_csv("censored-and-detection-limits.csv")
print(df.head())
print(df.dtypes)Related files
- csvCalibration Certificate — Error and Expanded Uncertainty (.csv)Five calibration points with the nominal value, the measured value, the signed error and the expanded uncertainty at k=2 with its confidence level stated. It is a synthetic illustration of the layout, not an accredited certificate, and nothing in it refers to a real instrument or laboratory.
- csvSignificant Figures Lost to Float Parsing (.csv)Eight written measurements where the trailing zeros carry the precision of the measurement, alongside what a float parse leaves behind. Parsing 1.200 to the double 1.2 is not a rounding error, it is the loss of the claim that the value is known to four figures.
- csvSpectrometer Absorbance With Expanded Uncertainty (.csv)A synthetic absorbance scan from 400 to 630 nm with a Gaussian band at 520 nm, carrying its expanded uncertainty at k=2 in a dedicated column and a per-row quality flag. The unit and the coverage factor live in the column names, which is the convention most instrument exports actually use.
- csvThermocouple Time Series With Standard Uncertainty (.csv)A full day of thermocouple readings at 30-minute cadence with a constant type-B standard uncertainty and two rows flagged as recovering. The timestamps are explicit UTC with a Z suffix so a reader has no excuse to guess a timezone.
- csvUnits on a Second Header Row — Not Data (.csv)The same instrument export with units on a second header row instead of inside the column names, which is how a great many laboratory instruments write CSV. An importer that treats row two as data types every measurement column as text and then quietly refuses to plot anything.
- csvBinary versus Decimal Arithmetic — Where Floats Disagree (.csv)Eight arithmetic expressions where binary floating point and exact decimal disagree, with both answers and the signed error side by side. Adding one cent a hundred times does not give one, and this file says by exactly how much.
Generated by generation/scientific.py. Free for any use, no attribution required — license.