Skip to content
Novus Examples

Data-engineering & pipeline fixtures

Columnar (Parquet/ORC/Feather), row (Avro), and messaging (MessagePack/CBOR/Protobuf) formats plus star-schema and log data — for testing ETL, data-lake ingestion, and warehouse loaders.

39 of 39 files
Preview of Parquet — Columnar Table
parquet
2.4 KB
Actual file preview for Parquet — Columnar Table

Parquet — Columnar Table

A small employee table as Apache Parquet — the columnar format at the heart of modern data lakes and analytics. For testing Parquet readers (pandas, Spark, DuckDB) and conversion.

File
PARQUET · Binary · 5 rows
Use case
Conversion testingData engineering· Conversion set
Preview of ORC — Columnar Table
orc
1.5 KB
Actual file preview for ORC — Columnar Table

ORC — Columnar Table

The same employee table as Apache ORC — the columnar format common in the Hive/Hadoop ecosystem. For testing ORC readers and Parquet↔ORC conversion.

File
ORC · Binary · 5 rows
Use case
Conversion testingData engineering· Conversion set
Preview of Feather — Arrow IPC Table
feather
1.8 KB
Actual file preview for Feather — Arrow IPC Table

Feather — Arrow IPC Table

The same table as Feather (Arrow IPC file) — the zero-copy on-disk form of an Apache Arrow table. For testing Arrow readers and fast columnar interchange.

File
FEATHER · Binary · 5 rows
Use case
Conversion testingData engineering· Conversion set
Preview of Avro — Row Binary + Schema
avro
783 B
Actual file preview for Avro — Row Binary + Schema

Avro — Row Binary + Schema

The same records as Apache Avro — a compact row-based binary format that embeds its own schema, widely used in Kafka pipelines. For testing Avro decoders and schema evolution.

File
AVRO · Binary · 5 rows
Use case
Conversion testingData engineering· Conversion set
Preview of HDF5 — Hierarchical Scientific Data
h5
6.3 KB
Actual file preview for HDF5 — Hierarchical Scientific Data

HDF5 — Hierarchical Scientific Data

A small HDF5 file with a compound 'employees' dataset and a numeric 'readings' grid — the hierarchical format used across science and ML. For testing h5py/HDF5 readers and conversion.

File
H5 · Binary · 5 rows
Preview of E-commerce Products (CSV, 200 rows)
csv
10.9 KB
Actual file preview for E-commerce Products (CSV, 200 rows)

E-commerce Products (CSV, 200 rows)

A realistic e-commerce product catalogue (200 rows) — part of a relational dataset (products, customers, orders) with CSV, JSON, SQL, and Parquet twins for testing joins, imports, and conversion.

File
CSV · Ecommerce · 200 rows
Use case
Data importConversion testing+1· Conversion set
Preview of E-commerce Customers (CSV, 500 rows)
csv
33.1 KB
Actual file preview for E-commerce Customers (CSV, 500 rows)

E-commerce Customers (CSV, 500 rows)

A realistic e-commerce customer directory (500 rows) — part of a relational dataset (products, customers, orders) with CSV, JSON, SQL, and Parquet twins for testing joins, imports, and conversion.

File
CSV · Ecommerce · 500 rows
Use case
Data importConversion testing+1· Conversion set
Preview of E-commerce Orders (CSV, 2000 rows)
csv
77.6 KB
Actual file preview for E-commerce Orders (CSV, 2000 rows)

E-commerce Orders (CSV, 2000 rows)

A realistic e-commerce order lines (customer_id → customers, product_id → products) (2000 rows) — part of a relational dataset (products, customers, orders) with CSV, JSON, SQL, and Parquet twins for testing joins, imports, and conversion.

File
CSV · Ecommerce · 2,000 rows
Use case
Data importConversion testing+1· Conversion set
Preview of Star Schema — Date Dimension (CSV)
csv
1.3 KB
Actual file preview for Star Schema — Date Dimension (CSV)

Star Schema — Date Dimension (CSV)

The date dimension of a star schema — one row per day with a YYYYMMDD surrogate key and calendar attributes (year, quarter, month, weekday). Joins to the sales fact table on date_key.

File
CSV · Star Schema · 31 rows
Use case
Data importData engineering+1· Conversion set
Preview of Star Schema — Product Dimension (CSV)
csv
918 B
Actual file preview for Star Schema — Product Dimension (CSV)

Star Schema — Product Dimension (CSV)

The product dimension of a star schema — surrogate key, SKU, name, category, and unit price. Joins to the sales fact table on product_key.

File
CSV · Star Schema · 20 rows
Use case
Data importData engineering+1· Conversion set
Preview of Star Schema — Customer Dimension (CSV)
csv
1.4 KB
Actual file preview for Star Schema — Customer Dimension (CSV)

Star Schema — Customer Dimension (CSV)

The customer dimension of a star schema — surrogate key, name, city, country, and segment. Joins to the sales fact table on customer_key. Names are synthetic.

File
CSV · Star Schema · 40 rows
Use case
Data importData engineering+1· Conversion set
Preview of Star Schema — Sales Fact Table (CSV, 300 rows)
csv
8.2 KB
Actual file preview for Star Schema — Sales Fact Table (CSV, 300 rows)

Star Schema — Sales Fact Table (CSV, 300 rows)

The sales fact table at the centre of a star schema — 300 sale lines with foreign keys to the date, product, and customer dimensions plus quantity and amount measures. A fixture for testing joins, star-schema imports, and BI tools.

File
CSV · Star Schema · 300 rows
Use case
Data importData engineering+1· Conversion set
Preview of Convert v2 ORC Employee Table Source
orc
1.5 KB
Actual file preview for Convert v2 ORC Employee Table Source

Convert v2 ORC Employee Table Source

Binary orc source for the five-row P8 employee conversion table, preserving ids, names, departments, booleans, and scores. Stable P8 artifact p8-convert-orc-source.

File
ORC · P8 Convert · 5 rows
Use case
Conversion testingData engineering+1· Conversion set
Preview of Convert v2 ORC Expected JSON
json
977 B
Actual file preview for Convert v2 ORC Expected JSON

Convert v2 ORC Expected JSON

Json semantic reference for the five-row P8 employee conversion table, preserving ids, names, departments, booleans, and scores. Stable P8 artifact p8-convert-orc-json.

File
JSON · P8 Convert · 5 rows
Use case
Conversion testingData engineering+1· Conversion set
Preview of Convert v2 ORC Expected CSV
csv
434 B
Actual file preview for Convert v2 ORC Expected CSV

Convert v2 ORC Expected CSV

Csv semantic reference for the five-row P8 employee conversion table, preserving ids, names, departments, booleans, and scores. Stable P8 artifact p8-convert-orc-csv.

File
CSV · P8 Convert · 5 rows
Use case
Conversion testingData engineering+1· Conversion set
Preview of Text Embeddings — 16-dim (Parquet)
parquet
9.3 KB
Actual file preview for Text Embeddings — 16-dim (Parquet)

Text Embeddings — 16-dim (Parquet)

The same 16-dimensional embeddings as Apache Parquet — id and text columns plus one column per dimension. The columnar twin, for testing analytics engines and Parquet-based vector pipelines.

File
PARQUET · Embeddings · 24 rows
Use case
ML training dataEmbeddings+2· Conversion set
Preview of Git Log - --numstat With a Binary Row
log
585 B
Actual file preview for Git Log - --numstat With a Binary Row

Git Log - --numstat With a Binary Row

Machine-readable insertion and deletion counts per file. Two rows break naive arithmetic: the binary file reports `-` in both numeric columns rather than 0, and the rename path is brace-compressed into `docs/{README.md => guide.md}` rather than written out twice.

File
LOG · History · UTF-8
Preview of Git Shortlog - Commits Grouped by Author
log
333 B
Actual file preview for Git Shortlog - Commits Grouped by Author

Git Shortlog - Commits Grouped by Author

The author-grouped summary git generates for release notes: a name, a commit count in parentheses, and the subjects indented by six spaces. The counts here are what the .mailmap fixture in this category produces after coalescing aliases, so the two files can be used together.

File
LOG · History · UTF-8
Preview of Changelog - Keep a Changelog Layout
md
957 B
Actual file preview for Changelog - Keep a Changelog Layout

Changelog - Keep a Changelog Layout

A hand-maintained changelog in the Keep a Changelog layout: an Unreleased section, dated version headings in square brackets, the standard change-type subheadings, and reference-style compare links at the bottom. It describes the same releases as the generated twin in this group, so a converter between the two can be scored.

File
MD · History · UTF-8
Use case
Config parsingVisual diff / regression+1· Conversion set
Preview of Changelog - Generated From Conventional Commits
md
1.3 KB
Actual file preview for Changelog - Generated From Conventional Commits

Changelog - Generated From Conventional Commits

The machine-generated twin of the hand-written changelog: version headings that are themselves compare links, a leading warning glyph on the breaking-changes section, and every entry ending in a short-hash link back to the commit. The hashes are the same fictional ids used by the log fixtures in this category.

File
MD · History · UTF-8
Use case
Config parsingVisual diff / regression+1· Conversion set
Preview of OTLP Trace — Spans Batched in Reverse Completion Order (json)
json
27.9 KB
Actual file preview for OTLP Trace — Spans Batched in Reverse Completion Order (json)

OTLP Trace — Spans Batched in Reverse Completion Order (json)

The canonical checkout trace with its spans emitted in the order they finished — children before parents, deepest first. This is what a real batch exporter produces, and a consumer that assumes parents arrive first will drop or orphan most of the trace.

File
JSON · Traces
Use case
ObservabilityJSON parsing+1· Conversion set
Preview of OTLP Traces — NDJSON Stream, One Resource per Line (jsonl)
jsonl
14.6 KB
Actual file preview for OTLP Traces — NDJSON Stream, One Resource per Line (jsonl)

OTLP Traces — NDJSON Stream, One Resource per Line (jsonl)

Two traces streamed as newline-delimited OTLP requests, one resource per line, so a single trace is split across several lines exactly as a line-oriented collector sink writes it. A fixture for streaming ingestion, back-pressure and partial-trace assembly.

File
JSONL · Traces
Preview of OTLP Logs — NDJSON Stream, One Record per Line (jsonl)
jsonl
7.2 KB
Actual file preview for OTLP Logs — NDJSON Stream, One Record per Line (jsonl)

OTLP Logs — NDJSON Stream, One Record per Line (jsonl)

The same five OTLP log records reframed as one self-contained export per line, with the resource attributes repeated on every line. The trade a line-oriented sink makes: each line stands alone, at the cost of repeating the resource block five times.

File
JSONL · Logs
Use case
ObservabilityJSON parsing+1· Conversion set
Preview of Structured Log — Sampling Decisions and Drop Counters (jsonl)
jsonl
12.5 KB
Actual file preview for Structured Log — Sampling Decisions and Drop Counters (jsonl)

Structured Log — Sampling Decisions and Drop Counters (jsonl)

Logs that record their own sampling decision, with periodic dropped_since_last counters so the true event rate can be reconstructed from a sampled stream. Counting sampled records as if they were the whole population is the mistake this fixture is built to expose.

File
JSONL · Logs
Use case
ObservabilityLog parsing+1· Conversion set
Preview of Kestrel Bay Territory — Line-Delimited GeoJSON
ndjson
3.3 KB
Actual file preview for Kestrel Bay Territory — Line-Delimited GeoJSON

Kestrel Bay Territory — Line-Delimited GeoJSON

The territory as one GeoJSON Feature per line, the shape streaming pipelines and BigQuery/Athena loaders expect. There is no enclosing FeatureCollection, so a reader that insists on one will reject it.

File
NDJSON · Territory · 14 records
Use case
GeospatialJSON parsing+1· Conversion set
Preview of Argo Workflow DAG Template
yaml
1.8 KB
Actual file preview for Argo Workflow DAG Template

Argo Workflow DAG Template

An Argo Workflows DAG template carrying the same six-task ETL topology as the Airflow, Graphviz and Mermaid fixtures in this category, so a converter or visualiser can be scored against one known answer across four formats.

File
YAML · Argo Tekton
Preview of Argo Workflow Steps Template
yaml
1.1 KB
Actual file preview for Argo Workflow Steps Template

Argo Workflow Steps Template

Argo's steps template, whose double-nested list is a genuine parser trap: the outer list is sequential and the inner list is parallel, so a reader that flattens it reports four sequential steps instead of three groups.

File
YAML · Argo Tekton
Preview of Airflow DAG Definition (Operator Style)
py
1.6 KB
Actual file preview for Airflow DAG Definition (Operator Style)

Airflow DAG Definition (Operator Style)

An Airflow DAG definition in the classic operator style, using only the no-op EmptyOperator so the file describes a topology and performs no work. Carries the six-task ETL graph shared across the Airflow JSON, Argo, Graphviz and Mermaid fixtures.

File
PY · Airflow
Use case
Graph dataData engineering+1· Conversion set
Preview of Airflow TaskFlow API DAG
py
1.1 KB
Actual file preview for Airflow TaskFlow API DAG

Airflow TaskFlow API DAG

A decorator-based Airflow TaskFlow DAG where dependencies are implied by function calls rather than >> operators — the shape static DAG extractors most often get wrong. Every task returns a literal, so nothing performs I/O.

File
PY · Airflow
Preview of Airflow-Shaped Serialized DAG (JSON)
json
2.5 KB
Actual file preview for Airflow-Shaped Serialized DAG (JSON)

Airflow-Shaped Serialized DAG (JSON)

The serialized-DAG shape Airflow stores in its metadata database, as standalone JSON: per-task metadata with explicit downstream_task_ids. Carries the same six-task topology as the Python, Argo, Graphviz and Mermaid fixtures.

File
JSON · Airflow
Use case
Graph dataData engineering+1· Conversion set
Preview of Pipeline DAG Topology (JSON, with Expected Answers)
json
1.6 KB
Actual file preview for Pipeline DAG Topology (JSON, with Expected Answers)

Pipeline DAG Topology (JSON, with Expected Answers)

The shared six-task topology as a plain node/edge JSON document that also publishes the expected answers — roots, leaves, longest path, and topological order — so a graph library or DAG visualiser can be scored without a hand-written oracle.

File
JSON · Airflow
Use case
Graph dataData engineering+1· Conversion set
Preview of CWL Workflow (Six-Step Topology)
cwl
1.3 KB
Actual file preview for CWL Workflow (Six-Step Topology)

CWL Workflow (Six-Step Topology)

A CWL Workflow carrying the same six-step ETL topology as the Airflow and Argo fixtures, with dependencies expressed as step output references and a multi-source input resolved by pickValue. For cross-format DAG conversion tests.

File
CWL · Cwl
Preview of CWL Scatter and Cross-Product
cwl
808 B
Actual file preview for CWL Scatter and Cross-Product

CWL Scatter and Cross-Product

CWL scatter in both forms: a single-parameter scatter over an array input, and a two-parameter scatter with scatterMethod flat_crossproduct. The scatter dimensionality is what distinguishes correct CWL engines from approximate ones.

File
CWL · Cwl
Preview of Graphviz Pipeline DAG
dot
1016 B
Actual file preview for Graphviz Pipeline DAG

Graphviz Pipeline DAG

The shared six-task ETL topology as a Graphviz digraph, with node and edge defaults, per-node attribute overrides, a same-rank constraint and C-style comments. Carries the identical graph to the Airflow, Argo, CWL and Mermaid fixtures.

File
DOT · Dag Graphs
Preview of Graphviz Pipeline Graph with a Cycle
dot
718 B
Actual file preview for Graphviz Pipeline Graph with a Cycle

Graphviz Pipeline Graph with a Cycle

The same topology plus one back edge, so the graph is deliberately not a DAG. A renderer must still draw it and a scheduler must reject it, which is why this is a reference fixture rather than a corrupt file.

File
DOT · Dag Graphs
Preview of Mermaid Pipeline Flowchart
mmd
714 B
Actual file preview for Mermaid Pipeline Flowchart

Mermaid Pipeline Flowchart

The shared six-task ETL topology as a Mermaid flowchart, with mixed node shapes, a subgraph, a classDef and %% comments. The Mermaid twin of the Graphviz fixture, for scoring diagram-format converters against one known answer.

File
MMD · Dag Graphs
Preview of Mermaid Deployment State Diagram
mmd
621 B
Actual file preview for Mermaid Deployment State Diagram

Mermaid Deployment State Diagram

A Mermaid stateDiagram-v2 describing environment promotion with canary and rollback transitions, start and end pseudostates, and a note block. A second Mermaid diagram type, so a renderer is not only tested on flowcharts.

File
MMD · Dag Graphs