Browse testing · Novus Examples

Coverage — Markdown Pull-Request Summary

The Markdown comment a coverage bot posts on a pull request: a headline percentage, a per-file table and the uncovered line numbers. Useful for testing Markdown table rendering and for checking that a bot's generated comment matches the machine-readable report it claims to summarise. Every file in this group describes the same four-file source tree and reports 127/140 lines, 17/24 branch outcomes and 18/20 functions, so a converter can be diffed against a known answer.

File: MD · Coverage
Use case: Conversion testingEditor testing+1· Conversion set

Coverage — Per-File Table (CSV)

The same coverage as a flat table with a TOTAL row, for spreadsheets, trend charts and diffing two runs without an XML parser. The TOTAL row is the arithmetic sum of the four file rows, so it doubles as a checksum on any tool that regenerates it. Every file in this group describes the same four-file source tree and reports 127/140 lines, 17/24 branch outcomes and 18/20 functions, so a converter can be diffed against a known answer.

File: CSV · Coverage · 5 rows
Use case: CSV parsingConversion testing+1· Conversion set

Cucumber JSON — Failing Step with Stack Trace

One step of one scenario fails with an assertion message and a stack frame; every other step passes. In cucumber.json a scenario has no status of its own — it is failed if any of its steps is — so this file catches reporters that look for a scenario-level field that does not exist.

File: JSON · Bdd · UTF-8
Use case: JSON parsingError handling+1· Conversion set

Cucumber JSON — Passing Checkout Report

The legacy cucumber.json report for the same run — the format nearly every BDD dashboard still ingests. Durations are in nanoseconds here and in a nested {seconds, nanos} object in the messages twin, which is the single most common unit bug when converting between the two.

File: JSON · Bdd · UTF-8
Use case: JSON parsingConversion testing+1· Conversion set

Cucumber JSON — Undefined, Pending and Skipped Steps

The three non-failure statuses BDD runners produce and dashboards routinely collapse into "failed": undefined (no step definition matched), pending (explicitly unimplemented) and skipped (never attempted). Whether the run is red depends on the runner's strict setting, which is exactly the decision this file forces.

File: JSON · Bdd · UTF-8
Use case: JSON parsingError handling+1· Conversion set

Cucumber JUnit XML — Checkout Run

The same six pickles reported as JUnit XML, which is what a BDD suite hands to a generic CI dashboard. One <testcase> per pickle means the three expanded outline rows appear as three tests with identical names — enough to break any store keyed on name alone.

File: XML · Bdd · UTF-8
Use case: Conversion testingLog parsing+1· Conversion set

Cucumber Messages — Failing Run (NDJSON)

The messages twin of the failing run: one testStepFinished carries status FAILED with both a human message and a structured exception object, and testRunFinished reports success false. Comparing it with the passing stream in the other group isolates exactly which envelopes a failure changes.

File: NDJSON · Bdd · UTF-8
Use case: JSON parsingError handling+1· Conversion set

Cucumber Messages — Passing Checkout Run (NDJSON)

The machine twin of checkout.feature: the cucumber-messages envelope stream a runner emits, with meta, source, gherkinDocument, one pickle per expanded scenario, and paired testStepStarted/testStepFinished envelopes carrying nanosecond durations. Written as NDJSON because that is the format's actual wire shape — one envelope per line, not a JSON array.

File: NDJSON · Bdd · UTF-8
Use case: JSON parsingConversion testing+1· Conversion set

Flaky Detection — 40-Build Test-Run History (JSONL)

Forty builds of six tests, one JSON object per line, with three tests that fail intermittently and three that never do. This is the input shape a flake detector ingests; the CSV twin in this group holds the aggregate it should produce.

File: JSONL · Flaky Runs · 240 records
Use case: JSON parsingTime-series data+1· Conversion set

Flaky Detection — Per-Test Failure-Rate Summary (CSV)

The aggregate answer key for the 40-build history in this group: runs, passes, failures and failure rate per test, with each test labelled stable or flaky. Use it to score a detector rather than eyeballing 240 JSONL records.

File: CSV · Flaky Runs · 6 rows
Use case: CSV parsingConversion testing+1· Conversion set

Flaky Run — Network Dependency, Build #301 (errored)

Build #301 of five consecutive runs of checkout.PaymentTest, in which capturesAuthorisedPayment is errored while the other five cases pass every time. The underlying cause is a real call to the fictional payments host instead of the stub. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Network Dependency, Build #302 (passing)

Build #302 of five consecutive runs of checkout.PaymentTest, in which capturesAuthorisedPayment is passing while the other five cases pass every time. The underlying cause is a real call to the fictional payments host instead of the stub. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Network Dependency, Build #303 (passing)

Build #303 of five consecutive runs of checkout.PaymentTest, in which capturesAuthorisedPayment is passing while the other five cases pass every time. The underlying cause is a real call to the fictional payments host instead of the stub. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Network Dependency, Build #304 (passing)

Build #304 of five consecutive runs of checkout.PaymentTest, in which capturesAuthorisedPayment is passing while the other five cases pass every time. The underlying cause is a real call to the fictional payments host instead of the stub. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Network Dependency, Build #305 (errored)

Build #305 of five consecutive runs of checkout.PaymentTest, in which capturesAuthorisedPayment is errored while the other five cases pass every time. The underlying cause is a real call to the fictional payments host instead of the stub. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Network Dependency, Flake-Rate Table (CSV)

The answer key for the network dependency family: the flake rate a detector should derive from the five build reports next to it. Exactly one of the six tests is unstable, at a rate of 2/5; the other five are stable at 0.00.

File: CSV · Flaky Runs · 6 rows
Use case: CSV parsingConversion testing+1· Conversion set

Flaky Run — Order Dependent, Build #201 (passing)

Build #201 of five consecutive runs of cart.DiscountTest, in which stacksDiscountsInDeclaredOrder is passing while the other five cases pass every time. The underlying cause is a shared coupon cache left populated by a sibling test under random ordering. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Order Dependent, Build #202 (failing)

Build #202 of five consecutive runs of cart.DiscountTest, in which stacksDiscountsInDeclaredOrder is failing while the other five cases pass every time. The underlying cause is a shared coupon cache left populated by a sibling test under random ordering. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Order Dependent, Build #203 (failing)

Build #203 of five consecutive runs of cart.DiscountTest, in which stacksDiscountsInDeclaredOrder is failing while the other five cases pass every time. The underlying cause is a shared coupon cache left populated by a sibling test under random ordering. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Order Dependent, Build #204 (passing)

Build #204 of five consecutive runs of cart.DiscountTest, in which stacksDiscountsInDeclaredOrder is passing while the other five cases pass every time. The underlying cause is a shared coupon cache left populated by a sibling test under random ordering. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Order Dependent, Build #205 (passing)

Build #205 of five consecutive runs of cart.DiscountTest, in which stacksDiscountsInDeclaredOrder is passing while the other five cases pass every time. The underlying cause is a shared coupon cache left populated by a sibling test under random ordering. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Order Dependent, Flake-Rate Table (CSV)

The answer key for the order dependent family: the flake rate a detector should derive from the five build reports next to it. Exactly one of the six tests is unstable, at a rate of 2/5; the other five are stable at 0.00.

File: CSV · Flaky Runs · 6 rows
Use case: CSV parsingConversion testing+1· Conversion set

Flaky Run — Quarantined After Two Failures, Build #401 (failing)

Build #401 of five consecutive runs of cart.PricingTest, in which appliesTaxToSubtotal is failing while the other five cases pass every time. The underlying cause is a rounding boundary that only trips when the tax rate table is reloaded. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Flaky Run — Quarantined After Two Failures, Build #402 (failing)

Build #402 of five consecutive runs of cart.PricingTest, in which appliesTaxToSubtotal is failing while the other five cases pass every time. The underlying cause is a rounding boundary that only trips when the tax rate table is reloaded. Read the whole family in build order to reproduce what a flake detector sees.

File: XML · Flaky Runs
Use case: Error handlingLog parsing+1· Conversion set

Explore the test library

Filter the library

142 results