JUnit XML — Error and Failure Distinguished
One <failure> (an assertion that did not hold) and one <error> (an exception the test never expected), side by side. Parsers that collapse the two lose the distinction between a broken assertion and a broken environment.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="checkout.AllTests" tests="3" failures="1" errors="1" skipped="0" time="1.614" timestamp="2026-08-07T09:14:22" hostname="ci-runner-04">
<testcase classname="checkout.SessionTest" name="createsSessionForNewCart" time="0.092"/>
<testcase classname="checkout.SessionTest" name="expiresSessionAfterTimeout" time="1.204">
<error message="Connection refused: sessions.example.com:5432" type="java.net.ConnectException"><![CDATA[java.net.ConnectException: Connection refused: sessions.example.com:5432
at checkout.SessionStore.open(SessionStore.java:41)
at checkout.SessionTest.expiresSessionAfterTimeout(SessionTest.java:57)
]]></error>
</testcase>
<testcase classname="cart.PricingTest" name="appliesTaxToSubtotal" time="0.318">
<failure message="expected:<12.60> but was:<12.59>" type="java.lang.AssertionError"><![CDATA[java.lang.AssertionError: expected:<12.60> but was:<12.59>
at cart.PricingTest.appliesTaxToSubtotal(PricingTest.java:88)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
]]></failure>
</testcase>
</testsuite>
</testsuites>
Specifications
- Seed
- 61200
- Dialect
- surefire
- Encoding
- UTF-8
- Line Endings
- LF
- Tests
- 3
- Failures
- 1
- Errors
- 1
- Skipped
- 0
- Passed
- 1
Testing contract
Expected to pass- Scenario
- Separate assertion failures from unexpected errors in one JUnit report.
- Expected result
- failures=1 and errors=1 are reported independently; the error carries type java.net.ConnectException and the failure java.lang.AssertionError.
What is a .xml file?
XML (Extensible Markup Language) is a verbose, self-describing markup language using nested tags, attributes, and namespaces to represent structured, hierarchical data. It supports schemas, entities, and validation and underlies many document and data formats. It remains common in enterprise, publishing, and interchange contexts.
How to use this file
Use an example XML file to test parsers, namespace and schema validation, XPath queries, and protection against entity-expansion and external-entity attacks.
How to use this file for testing
“JUnit XML — Error and Failure Distinguished” is a deterministic Novus Examples fixture for Error handling, Conversion testing, Log parsing. Deliberately corrupt and invalid files, clearly labelled, for testing how your tool fails.
Documented properties for this file: seed 61200 · UTF-8 · LF. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Test and coverage reports document their totals (suites, cases, passes, failures, skips, covered lines) in the spec table. Point your CI dashboard, coverage gate, or report converter at the file and assert those counts survive; format twins carry identical numbers so a conversion can be scored exactly.
Code examples
import xml.etree.ElementTree as ET
tree = ET.parse("outcome-error-vs-failure.xml")
root = tree.getroot()
print(root.tag, [c.tag for c in root][:5])Related files
- xmlJUnit XML — @Playwright/Test Dialect (playwright)The same 12-case Novus Checkout run written the way @playwright/test 1.47 junit reporter writes it. Playwright puts the run totals on the <testsuites> root, names each suite after its spec file, and uses the bare describe title as classname. All eight dialect files in this group report 12 tests, 1 failure, 1 error and 1 skip, so a JUnit parser can be held to identical totals across every producer.
- xmlJUnit XML — Console Output in CDATACaptured stdout and stderr wrapped in CDATA, where the stdout text itself looks like XML markup. A parser must treat the CDATA content as opaque characters; one that re-parses it finds a <report> element that does not exist.
- xmlJUnit XML — Go-Junit-Report Dialect (go-junit-report)The same 12-case Novus Checkout run written the way go-junit-report 2.1 writes it. go-junit-report names each suite after a Go package and carries the toolchain version in a go.version property. All eight dialect files in this group report 12 tests, 1 failure, 1 error and 1 skip, so a JUnit parser can be held to identical totals across every producer.
- xmlJUnit XML — Gradle Dialect (gradle)The same 12-case Novus Checkout run written the way Gradle 8.9 Test task writes it. Gradle writes hostname and timestamp on every suite and an empty self-closing <properties/> element that some parsers mishandle. All eight dialect files in this group report 12 tests, 1 failure, 1 error and 1 skip, so a JUnit parser can be held to identical totals across every producer.
- xmlJUnit XML — Jest-Junit Dialect (jest-junit)The same 12-case Novus Checkout run written the way jest-junit 16 writes it. jest-junit repeats the whole "describe > it" title in both classname and name, which breaks dashboards that assume classname is a package. All eight dialect files in this group report 12 tests, 1 failure, 1 error and 1 skip, so a JUnit parser can be held to identical totals across every producer.
- xmlJUnit XML — Maven-Surefire Dialect (surefire)The same 12-case Novus Checkout run written the way maven-surefire 3.2 writes it. Maven Surefire merges its per-class files under a <testsuites> root and adds a <properties> block plus system-out/system-err on every suite. All eight dialect files in this group report 12 tests, 1 failure, 1 error and 1 skip, so a JUnit parser can be held to identical totals across every producer.
Generated by generation/testing.py. Free for any use, no attribution required — license.