Novus Examples, Real, documented example files for testing

robots.txt

A robots.txt with wildcard and per-agent rules, a crawl-delay, and a sitemap reference, for testing robots parsers and crawler policy handling.

File: TXT · Crawlers · UTF-8
Use case: Web assetsEditor testing

robots.txt - block everything except one path

A robots.txt that disallows the entire site for every crawler while allowing one media path for a single image agent. For testing full-block handling and the common misconception that a Disallow removes a URL from a search index.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - Crawl-delay, Request-rate and Visit-time

A robots.txt carrying three different crawl-rate hints - an integer Crawl-delay, a fractional one alongside Request-rate and Visit-time, and a very large one - none of which are part of RFC 9309. For testing that a crawler reads or ignores rate hints without dropping the Disallow rules that share the group.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - field case, tabs and indentation

A robots.txt using upper-case, mixed-case and indented field names, a tab-indented rule, a value with no space after the colon, and both trailing and full-line comments. For testing that field names are treated case-insensitively while path values stay case-sensitive.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - five Sitemap directives

A robots.txt declaring five sitemaps - before the first group, inside two different groups, in lower case, gzipped, and on another host. For testing that a discovery crawler collects Sitemap as a file-global field instead of scoping it to the group it sits in.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - merged groups and Allow/Disallow precedence

A robots.txt where Googlebot is named by two separate groups and every Disallow has an equal-length Allow competing with it. For testing group merging, case-insensitive product tokens, and the rule that the most specific match wins with Allow breaking ties.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - unknown and vendor-specific fields

A robots.txt in which real Disallow rules are surrounded by Noindex, Host, Clean-param, Nofollow and two invented fields, several with inline comments. For testing that a parser skips fields it does not implement instead of aborting or mis-binding the rules that follow.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - UTF-8 BOM with CRLF line endings

A robots.txt that begins with a UTF-8 byte-order mark, uses CRLF line endings, and leaves trailing spaces after two rule values. For testing the byte-level tolerances RFC 9309 requires - a leading BOM must be discarded rather than glued onto the first field name.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt - wildcard and $ anchor matching

A robots.txt built entirely from pattern rules: `*` inside a path, `$` anchoring the end of a URL, a query-string pattern, and an Allow that carves one file out of a disallowed subtree. For testing that a robots parser implements RFC 9309 path matching rather than prefix comparison.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt that is actually an HTML 404 page

The single most common broken robots.txt in the wild: a server that answers /robots.txt with its HTML 404 page instead of a 404 status. For testing that a crawler treats unparseable markup as 'no robots.txt' and allows the site rather than inventing rules from tag names.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt with deliberately invalid syntax

A deliberately invalid robots.txt: a rule before any group, an empty product token, an absolute URL where a path belongs, whitespace before the colon, a non-numeric Crawl-delay, and a line with no colon at all. For testing that a parser degrades line-by-line instead of failing the whole file.

File: TXT · Crawlers
Use case: Web assetsWeb scraping+1· Conversion set

robots.txt: Wave G SAMPLE

robots.txt: Wave G SAMPLE for crawler and .well-known tooling.

File: TXT · Well Known
Use case: Web assets

ROC Curve Points (CSV)

An ROC curve as CSV: decision threshold with the corresponding false-positive and true-positive rates, monotonic from (0,0) to (1,1). A fixture for testing chart tools and AUC calculators.

File: CSV · Eval
Use case: Model evaluationCSV parsing+1

ROC Curve Points (JSON)

ROC curve coordinate list for AUC calculator tests.

File: JSON · Eval
Use case: Model evaluation

Rotation Metadata: 180°

A 180-degree rotation carried in the container's display matrix while the coded pixels stay in their original orientation, exactly what a phone writes when you record holding it sideways. A player that ignores the matrix shows it upside down. Thumbnailers and transcoders that read dimensions from the video stream rather than the display matrix get this wrong constantly.

File: MP4 · Rotation · H.264 (libx264)
Use case: Video codecsConversion testing+1· Conversion set

Rotation Metadata: 270°

A 270-degree rotation carried in the container's display matrix while the coded pixels stay in their original orientation, exactly what a phone writes when you record holding it sideways. A player that ignores the matrix shows this 270x480 clip as 480x270, on its side. Thumbnailers and transcoders that read dimensions from the video stream rather than the display matrix get this wrong constantly.

File: MP4 · Rotation · H.264 (libx264)
Use case: Video codecsConversion testing+1· Conversion set

Rotation Metadata: 90°

A 90-degree rotation carried in the container's display matrix while the coded pixels stay in their original orientation, exactly what a phone writes when you record holding it sideways. A player that ignores the matrix shows this 270x480 clip as 480x270, on its side. Thumbnailers and transcoders that read dimensions from the video stream rather than the display matrix get this wrong constantly.

File: MP4 · Rotation · H.264 (libx264)
Use case: Video codecsConversion testing+1· Conversion set

RSA 2048 Private Key (PEM)

A 2048-bit RSA private key in unencrypted PKCS#8 PEM: a deliberately PUBLISHED, sample-only key for testing PEM key parsers and PKCS#8 decoders. Never use it for anything real.

File: KEY · Security
Use case: Certificate & key testingEditor testing

RSA-2048 Private Key (PEM): SAMPLE

Published SAMPLE RSA-2048 private key (PKCS#8 PEM). Never use for a real identity.

File: KEY · Pki
Use case: Certificate & key testing· Conversion set

RSS 2.0 Feed

A well-formed RSS 2.0 feed with a channel and multiple items, for testing feed readers and parsers.

File: RSS · Feeds · UTF-8
Use case: Feed parsing

RTF: Formatted Document

A Rich Text Format document exercising bold, italic, underline, colour, headings, and a bullet list: all as plain control words. For testing RTF parsers, text extraction, and RTF→DOCX/PDF conversion.

File: RTF · Rich Text · ASCII
Use case: Conversion testing

RTF: Plain Document

The simplest possible RTF: one monospace font and a couple of paragraphs, no styling. A minimal baseline for RTF parsers and conversion tools.

File: RTF · Rich Text · ASCII
Use case: Conversion testing

RTL UI Pack: Arabic

Arabic RTL UI string pack SAMPLE for bidirectional layout and font-fallback tests.

File: JSON · Rtl
Use case: InternationalizationLocalization catalogs+1· Conversion set

RTL UI Pack: Arabic Egypt

Arabic Egypt RTL UI string pack SAMPLE for bidirectional layout and font-fallback tests.

File: JSON · Rtl
Use case: InternationalizationLocalization catalogs+1· Conversion set

Explore the test library

Filter the library

5297 results