UTF-8 Text (LF)
A UTF-8 text file with Unix (LF) line endings and no byte-order mark, containing accented and symbol characters — for testing charset detection.
Text encoding test files
UTF-8, UTF-8-BOM, UTF-16, and Latin-1 files with documented encodings and line endings for testing charset detection.
UTF-8 Text with BOM (CRLF)
A UTF-8 text file that starts with a byte-order mark (EF BB BF) and uses Windows (CRLF) line endings — for testing BOM handling and line-ending detection.
UTF-16 LE Text (CRLF)
A UTF-16 little-endian text file with a BOM (FF FE) and CRLF line endings — for testing wide-character decoding and encoding detection.
UTF-16 BE Text (BOM)
A UTF-16 big-endian text file with a BOM (FE FF) and CRLF line endings — for testing wide-character decoding and byte-order detection.
UTF-32 LE Text (BOM)
A UTF-32 little-endian text file with a BOM (FF FE 00 00) — four bytes per code point, for testing full-width Unicode decoders and byte-order handling.
Latin-1 (ISO-8859-1) Text
A single-byte Latin-1 (ISO-8859-1) text file using high-range accented characters — for testing legacy 8-bit charset detection and transcoding to UTF-8.
Windows-1252 Text
A Windows-1252 text file using the 0x80–0x9F range (smart quotes, em dash, euro, bullet) that naive Latin-1 decoders get wrong — for testing codepage detection and transcoding.
Shift-JIS Japanese Text
A Shift-JIS encoded Japanese text file — a multi-byte East-Asian encoding, for testing CJK charset detection and Shift-JIS→UTF-8 conversion.
Mixed Line Endings Text
A UTF-8 text file that deliberately mixes LF, CRLF, and lone-CR line endings within one file — for testing newline normalisation and line-counting logic.
Right-to-Left / Bidi Text
A UTF-8 file mixing left-to-right and right-to-left scripts (Arabic and Hebrew alongside English) — for testing bidirectional text handling, reordering, and rendering.
Emoji & ZWJ Sequences Text
A UTF-8 file of emoji including zero-width-joiner sequences, regional-indicator flag pairs, and skin-tone modifiers — multi-code-point grapheme clusters for testing grapheme segmentation and display-width calculation.
Combining Marks (NFC vs NFD) Text
A UTF-8 file with the same words in both NFC (precomposed) and NFD (decomposed combining marks) normalisation forms — for testing normalisation-aware comparison, search, and dedup.
Very Long Single Line Text
A UTF-8 file consisting of a single very long line (~18,000 characters with no interior newline) — for testing editors, buffers, and line-oriented parsers against long-line handling.
Latin-1 Encoded CSV
A CSV encoded in Latin-1 (ISO-8859-1) with accented names — for testing encoding detection and mis-decoding (it will look wrong if read as UTF-8).