Build a Search Index You Can Actually Test

A failing API returns an error. A failing search returns results. They are simply the wrong ones, or in the wrong order, or empty for a query that should have matched, and none of that raises anything a monitor can see.

This site runs a client-side index over several thousand entries, and the failures below are ones we have actually hit rather than ones from a textbook.

Relevance scoring is a weighted sum, and a naive weighting produces the result everyone recognises as broken: you type the precise name of a thing, and the thing itself is fourth.

It happens because a long document can accumulate more partial-match score than a short document whose title is the query. The fix is not a better algorithm, it is a rule: an exact title match outranks everything. Write the assertion that way.

The important detail is what you assert. Testing that the right item is somewhere in the results is the test that passes forever while the product gets worse. Assert the rank.

Anything the tokeniser discards cannot be searched for, and the discards are usually invisible until someone complains.

The cases worth a fixture each:

• Punctuation inside a term. utf-8, x-www-form-urlencoded, .tar.gz. A tokeniser that splits on every non-alphanumeric turns .tar.gz into tar and gz, and a search for the exact extension now matches every tarball mention on the site.
• Digits attached to letters. h264, sha256, base64. Splitting these is how a search for sha256 returns everything about hashing and nothing about SHA-256.
• Case. Almost always folded, and it must be folded the same way at index time and query time. Folding one side only is a bug that presents as "search only works in lowercase".
• Accents. If your content contains them and your index folds them, the query must fold too.

The general rule: the index and the query must run through the same function. Two implementations that agree today will disagree the first time one is edited.

Someone typing geo expects geojson and geospatial. They do not expect every item containing the letters g-e-o anywhere. Prefix matching on token boundaries gives the first behaviour; naive substring gives the second, and the second is why some search boxes feel like they are guessing.

Zero results is a legitimate outcome and the most common thing teams leave unfinished. What the user needs at that moment:

1. Confirmation of what was searched, exactly as interpreted.
2. Which filters are active, because most zero-result states are caused by a filter the user forgot rather than by the query.
3. A way out that is one click: clear filters, or a broader search.

This is testable. Drive a query with no matches, assert the copy names the query, assert an active filter is visible, and assert a control exists that produces results. A zero-result state with no recovery path is a dead end the analytics will show as an exit.

The most valuable search tests do not open a page. They load the index and assert its shape:

• Every entry has the fields the renderer requires. A missing title becomes a blank row.
• No duplicate ids, because duplicates render twice and rank twice.
• The entry count matches the source of truth. An index built from a stale snapshot is the failure mode that produces "I can see the page but search cannot find it".
• Size. A client-side index is downloaded by every visitor who searches, so it belongs under an explicit byte budget, checked in the build rather than noticed later.

That last one has teeth here: the index is capped, and the cap is enforced at build time rather than discovered in a performance report.

A pre-built index is a cache, and caches go stale. The specific shape to guard against: an index built before a content change, served to a browser that then renders results describing entries that have since changed or gone.

The symptom is subtle. Results look right, the summary text is slightly wrong, and a click leads somewhere unexpected. The defence is to treat the index as a build artifact with the same freshness contract as the pages, and to test the case explicitly by serving an older index against newer content and asserting the UI still renders something coherent rather than throwing.

1. Exact title match ranks first. Assert the rank, not the presence.
2. A term with punctuation survives tokenisation.
3. A prefix query returns the expected items and not arbitrary substring hits.
4. Zero results names the query, shows active filters, and offers one-click recovery.
5. The index has no duplicate ids and its count matches the source.
6. The index stays under its byte budget, enforced by the build.

None of these need a search engine. They need the index and a handful of assertions, which is why they are cheap enough to keep.