jsonl12.5 KB
Structured Log — Sampling Decisions and Drop Counters (jsonl)
Logs that record their own sampling decision, with periodic dropped_since_last counters so the true event rate can be reconstructed from a sampled stream. Counting sampled records as if they were the whole population is the mistake this fixture is built to expose.
Preview — first 31 linesjsonl
{"ts":"2026-03-17T09:14:33.044Z","level":"warn","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"slow downstream call","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":125.6},"request_id":"req-000001","order_id":"ord-2026-0317-001650","seq":1,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.117Z","level":"warn","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"slow downstream call","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":342.27},"request_id":"req-000002","order_id":"ord-2026-0317-002646","seq":2,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.212Z","level":"warn","service":"inventory-api","version":"2.3.0","env":"staging","host":"node-b1","logger":"shop/inventory/reserve","msg":"reservation retried after conflict","http":{"method":"POST","route":"/api/orders","status":200,"duration_ms":199.27},"request_id":"req-000003","order_id":"ord-2026-0317-004232","seq":3,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.260Z","level":"info","service":"catalog","version":"0.0.1","env":"staging","host":"node-c1","logger":"shop/catalog/handler","msg":"request completed","http":{"method":"GET","route":"/api/catalog","status":200,"duration_ms":138.59},"request_id":"req-000004","order_id":"ord-2026-0317-004598","seq":4,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.305Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":130.95},"request_id":"req-000005","order_id":"ord-2026-0317-008180","seq":5,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.389Z","level":"error","service":"inventory-api","version":"2.3.0","env":"staging","host":"node-b1","logger":"shop/inventory/reserve","msg":"insufficient stock for reservation","http":{"method":"POST","route":"/api/orders","status":409,"duration_ms":458.62},"request_id":"req-000006","order_id":"ord-2026-0317-009283","seq":6,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.430Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":341.77},"request_id":"req-000007","order_id":"ord-2026-0317-004803","seq":7,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.627Z","level":"info","service":"catalog","version":"0.0.1","env":"staging","host":"node-c1","logger":"shop/catalog/handler","msg":"request completed","http":{"method":"GET","route":"/api/catalog","status":200,"duration_ms":310.68},"request_id":"req-000008","order_id":"ord-2026-0317-003944","seq":8,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.794Z","level":"info","service":"catalog","version":"0.0.1","env":"staging","host":"node-c1","logger":"shop/catalog/handler","msg":"request completed","http":{"method":"GET","route":"/api/catalog","status":200,"duration_ms":68.98},"request_id":"req-000009","order_id":"ord-2026-0317-004626","seq":9,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:33.852Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":361.38},"request_id":"req-000010","order_id":"ord-2026-0317-008473","seq":10,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"},"dropped_since_last":28}
{"ts":"2026-03-17T09:14:33.926Z","level":"error","service":"payments-api","version":"3.1.4","env":"staging","host":"node-b2","logger":"shop.payments.authorize","msg":"upstream payment provider timed out","http":{"method":"POST","route":"/api/checkout","status":502,"duration_ms":446.22},"request_id":"req-000011","order_id":"ord-2026-0317-007757","seq":11,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.048Z","level":"info","service":"notifier","version":"0.4.11","env":"staging","host":"node-c1","logger":"@example/shop-notifier","msg":"notification dispatched","http":{"method":"GET","route":"/healthz","status":200,"duration_ms":164.05},"request_id":"req-000012","order_id":"ord-2026-0317-009264","seq":12,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.188Z","level":"error","service":"inventory-api","version":"2.3.0","env":"staging","host":"node-b1","logger":"shop/inventory/reserve","msg":"insufficient stock for reservation","http":{"method":"POST","route":"/api/orders","status":409,"duration_ms":34.64},"request_id":"req-000013","order_id":"ord-2026-0317-002543","seq":13,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.274Z","level":"info","service":"cart-api","version":"0.9.7","env":"staging","host":"node-a2","logger":"shop/cart/handler","msg":"request completed","http":{"method":"GET","route":"/api/cart","status":200,"duration_ms":369.05},"request_id":"req-000014","order_id":"ord-2026-0317-004446","seq":14,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.464Z","level":"info","service":"catalog","version":"0.0.1","env":"staging","host":"node-c1","logger":"shop/catalog/handler","msg":"request completed","http":{"method":"GET","route":"/api/catalog","status":200,"duration_ms":253.23},"request_id":"req-000015","order_id":"ord-2026-0317-001787","seq":15,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.519Z","level":"debug","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"recomputed cart totals","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":93.21},"request_id":"req-000016","order_id":"ord-2026-0317-001949","seq":16,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.702Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":424.48},"request_id":"req-000017","order_id":"ord-2026-0317-007385","seq":17,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.758Z","level":"warn","service":"inventory-api","version":"2.3.0","env":"staging","host":"node-b1","logger":"shop/inventory/reserve","msg":"reservation retried after conflict","http":{"method":"POST","route":"/api/orders","status":200,"duration_ms":411.53},"request_id":"req-000018","order_id":"ord-2026-0317-001712","seq":18,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.826Z","level":"info","service":"catalog","version":"0.0.1","env":"staging","host":"node-c1","logger":"shop/catalog/handler","msg":"request completed","http":{"method":"GET","route":"/api/catalog","status":200,"duration_ms":143.81},"request_id":"req-000019","order_id":"ord-2026-0317-009381","seq":19,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:34.904Z","level":"info","service":"cart-api","version":"0.9.7","env":"staging","host":"node-a2","logger":"shop/cart/handler","msg":"request completed","http":{"method":"GET","route":"/api/cart","status":200,"duration_ms":337.53},"request_id":"req-000020","order_id":"ord-2026-0317-003827","seq":20,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"},"dropped_since_last":7}
{"ts":"2026-03-17T09:14:34.963Z","level":"debug","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"recomputed cart totals","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":293.02},"request_id":"req-000021","order_id":"ord-2026-0317-007406","seq":21,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.051Z","level":"info","service":"cart-api","version":"0.9.7","env":"staging","host":"node-a2","logger":"shop/cart/handler","msg":"request completed","http":{"method":"GET","route":"/api/cart","status":200,"duration_ms":184.77},"request_id":"req-000022","order_id":"ord-2026-0317-003470","seq":22,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.122Z","level":"debug","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"recomputed cart totals","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":63.86},"request_id":"req-000023","order_id":"ord-2026-0317-003856","seq":23,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.247Z","level":"warn","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.PricingService","msg":"slow downstream call","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":70.58},"request_id":"req-000024","order_id":"ord-2026-0317-004385","seq":24,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.290Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":229.02},"request_id":"req-000025","order_id":"ord-2026-0317-007953","seq":25,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.445Z","level":"info","service":"notifier","version":"0.4.11","env":"staging","host":"node-c1","logger":"@example/shop-notifier","msg":"notification dispatched","http":{"method":"GET","route":"/healthz","status":200,"duration_ms":12.58},"request_id":"req-000026","order_id":"ord-2026-0317-001731","seq":26,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.616Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":226.98},"request_id":"req-000027","order_id":"ord-2026-0317-001990","seq":27,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.780Z","level":"info","service":"checkout-api","version":"1.14.2","env":"staging","host":"node-a1","logger":"com.example.shop.checkout.OrderController","msg":"request completed","http":{"method":"POST","route":"/api/checkout","status":201,"duration_ms":12.51},"request_id":"req-000028","order_id":"ord-2026-0317-001167","seq":28,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:35.961Z","level":"error","service":"payments-api","version":"3.1.4","env":"staging","host":"node-b2","logger":"shop.payments.authorize","msg":"upstream payment provider timed out","http":{"method":"POST","route":"/api/checkout","status":502,"duration_ms":141.78},"request_id":"req-000029","order_id":"ord-2026-0317-005255","seq":29,"sampling":{"decision":"keep","rate":0.25,"policy":"level>=error or 1-in-4"}}
{"ts":"2026-03-17T09:14:36.056Z","level":"info","service":"cart-api","version":"0.9.7","env":"staging","host":"node-a2","logger":"shop/cart/handler","msg":"request completed","http":{"method":"GET","route":"/api/cart","status":200,"duration_ms":439.45},"request_id":"req-000030","order_id":"ord-2026-0317-009886","seq":30,"sampling":{"decision":"drop","rate":0.25,"policy":"level>=error or 1-in-4"},"dropped_since_last":39}
Specifications
- Lines
- 30
- Kept Records
- 10
- Dropped Records
- 20
- Sampling Rate
- 0.25
- Drop Counter Records
- 3
- Policy
- keep every error plus 1 in 4
- Seed
- 7411
Testing contract
Expected to pass- Scenario
- Estimate the true event rate from the sampled stream.
- Expected result
- 10 kept records plus the three drop counters reconstruct the full population, and every error-level record is present because the policy keeps them unconditionally.
What is a .jsonl file?
JSONL (JSON Lines) is a text format where each line is a complete, independent JSON value, allowing records to be streamed and appended without parsing the whole file. It is not itself a JSON array and each line must stand alone. It is common in logging, machine learning datasets, and data pipelines.
How to use this file
Use an example JSONL to test line-by-line streaming parsers, append-and-resume ingestion, and batch pipelines that process one record per line.
How to use this file for testing
“Structured Log — Sampling Decisions and Drop Counters (jsonl)” is a deterministic Novus Examples fixture for Observability, Log parsing, Data engineering. Structured and plain-text telemetry with known timestamps, levels, request identifiers, and error states for testing log ingestion, correlation, dashboards, and alert pipelines.
Documented properties for this file: seed 7411. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Telemetry fixtures use fixed trace IDs, span IDs, and timestamps so ingestion is reproducible run to run. Point your collector, parser, or query layer at the file and assert the documented span tree, metric families, or severity mix; service and host names are invented.
Code examples
import json
with open("app-sampled-with-drop-counts.jsonl") as f:
rows = [json.loads(line) for line in f]
print(len(rows), rows[0])Related files
- log.NET Exception with an Inner Exception Chain (log)A .NET exception whose inner exception is introduced by the ---> marker and closed by End of inner exception stack trace, wrapped in pipe-delimited log lines. Error groupers that key on the outermost message alone merge two genuinely different faults here.
- logCRI Container Log — Partial Lines and Two Streams (log)The CRI log format Kubernetes nodes actually write: an RFC 3339 nanosecond timestamp, a stream name, and an F or P tag marking whether the line is full or a partial continuation. One long message is split across several P lines that must be rejoined before parsing.
- logGo Panic with Full Goroutine Dump (log)A Go panic followed by the runtime's full goroutine dump — four goroutines with their wait states, two-line frames and a created by attribution, separated by blank lines. Blank-line separation defeats multi-line rules that treat an empty line as the end of an event.
- loglogfmt — Quoting, Escaping and Bare-Key Edge Cases (log)logfmt lines built to break naive splitting on spaces and equals signs: a value containing an equals sign, escaped quotes, a trailing backslash before the closing quote, a bare key with no value, duplicate keys, non-ASCII values and one unterminated quote followed by a clean line.
- jsonOTLP Logs — Export Document with Severity and Trace Correlation (json)An OTLP logs export across two resources: five records spanning DEBUG to FATAL with both severityNumber and severityText, separate event and observed timestamps, trace and span correlation, and one record whose body is a full multi-line Python traceback.
- jsonlOTLP Logs — NDJSON Stream, One Record per Line (jsonl)The same five OTLP log records reframed as one self-contained export per line, with the resource attributes repeated on every line. The trade a line-oriented sink makes: each line stands alone, at the cost of repeating the resource block five times.
Generated by generation/observability.py. Free for any use, no attribution required — license.