Evaluation & metrics fixtures

Survey Responses — Likert + NPS (CSV, 60 rows)

A customer-survey response set — 60 respondents rating five statements on a 1–5 Likert scale plus a 0–10 NPS score and a segment. A fixture for testing survey analysis, aggregation, and NPS/CSAT calculations.

File: CSV · Surveys · 60 rows
Use case: Data importConversion testing+1

Recommender Ratings — user/item/rating (CSV, 240 rows)

A MovieLens-style ratings log — 240 user/item/rating events with Unix timestamps across 50 users and 40 items. A fixture for testing recommender pipelines, collaborative-filtering loaders, and sparse-matrix builders.

File: CSV · Recommender · 240 rows
Use case: Data importModel evaluation+1

Model Benchmark Results (JSON)

A model-evaluation summary in JSON — per-task scores for a fictional model across sentiment, NER, summarization, translation, and QA, each with its metric and sample size. A fixture for testing eval dashboards and leaderboard importers.

File: JSON · Eval
Use case: Model evaluationJSON parsing+1

Confusion Matrix — 3-class (CSV)

A 3-class confusion matrix as CSV — rows are the true class, columns the predicted class, cells the counts. Paired with a JSON twin for testing metric parsers and evaluation visualisers.

File: CSV · Eval
Use case: Model evaluationCSV parsing+1· Paired fixture

Confusion Matrix — 3-class (JSON)

The same 3-class confusion matrix as JSON — a labels array plus a nested counts matrix. The structured twin of the CSV, for testing evaluation tooling.

File: JSON · Eval
Use case: Model evaluationJSON parsing+1· Paired fixture

ROC Curve Points (CSV)

An ROC curve as CSV — decision threshold with the corresponding false-positive and true-positive rates, monotonic from (0,0) to (1,1). A fixture for testing chart tools and AUC calculators.

File: CSV · Eval
Use case: Model evaluationCSV parsing+1

Classification Report (JSON)

A per-class classification report in the scikit-learn structure — precision, recall, F1, and support for each class plus accuracy and macro/weighted averages. A fixture for testing metric parsers and report renderers.

File: JSON · Eval
Use case: Model evaluationJSON parsing

Detection Eval Metrics (JSON)

Synthetic mAP evaluation summary for object-detection benchmark harness tests.

File: JSON · Eval
Use case: Model evaluationJSON parsing

BM25 Retrieval Scores (JSON)

BM25 score snapshot for comparing neural rerankers against a lexical baseline.

File: JSON · Ranking
Use case: Model evaluationJSON parsing

MRR Eval Ground Truth (JSON)

Mean reciprocal rank ground truth for search eval harness tests.

File: JSON · Ranking
Use case: Model evaluation

nDCG Ground Truth (JSON)

Precomputed DCG/nDCG values for ranking metric unit tests.

File: JSON · Ranking
Use case: Model evaluation

Cosine Similarity Pairs (JSONL)

Precomputed cosine similarity pairs for embedding eval scripts.

File: JSONL · Embeddings · 6 records
Use case: Model evaluation

Confusion Matrix — 3 Classes (CSV)

3×3 confusion matrix CSV for classification metric calculators.

File: CSV · Eval
Use case: Model evaluationCSV parsing· Conversion set

Confusion Matrix — 3 Classes (JSON)

JSON twin of the 3-class confusion matrix.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

Confusion Matrix — 5 Classes (CSV)

5×5 confusion matrix with off-diagonal noise for multi-class eval.

File: CSV · Eval
Use case: Model evaluation· Conversion set

Confusion Matrix — 5 Classes (JSON)

JSON twin of the 5-class confusion matrix.

File: JSON · Eval
Use case: Model evaluation· Conversion set

Per-class Metrics (JSON)

Per-class precision/recall/F1 snapshot for classification dashboards.

File: JSON · Eval
Use case: Model evaluation

sklearn Classification Report (TXT)

sklearn-style classification report text for parser snapshot tests.

File: TXT · Eval
Use case: Model evaluation

ROC Curve Points (JSON)

ROC curve coordinate list for AUC calculator tests.

File: JSON · Eval
Use case: Model evaluation

Precision-Recall Curve Points (JSON)

PR curve coordinates for average-precision metric tests.

File: JSON · Eval
Use case: Model evaluation

Caption Eval References (JSON)

Multiple reference captions per image for BLEU/CIDEr caption eval.

File: JSON · Vision
Use case: Model evaluation

Ranking Eval Queries (JSONL)

Short query list for ranking benchmark harness smoke tests.

File: JSONL · Eval · 6 records
Use case: Model evaluationJSON parsing

Embedding Similarity Matrix (JSON)

Tiny 2×2 cosine similarity matrix for embedding eval unit tests.

File: JSON · Embeddings
Use case: Model evaluation

Eval Metric — Accuracy Mini

Minimal SAMPLE eval metric JSON (accuracy) for dashboard parsers.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

Eval Metric — F1 Mini

Minimal SAMPLE eval metric JSON (f1) for dashboard parsers.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

Eval Metric — Latency P95

Minimal SAMPLE eval metric JSON (latency_ms) for dashboard parsers.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

Eval Metric — Toxicity Score

Minimal SAMPLE eval metric JSON (toxicity) for dashboard parsers.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

Eval Metric — Bleu Mini

Minimal SAMPLE eval metric JSON (bleu) for dashboard parsers.

File: JSON · Eval
Use case: Model evaluationJSON parsing· Conversion set

ONNX Inference Contract - Static Identity Model

A one-node identity graph with a fixed 1x4 float tensor, useful as the smallest loader and inference smoke test. The matching JSON input and expected output make the model immediately usable in a CI inference assertion.

File: ONNX · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Static Identity Input

The named float32 input tensor payload for the Static Identity model, with explicit shapes and JSON values for runner-independent test setup.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Static Identity Expected Output

The reference-evaluator output for the Static Identity model and supplied input, including shape, dtype, exact values and a 1e-6 tolerance.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Affine MatMul plus Add Model

A two-node affine transform with fixed weights and bias, exercising initializers, MatMul dimensions and Add broadcasting. The matching JSON input and expected output make the model immediately usable in a CI inference assertion.

File: ONNX · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Affine MatMul plus Add Input

The named float32 input tensor payload for the Affine MatMul plus Add model, with explicit shapes and JSON values for runner-independent test setup.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Affine MatMul plus Add Expected Output

The reference-evaluator output for the Affine MatMul plus Add model and supplied input, including shape, dtype, exact values and a 1e-6 tolerance.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Two-Input Broadcast Add Model

A two-input Add graph where a length-two bias broadcasts across a 2x2 matrix, checking input binding and broadcasting rules. The matching JSON input and expected output make the model immediately usable in a CI inference assertion.

File: ONNX · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Two-Input Broadcast Add Input

The named float32 input tensor payload for the Two-Input Broadcast Add model, with explicit shapes and JSON values for runner-independent test setup.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Two-Input Broadcast Add Expected Output

The reference-evaluator output for the Two-Input Broadcast Add model and supplied input, including shape, dtype, exact values and a 1e-6 tolerance.

File: JSON · Model Inference
Use case: Model inference testingModel evaluation+1· Conversion set

ONNX Inference Contract - Dynamic-Batch ReduceSum Model

A ReduceSum graph with a symbolic batch dimension and fixed feature width, for dynamic-shape import and batch-output tests. The matching JSON input and expected output make the model immediately usable in a CI inference assertion.