Quality scoring
The checks every record passes before delivery, and how results are reported.
The checks
Every record carries the result of each check in its manifest checks object, as pass or fail. A record that fails a blocking check isn't delivered.
| Check | What it tests | Blocking |
|---|---|---|
pii_scan | No personal data left in audio, transcript, headers, pixels or report | Yes |
burned_in_text | No text burned into image pixels (imaging only) | Yes |
dedup_web | The item doesn't duplicate content already on the public web | Yes |
dedup_internal | No duplicate of another record in the dataset | Yes |
benchmark_contamination | No overlap with public benchmarks | Yes |
quality_score | The item meets the dataset's quality bar | Per spec |
The quality score
The quality bar is set with your research team when a dataset is scoped: for audio, things like intelligibility, channel separation and transcript alignment; for imaging, completeness of series, report presence and acquisition consistency. The score definition and threshold for each dataset are written into its datasheet, so you can see exactly what "pass" meant.
Dataset-level results
The datasheet summarises results across the dataset: how many records were checked, how many passed each check, and what was excluded and why.