Data quality report

What is actually true about this dataset, as of the last nightly run. The headline is the first table: every figure the dashboard publishes, recomputed here in SQL from the same artifacts by a different route. Nine of those are row counts and seven compare the output of a rule that had to be reimplemented, number for number. Everything below that is either a defect worth knowing about or a characteristic that surprises people and is nonetheless correct.

Do the published numbers hold up?

build_aggregates.py counts in pandas and writes coverage_summary.json, which the dashboard prints. These models recount the same things in SQL from the same files. Where the two agree, a published figure is corroborated by an independent implementation rather than merely asserted. The value checks matter most: a count check passes even if the heatmap put every incident in the wrong hour, because moving weight between hours conserves the total.

When does Oak Park report crime?

All 168 hour-by-weekday cells. An incident reported as a range contributes a fraction to each hour it covers rather than a whole incident to the first hour, which is why the cells hold decimals. The 168 cells sum to exactly the number of timed incidents, and a test enforces that.

Weather moves reported crime more than anything else here

Mean incidents per day inside each cohort, against the all-days baseline of 6.01 incidents per day. Warm bars run above the baseline, cool bars below. These cohorts overlap: a day can be freezing and a weekend at once, so the rows share days and must never be added together. Each is only comparable to the baseline.

Darkness explains less than people expect

Across all incidents the mean dark fraction is 0.366 against a baseline of 0.439: the average incident falls in civil darkness about 7 points less than its own day's night length would predict. Below, the offenses that bend furthest either way. Positive means the offense happens in darkness more than the season alone explains. Read it with the caveat that a wide reported window spreads its weight uniformly, so offenses usually discovered later rather than witnessed live drift darker than they were.

An exact timestamp is not an exact time

The open-data stream carries a precise occur_datetime, which invites reading it as a real instant. Times pile up on round values far past anything a real distribution would produce. Exactly midnight carries 55 times the uniform per-minute rate and exactly noon 50 times. Those two are the signature of a date-only record or a form defaulting to 12:00 AM, not of 535 incidents happening at the same second. Across 45 clustered values, 5,707 incidents sit on a value carrying at least three times the uniform rate, and 4,293 of those sit exactly on the hour: 31% of every timestamped incident. Read any hour-of-day figure with that in mind.

Where the two ingest streams came from

Two independent sources feed this dataset upstream, and they overlap only partly. The dashboard-only share is the reason the second stream exists, and the reason a third of incidents have no narrative text for topics or keyword flags to read.

Defects

Things that are wrong and could in principle be fixed. Classes overlap, so the counts do not sum: the flagged-rows tile above counts distinct incidents rather than adding these rows. Note the grain column, because not every defect is counted in incidents.

Keyword flag prevalence

How often each gazetteer flag fires, over the 15,508 narratives that exist to be scanned. The terms behind each flag stay private to the pipeline; only the names and the counts are here. 3 flags never fire at all, which means the dashboard offers filters that return nothing: outcome_flags_refused_charges, firearm_terms_firearm_recovery, items_stolen_firearm_taken.

Characteristics

The numbers people ask about. None of these is a defect. They are here because every one of them has been mistaken for one at least once.

Test outcomes

dbt docs renders test definitions but never their results, so this is the half the docs site cannot show. Warnings are deliberate and documented in the test file itself; a failure here would be a genuine regression.