{"id":6672,"date":"2026-09-18T21:17:04","date_gmt":"2026-09-18T20:17:04","guid":{"rendered":"https:\/\/idiolect.org.uk\/notes\/?p=6672"},"modified":"2026-09-18T21:17:04","modified_gmt":"2026-09-18T20:17:04","slug":"discontinuity-tells","status":"publish","type":"post","link":"https:\/\/idiolect.org.uk\/notes\/2026\/09\/18\/discontinuity-tells\/","title":{"rendered":"Discontinuity tells"},"content":{"rendered":"\n

A particular pattern in the numbers, a misbehaviour which shows that the scale is more than just a record:<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

Source: Reddit Distribution of results of the Matura (high school exit exam) in Poland in 2013. The minimum score to pass is 30%.<\/a> Commentary: Unusual distributions of scores on final high-school exams in Poland<\/a>.<\/p>\n\n\n\n

But let’s be clear \u2014 this is not about Poland. The x-axis pretends to innocence, just a passive measure of how the data fall. This pattern, though, the cratering before the threshold and then the pile-up just after, show that the threshold itself has a distorting effect.<\/p>\n\n\n\n

Here’s data on the New York high school exit exam<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

Source: Dee, T. S., Dobbie, W., Jacob, B. A., & Rockoff, J. (2019). The causes and consequences of test score manipulation: Evidence from the New York regents examinations. American Economic Journal: Applied Economics<\/em>, 11<\/em>(3), 382-423. (NBER working paper 22165<\/a>).<\/p>\n\n\n\n

There is some discretion in the exam marking. The grade matters to what happens to the students (or the schools and teachers which produce the students), so individual data points near the threshold “sort themselves” over the line.<\/p>\n\n\n\n

Here’s a non-exam example, from scoring eligibility for a social programme in Colombia. The threshold to receive benefits was 47:<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

Source: Camacho, A., & Conover, E. (2011). Manipulation of Social Program Eligibility<\/a>. American Economic Journal: Economic Policy<\/em>, 3<\/em>(2), 41\u201365. DOI: 10.1257\/pol.3.2.41<\/p>\n\n\n\n

Interesting because it is an example of the whole population data having a particular “tell”: there’s a story around how points fall near these thresholds.<\/p>\n\n\n\n

Also, any regression discontinuity design which assumes random assignment is going to be confounded by such near-threshold sorting. The check is to look at the count density around the threshold (and cross your fingers for no cratering\/piling up).<\/p>\n","protected":false},"excerpt":{"rendered":"

A particular pattern in the numbers, a misbehaviour which shows that the scale is more than just a record: Source: Reddit Distribution of results of the Matura (high school exit exam) in Poland in 2013. The minimum score to pass is 30%. Commentary: Unusual distributions of scores on final high-school exams in Poland. But let’s […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"federated","footnotes":""},"categories":[9],"tags":[],"class_list":["post-6672","post","type-post","status-publish","format-standard","hentry","category-science"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/posts\/6672","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/comments?post=6672"}],"version-history":[{"count":1,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/posts\/6672\/revisions"}],"predecessor-version":[{"id":6676,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/posts\/6672\/revisions\/6676"}],"wp:attachment":[{"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/media?parent=6672"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/categories?post=6672"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/idiolect.org.uk\/notes\/wp-json\/wp\/v2\/tags?post=6672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}