Bibliographic record
Abstract
The Washington Post, one of the main newspapers in Washington D.C., cites in Reference 1 the case of a decision made based on a computer system error that caused 80,000 people to stop receiving their monthly benefits.It all started in 1996 when a federal law was promulgated to prevent justice fugitives from receiving any benefit from the government.In order to implement the measure, information from several government databases was compiled and, based on that, several thousand people had their benefits cancelled.The measure helped to capture several justice fugitives, but it also affected several people who had no criminal records.The article cites the case of Rosa Martinez, a resident from Redwood City California, whose monthly disability check was cancelled because the system had confused her records with those of another Rosa Martinez from Miami Florida.Apart from the hardships caused to people affected by the mistakes, the government faced several lawsuits that resulted in a considerable amount of vindications. System CharacteristicsThe system described in the case study has typical components of a decision support system: data sources and a series of rules to analyze the data and recommend actions based on it.In this case, the quality of the data and the quality of the implemented algorithms to analyze it were not good enough to prevent problems for the system's owner.176 Appendix Cost of Missing Quality: Case Studies Missing Quality• Functionality.The system was not able to correctly perform its task.The data used in the system did not contain enough information to identify in a conclusive way every person in the databases.Due of this reason, when information from several data sources was aggregated, the implementers very likely had to resort to manual matching or heuristic algorithms.• Testability.Even though the newspaper article does not mention this information, the outcome suggests that the system designers did not have enough elements to be able to correctly match the data from different data sources and be able to test different scenarios.Its lack of testability is very likely the cause of several of the errors when government policies changed.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.043 | 0.165 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.005 | 0.010 |
| Science and technology studies | 0.008 | 0.003 |
| Scholarly communication | 0.004 | 0.005 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.007 | 0.004 |
| Insufficient payload (model declined to judge) | 0.006 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".