Forecasting Debt Crises with Regression: Testing the Performance of Simple Models
Bibliographic record
Abstract
This project provides a supplemental test to our pre-registered machine learning predictions (https://osf.io/mvwjy/?view_only=7bba3fe5e3164d80a9b338bd83583fcc) to evaluate whether logistic regression models can effectively predict sovereign debt crises. Building on recent theoretical work demonstrating that social science data has low intrinsic dimension (Morucci & Spirling, 2024), and our own machine learning analysis that identified key predictors of debt crises, we develop four regression models to forecast crisis onset. We aim to contribute to the ongoing debate about model complexity in prediction tasks. If simple OLS models perform similarly to random forests, this would support the argument that the curated nature of social science data limits the returns to model complexity. We will estimate the probability of debt crisis onset within a three-year window (2022-2024) using four logistic regression models with country fixed effects. Our predictions indicate the probability that a country will experience at least one debt crisis onset during 2022, 2023, or 2024. Following Chakrabarti and Zeaiter (2014), we define crisis onset as: (1) new default episode (minimum $1 million), (2) 30%+ increase in defaulted debt, or (3) IMF financing above 200% of annual quota. Each model is specified as: Pr(Crisis_it+1,t+2,t+3 = 1) = Λ(α_i + β'X_it) where α_i are country fixed effects, X_it are the predictor variables measured in 2021, and Λ is the logistic function. All predictor variables are measured as of 1960-2021. Data sources include World Bank, IMF, V-Dem, and authors' calculations as described in the original machine learning models (https://osf.io/mvwjy/?view_only=7bba3fe5e3164d80a9b338bd83583fcc). Models were estimated using the glm function in R with family=binomial(link="logit"). Model 1 - Core Economic Fundamentals: GDP per capita (ny_gdp_pcap_kd) Total reserves as % of external debt (fi_res_totl_dt_zs) Short-term external debt as % of total (stdebtall) External debt to reserves ratio (xtdebtres) Central government debt (cg) Crisis history variables (dyrs, d2, d3) Model 2 - Political Institutions: Proportion of years in default (ccrisis) Rule of law index (v2x_rule) Grants and other revenue (gc_rev_gotr_zs) Electoral democracy index (v2x_polyarchy) Crisis history variables (dyrs, d2, d3) Model 3 - Comprehensive Model: Top economic predictors (ny_gdp_pcap_kd, fi_res_totl_dt_zs, stdebtall) Top political predictors (ccrisis, v2x_rule, gc_rev_gotr_zs) Regional indicators (regionAfrica, regionEAsia) Crisis history variables (dyrs, d2, d3) Model 4 - Parsimonious Benchmark: GDP per capita (ny_gdp_pcap_kd) Years since last crisis (dyrs) Proportion of years in default (ccrisis) We hypothesize that these models will achieve predictive performance within 5% of our best random forest model. Our predictions from each model are found here: https://www.dropbox.com/scl/fi/scqyny7t4a2xpoafvln3w/country_predictions_2022_2024_fixed_effects.csv?rlkey=4vwy3baq9vlr2ch80qpwngus0&dl=0. The CSV file contains: country code from Correlates of War and predicted probabilities (0-1) for each of the four models. Predictions were generated on 20 June 2025 using R version 4.3.0. We will evaluate each model's prediction performance using: 1. ROC/AUC Analysis: Each model's AUC will be compared against: - Our benchmark confidence interval: 0.79 ± 1/√179 = [0.72, 0.86] - Our best random forest model (AUC = 0.793) - Each other to assess relative performance 2. Hosmer-Lemeshow Goodness-of-Fit Test: - Divide 176 countries into 10 bins (excluding Zambia, Ghana, Sri Lanka) - Compare observed vs. expected crises per bin - Use Pearson χ² statistic to assess calibration 3. Direct Prediction Comparison: - Compare country-level predictions between regression and random forest models - Identify countries with the largest prediction differences Analysis: All countries in our sample (179 total, excluding micro-states and North Korea) 179 countries (176 for evaluation, excluding 3 countries with observed 2022 crises noted in first preregistration: Sri Lanka, Zambia, Ethiopia). Data on debt crises will become available in August 2025 from the Bank of Canada. We commit to evaluating these predictions and posting results, regardless of outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.022 | 0.061 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".