Forecasting Debt Crises with Regression: Testing the Performance of Simple Models

This project provides a supplemental test to our pre-registered machine learning predictions (https://osf.io/mvwjy/?view_only=7bba3fe5e3164d80a9b338bd83583fcc) to evaluate whether logistic regression models can effectively predict sovereign debt crises. Building on recent theoretical work demonstrating that social science data has low intrinsic dimension (Morucci & Spirling, 2024), and our own machine learning analysis that identified key predictors of debt crises, we develop four regression models to forecast crisis onset. We aim to contribute to the ongoing debate about model complexity in prediction tasks. If simple OLS models perform similarly to random forests, this would support the argument that the curated nature of social science data limits the returns to model complexity. We will estimate the probability of debt crisis onset within a three-year window (2022-2024) using four logistic regression models with country fixed effects. Our predictions indicate the probability that a country will experience at least one debt crisis onset during 2022, 2023, or 2024. Following Chakrabarti and Zeaiter (2014), we define crisis onset as: (1) new default episode (minimum $1 million), (2) 30%+ increase in defaulted debt, or (3) IMF financing above 200% of annual quota. Each model is specified as: Pr(Crisis_it+1,t+2,t+3 = 1) = Λ(α_i + β'X_it) where α_i are country fixed effects, X_it are the predictor variables measured in 2021, and Λ is the logistic function. All predictor variables are measured as of 1960-2021. Data sources include World Bank, IMF, V-Dem, and authors' calculations as described in the original machine learning models (https://osf.io/mvwjy/?view_only=7bba3fe5e3164d80a9b338bd83583fcc). Models were estimated using the glm function in R with family=binomial(link="logit"). Model 1 - Core Economic Fundamentals: GDP per capita (ny_gdp_pcap_kd) Total reserves as % of external debt (fi_res_totl_dt_zs) Short-term external debt as % of total (stdebtall) External debt to reserves ratio (xtdebtres) Central government debt (cg) Crisis history variables (dyrs, d2, d3) Model 2 - Political Institutions: Proportion of years in default (ccrisis) Rule of law index (v2x_rule) Grants and other revenue (gc_rev_gotr_zs) Electoral democracy index (v2x_polyarchy) Crisis history variables (dyrs, d2, d3) Model 3 - Comprehensive Model: Top economic predictors (ny_gdp_pcap_kd, fi_res_totl_dt_zs, stdebtall) Top political predictors (ccrisis, v2x_rule, gc_rev_gotr_zs) Regional indicators (regionAfrica, regionEAsia) Crisis history variables (dyrs, d2, d3) Model 4 - Parsimonious Benchmark: GDP per capita (ny_gdp_pcap_kd) Years since last crisis (dyrs) Proportion of years in default (ccrisis) We hypothesize that these models will achieve predictive performance within 5% of our best random forest model. Our predictions from each model are found here: https://www.dropbox.com/scl/fi/scqyny7t4a2xpoafvln3w/country_predictions_2022_2024_fixed_effects.csv?rlkey=4vwy3baq9vlr2ch80qpwngus0&dl=0. The CSV file contains: country code from Correlates of War and predicted probabilities (0-1) for each of the four models. Predictions were generated on 20 June 2025 using R version 4.3.0. We will evaluate each model's prediction performance using: 1. ROC/AUC Analysis: Each model's AUC will be compared against: - Our benchmark confidence interval: 0.79 ± 1/√179 = [0.72, 0.86] - Our best random forest model (AUC = 0.793) - Each other to assess relative performance 2. Hosmer-Lemeshow Goodness-of-Fit Test: - Divide 176 countries into 10 bins (excluding Zambia, Ghana, Sri Lanka) - Compare observed vs. expected crises per bin - Use Pearson χ² statistic to assess calibration 3. Direct Prediction Comparison: - Compare country-level predictions between regression and random forest models - Identify countries with the largest prediction differences Analysis: All countries in our sample (179 total, excluding micro-states and North Korea) 179 countries (176 for evaluation, excluding 3 countries with observed 2022 crises noted in first preregistration: Sri Lanka, Zambia, Ethiopia). Data on debt crises will become available in August 2025 from the Bank of Canada. We commit to evaluating these predictions and posting results, regardless of outcomes.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.