Optimal Subsampling for Functional Quasi-Mode Regression with Big Data
Bibliographic record
Abstract
We propose investigating optimal subsampling for functional regression with massive datasets based on the mode value, which is referred to as functional quasi-mode regression, to reduce data volume and alleviate computational burden. Using data-adaptive weights derived from regression residuals, the suggested regression offers enhanced robustness against nonnormal errors compared to traditional least squares or maximum likelihood estimation methods. To estimate the model, we employ B-spline basis functions to approximate the functional coefficient and include a penalty term in the objective function for enforcing smoothness in the resulting estimator. We adopt a computationally efficient mode-expectation-maximization algorithm, augmented by a Gaussian kernel, for numerical estimation. Under mild regularity conditions, we derive the asymptotic distributions of both full data and subsample quasi-mode estimators. The optimal subsampling probabilities by minimizing the asymptotic variance-covariance matrix under A- and L-optimality criteria are identified. These optimal probabilities rely on the full data estimate, prompting the development of a two-step algorithm to approximate the optimal subsampling procedure. The resultant algorithm is processing-efficient and can significantly reduce computational time compared to the full data approach. We also establish the asymptotic normality of the quasi-mode estimator obtained through this two-step algorithm. To assess finite sample performance, we conduct Monte Carlo simulations and analyze air quality data, showcasing the effectiveness of the developed estimator. Supplemental materials for this article are available online.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.022 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".