Estimating the Minimum Sample Size for Neural Network Model Fitting—A Monte Carlo Simulation Study
Bibliographic record
Abstract
In the era of machine learning, many psychological studies use machine learning methods. Specifically, neural networks, a set of machine learning methods that exhibit exceptional performance in various tasks, have been used on psychometric datasets for supervised model fitting. From the computer scientist's perspective, psychometric independent variables are typically ordinal and low-dimensional-characteristics that can significantly impact model performance. To our knowledge, there is no guidance about the sample planning suggestion for this task. Therefore, we conducted a simulation study to test the performance of an NN with different sample sizes and the simulation of both linear and nonlinear relationships. We proposed the minimum sample size for the neural network model fitting with two criteria: the performance of 95% of the models is close to the theoretical maximum, and 80% of the models can outperform the linear model. The findings of this simulation study show that the performance of neural networks can be unstable with ordinal variables as independent variables, and we suggested that neural networks should not be used on ordinal independent variables with at least common nonlinear relationships in psychology. Further suggestions and research directions are also provided.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".