Optimum Regularization Parameter C in Support Vector Machine (SVM) Binary Classification
Bibliographic record
Abstract
Support Vector Machines (SVMs) are widely used learning algorithms for data classification. In machine learning algorithms, such as the SVM approach, one or more parameters control the smoothness of the solution and are required to be tuned for the optimum solution. Such parameters are called regularization parameters, which are critical in building robust and accurate algorithms to prevent overfitting and underfitting. In SVM, the regularization parameter, denoted by C, regularizes the training loss of misclassified data. Traditionally, the value of C is first set to one, and if after training the data, misclassifications are observed, C is tuned by K-fold Cross-Validation (CV) method, which is a time- consuming process. This thesis aims to rigorously analyze and study the behavior of the C value in SVM. The analysis shows that for the case of a linearly separable dataset, setting the value of C to one does not always provide the optimum solution, and in addition, it is shown that there exists a Minimum Acceptance Value (MAV) for C as a function of Separability and Scatteredness (S&S). S&S is a new notion that is defined in this thesis, inspired by the Signal-to-Noise ratio (SNR) definition and is shown to be a critical parameter in the analysis of SVM classifiers. The study is further extended for the case of linearly non-separable dataset, and it has shown that a lookup table based on the analysis of bias-variance tradeoff (BVB C-Table) provides the optimum value of C, which not only outperforms but also is much faster than, the existing k-fold CV. For example, in a simple binary classification scenario, a typical k-fold cross-validation can take more than two hours, whereas the proposed method requires only a couple of minutes in a python-based environment. Due to its efficiency, the proposed method of choosing the regularization parameter enables online binary classification and will have potential benefits in One-vs-All and One-vs-One SVM classification.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.019 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".