Low Carbon Footprint Training for 1D-CNNs with Temporal Max-Pooling
Bibliographic record
Abstract
Training convolutional neural networks (CNNs) demands huge GPU memory consumption and training time, leading to increased carbon emissions, and impacting sustainability. In this paper, we propose HotConv, a low GPU memory and low carbon footprint learning strategy for training the class of 1D CNNs that have a temporal max-pooling layer. Such CNNs are widely used in various domains for learning large-sized inputs, including genomics and malware detection. HotConv reduces the GPU memory usage of such CNNs by harnessing the sparsity of relevant activations and gradients at the temporal max-pooling layer, which produces the same model as the full computation of activations and gradients, without trading-off model performance. Evaluations using the public benchmark BODMAS and VirusTotal datasets for malware detection with HotConv applied to the public MalConv network architecture show that the carbon footprint reduction using HotConv is superior to existing approaches. For instance, HotConv uses only 1/22 of the GPU memory used by MalConv2 - the memory-efficient variant of MalConv, while also consuming less training time than MalConv2. This is equivalent to reducing the carbon footprint up to 1/4 of that of MalConv2 without compromising performance.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".