Two-stage filter pruning incorporating cosinespatial correlation
Bibliographic record
Abstract
目的深度神经网络在图形图像、计算机视觉等众多应用领域取得了令人瞩目的效果，但是一直以来深度学习网络模型由于其庞大的计算量以及存储资源而无法部署在资源受限的嵌入式设备端。为了解决模型所需的计算资源和嵌入式设备资源受限之间的矛盾，提出了一种引入余弦空间相关的两阶段滤波器剪枝方法，旨在利用滤波器间的空间相关性实现更优的剪枝方式。方法在预剪枝阶段引入L范数记录下范数值最高的滤波器，本文称为关键滤波器；在剪枝阶段引入余弦距离保留和关键滤波器空间相关性高的滤波器。结果本文提出的剪枝方法在CIFAR（Canadian Institute for Advanced Research）数据集上取得了优于其他对比方法的效果，在CIFAR10数据集上将VGG（Visual Geometry Group）16的参数量和浮点运算量分别压缩了72.9%和73.5%，同时模型精度提升了0.1%。对于高效的残差网络ResNet（residual neural network）56和深度可分离网络MobileNet V1也可以有效地压缩，该方法在CIFAR100数据集上对ResNet56网络在更高的压缩率下实现了更小的精度损失（精度提升0.48%）。对于MobileNet V1网络，压缩了46.89%的参数量和46.23%的浮点运算量，而模型精度提升了0.11%。结论引入余弦空间相关性的两阶段滤波器剪枝策略避免了网络剪枝中“衡量指标小，则衡量对象不重要”和“相似即冗余”两种假设不成立而导致模型陷入次优结果，从滤波器空间的角度挖掘相关性，在保证模型准确率的前提下能够压缩更多的参数量和浮点运算量。
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.008 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.009 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".