What quantitative accuracy improvements did the introduction of S&E layers lead to?
- User
What quantitative accuracy improvements did the introduction of S&E layers lead to?
- Assistant
The introduction of Squeeze-and-Excitation (SE) layers led to significant quantitative improvements in image classification accuracy across several popular convolutional neural network (CNN) architectures. These improvements were demonstrated in the original paper:
"Squeeze-and-Excitation Networks" by Jie Hu, Li Shen, and Gang Sun (CVPR 2018).
Below are some key quantitative results from the paper, primarily based on the ImageNet-1K classification benchmark:
1. SE-ResNet vs. ResNet
Model Top-1 Error (%) Top-5 Error (%) ResNet-50 23.85 7.13 SE-ResNet-50 22.28 6.03 ResNet-101 22.63 6.44 SE-ResNet-101 21.71 5.78 ResNet-152 21.69 5.94 SE-ResNet-152 21.43 5.73 Improvement:
- Up to 1.6% absolute improvement in Top-1 accuracy for ResNet-50.
- Consistent improvements across deeper variants.
2. SE-Inception vs. Inception
Model Top-1 Error (%) Top-5 Error (%) Inception-ResNet-v2 19.86 4.90 SE-Inception-ResNet-v2 18.68 4.31 Improvement:
- Over 1.1% absolute gain in Top-1 accuracy.
3. SE-Net Performance on ImageNet
Model Top-1 Error (%) Top-5 Error (%) Parameters (M) SENet-154 18.68 4.47 115.1 Inception-ResNet-v2 19.86 4.90 55.8 ResNeXt-101 (64x4d) 20.7 5.3 83.6 SENet-154 achieved state-of-the-art performance on ImageNet at the time of publication.
4. ILSVRC 2017 Classification Challenge
- SENet won 1st place in the ILSVRC 2017 classification task.
- Achieved a Top-5 error of 2.251%, which was the best result at the time.
Summary of Accuracy Gains
- Top-1 accuracy improvements of 1–2% across various architectures.
- Top-5 accuracy improvements of 0.5–1%.
- These gains were achieved with minimal additional computational cost and parameter overhead.
Conclusion
The introduction of SE layers provided a simple yet powerful mechanism to improve CNN performance by modeling channel-wise dependencies. The quantitative improvements demonstrated that SE blocks could significantly enhance the accuracy of existing architectures with only a modest increase in complexity.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.