论文信息

  • 标题: Deep Residual Learning for Image Recognition
  • 作者: Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research)
  • 发表年份: 2015
  • arXiv链接: https://arxiv.org/abs/1512.03385

核心贡献

ResNet提出了残差学习框架，通过引入跳跃连接(skip connections)解决了深层神经网络的退化问题，使得训练数百层甚至上千层的网络成为可能。

主要内容

1. 退化问题

研究发现，随着网络深度增加，准确率会先上升后下降（退化），这不是过拟合导致的，而是优化困难。

2. 残差学习

核心思想是让网络学习残差映射 F(x) = H(x) - x，而不是直接学习目标映射 H(x)：

输出 = F(x) + x  (通过跳跃连接实现)

3. 网络架构

  • 基本残差块: 两个3×3卷积层 + 跳跃连接
  • 瓶颈残差块: 1×1 → 3×3 → 1×1卷积，用于更深网络
  • 网络变体: ResNet-18, 34, 50, 101, 152

4. 实验结果

  • 在ImageNet上，152层ResNet取得3.57%的top-5错误率
  • 赢得ILSVRC 2015分类任务冠军
  • 证明了更深的网络确实能带来更好的性能

技术细节

恒等映射

当输入输出维度相同时，直接使用恒等跳跃连接；维度不同时，使用1×1卷积进行投影。

批归一化

每个卷积层后使用Batch Normalization，加速训练并提供正则化效果。

影响与应用

ResNet是深度学习历史上最具影响力的架构之一：

  1. 计算机视觉: 成为图像分类、目标检测、语义分割的标准骨干网络
  2. 迁移学习: 预训练ResNet广泛用于各种下游任务
  3. 架构设计: 残差连接思想被广泛应用于Transformer等后续架构
  4. 引用量: 超过20万次引用，是AI领域引用最高的论文之一

总结

ResNet通过简单而优雅的残差连接设计，突破了深度网络训练的瓶颈，开启了”深度学习越深越好”的新时代，对整个AI领域产生了深远影响。

Paper Information

  • Title: Deep Residual Learning for Image Recognition
  • Authors: Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research)
  • Year: 2015
  • arXiv: https://arxiv.org/abs/1512.03385

Core Contribution

ResNet introduced the residual learning framework with skip connections, solving the degradation problem in deep neural networks and enabling training of networks with hundreds or thousands of layers.

Key Innovations

1. Degradation Problem

Deeper networks showed higher training error - not due to overfitting but optimization difficulty.

2. Residual Learning

Instead of learning H(x) directly, learn the residual F(x) = H(x) - x：

Output = F(x) + x  (via skip connection)

3. Architecture Variants

  • Basic block: Two 3×3 conv layers + skip connection
  • Bottleneck block: 1×1 → 3×3 → 1×1 convolutions for deeper networks
  • Models: ResNet-18, 34, 50, 101, 152

4. Results

  • 152-layer ResNet achieved 3.57% top-5 error on ImageNet
  • Won ILSVRC 2015 classification challenge
  • Proved deeper networks can achieve better performance

Impact

ResNet is one of the most influential architectures in deep learning history：

  1. Computer Vision: Standard backbone for classification, detection, segmentation
  2. Transfer Learning: Pre-trained ResNets widely used for downstream tasks
  3. Architecture Design: Residual connections adopted in Transformers and beyond
  4. Citations: Over 200,000 citations, among the most cited AI papers

Summary

ResNet’s elegant residual connection design broke through the depth barrier, ushering in the era of “deeper is better” in deep learning.