YOLOv5/v7 更换骨干网络之 MobileNetV3

论文地址：https://arxiv.org/abs/1905.02244
代码地址：https://github.com/xiaolai-sqlai/mobilenetv3

我们展示了基于互补搜索技术和新颖架构设计相结合的下一代 MobileNets。MobileNetV3通过结合硬件感知网络架构搜索（NAS）和 NetAdapt算法对移动设计如何协同工作，利用互补的方法来提高移动端CPU推理整体水平。通过这个过程，创建了两个新的发布的 MobileNet模型：MobileNetV3-Large 和 MobileNetV3-Small，分别针对高资源和低资源用例。然后将这些模型应用于目标检测和语义分割。针对语义分割（或任何密集像素预测）任务，提出了一种新的高效分割解码器 Lite reduce Atrous Spatial Pyramid Pooling（LR-ASPP）。实现了移动端分类，检测和分割的最新SOTA成果。与 MobileNetV2 相比，MobileNetV3-Large 在 ImageNet 分类上的准确率提高了 3.2%，同时延迟降低了 20%。与 MobileNetV2 相比，MobileNetV3-Small 的准确率高 6.6%，同时延迟相当。 MobileNetV3-Large 检测速度比 MobileNetV2 快 25%，在COCO检测上的精度大致相等。MobileNetV3-Large LR-ASPP 的速度比 MobileNetV2 R-ASPP 快 30%，在 Cityscapes segmentation分割数据集上，MobileNetV3-Large LR-ASPP 比 MobileNet V2 R-ASPP 快 34%。

MobileNet V3 主要特点

1. 论文推出两个版本：Large和Small，分别适用于不同的资源用例;
2. 使用NetAdapt算法获得卷积核和通道的最佳数量;
3. 继承V1的深度可分离卷积;
4. 继承V2的具有线性瓶颈的残差结构;
5. 引入SE通道注意力结构;
6. 使用了一种新的激活函数h-swish(x)代替Relu6，h的意思表示hard;
7. 使用了Relu6(x + 3)/6来近似SE模块中的sigmoid;
8. 修改了MobileNetV2后端输出head。

MobileNetV3网络结构

MobileNetV3-Large结构

将YOLOv5主干网络替换为MobileNetV3-large：
yolov5-mobilenetv3large-backbone.yaml

# parameters
nc: 80  # number of classes
depth_multiple: 1.0  
width_multiple: 1.0  

# anchors
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32

   # MobileNetV3-large backbone
  # MobileNetV3_InvertedResidual [out_ch, hid_ch, k_s, stride, SE, HardSwish]
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv_BN_HSwish, [16, 2]],                               # 0-p1/2
   [-1, 1, MobileNetV3_InvertedResidual, [ 16,  16, 3, 1, 0, 0]],  # 1-p1/2
   [-1, 1, MobileNetV3_InvertedResidual, [ 24,  64, 3, 2, 0, 0]],  # 2-p2/4
   [-1, 1, MobileNetV3_InvertedResidual, [ 24,  72, 3, 1, 0, 0]],  # 3-p2/4
   [-1, 1, MobileNetV3_InvertedResidual, [ 40,  72, 5, 2, 1, 0]],  # 4-p3/8
   [-1, 1, MobileNetV3_InvertedResidual, [ 40, 120, 5, 1, 1, 0]],  # 5-p3/8
   [-1, 1, MobileNetV3_InvertedResidual, [ 40, 120, 5, 1, 1, 0]],  # 6-p3/8
   [-1, 1, MobileNetV3_InvertedResidual, [ 80, 240, 3, 2, 0, 1]],  # 7-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [ 80, 200, 3, 1, 0, 1]],  # 8-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [ 80, 184, 3, 1, 0, 1]],  # 9-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [ 80, 184, 3, 1, 0, 1]],  # 10-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [112, 480, 3, 1, 1, 1]],  # 11-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [112, 672, 3, 1, 1, 1]],  # 12-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [160, 672, 5, 1, 1, 1]],  # 13-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [160, 672, 5, 2, 1, 1]],  # 14-p5/32
   [-1, 1, MobileNetV3_InvertedResidual, [160, 960, 5, 1, 1, 1]],  # 15-p5/32
  ]

head:
  [[-1, 1, Conv, [160, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 13], 1, Concat, [1]],  # cat backbone P4
   [-1, 1, C3, [320, False]],   # 19

   [-1, 1, Conv, [320, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 6], 1, Concat, [1]],   # cat backbone P3
   [-1, 1, C3, [360, False]],   # 23 (P3/8-small)

   [-1, 1, Conv, [360, 3, 2]],
   [[-1, 20], 1, Concat, [1]],  # cat head P4
   [-1, 1, C3, [680, False]],   # 26 (P4/16-medium)

   [-1, 1, Conv, [680, 3, 2]],
   [[-1, 16], 1, Concat, [1]],  # cat head P5
   [-1, 1, C3, [840, False]],   # 29 (P5/32-large)

   [[23, 26, 29], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

MobileNetV3-Small结构

将YOLOv5主干网络替换为MobileNetV3-Small：

yolov5-mobilenetv3small-backbone.yaml

# YOLOv5 🚀 by Ultralytics, GPL-3.0 license

# Parameters
nc: 80  # number of classes
depth_multiple: 1.0  # model depth multiple
width_multiple: 1.0  # layer channel multiple
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32

   # Mobilenetv3-small backbone
   # MobileNetV3_InvertedResidual [out_ch, hid_ch, k_s, stride, SE, HardSwish]
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv_BN_HSwish, [16, 2]],                              # 0-p1/2
   [-1, 1, MobileNetV3_InvertedResidual, [16,  16, 3, 2, 1, 0]],  # 1-p2/4
   [-1, 1, MobileNetV3_InvertedResidual, [24,  72, 3, 2, 0, 0]],  # 2-p3/8
   [-1, 1, MobileNetV3_InvertedResidual, [24,  88, 3, 1, 0, 0]],  # 3
   [-1, 1, MobileNetV3_InvertedResidual, [40,  96, 5, 2, 1, 1]],  # 4-p4/16
   [-1, 1, MobileNetV3_InvertedResidual, [40, 240, 5, 1, 1, 1]],  # 5
   [-1, 1, MobileNetV3_InvertedResidual, [40, 240, 5, 1, 1, 1]],  # 6
   [-1, 1, MobileNetV3_InvertedResidual, [48, 120, 5, 1, 1, 1]],  # 7
   [-1, 1, MobileNetV3_InvertedResidual, [48, 144, 5, 1, 1, 1]],  # 8
   [-1, 1, MobileNetV3_InvertedResidual, [96, 288, 5, 2, 1, 1]],  # 9-p5/32
   [-1, 1, MobileNetV3_InvertedResidual, [96, 576, 5, 1, 1, 1]],  # 10
   [-1, 1, MobileNetV3_InvertedResidual, [96, 576, 5, 1, 1, 1]],  # 11
  ]

# YOLOv5 v6.0 head
head:
  [[-1, 1, Conv, [96, 1, 1]],  # 12
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 8], 1, Concat, [1]],  # cat backbone P4
   [-1, 3, C3, [144, False]],  # 15

   [-1, 1, Conv, [144, 1, 1]], # 16
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 3], 1, Concat, [1]],  # cat backbone P3
   [-1, 3, C3, [168, False]],  # 19 (P3/8-small)

   [-1, 1, Conv, [168, 3, 2]],
   [[-1, 16], 1, Concat, [1]], # cat head P4
   [-1, 3, C3, [312, False]],  # 22 (P4/16-medium)

   [-1, 1, Conv, [312, 3, 2]],
   [[-1, 12], 1, Concat, [1]], # cat head P5
   [-1, 3, C3, [408, False]],  # 25 (P5/32-large)

   [[19, 22, 25], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

在YOLOv5项目中添加方式：

common.py中加入以下代码：

# Mobilenetv3Small
class SeBlock(nn.Module):
    def __init__(self, in_channel, reduction=4):
        super().__init__()
        self.Squeeze = nn.AdaptiveAvgPool2d(1)

        self.Excitation = nn.Sequential()
        self.Excitation.add_module('FC1', nn.Conv2d(in_channel, in_channel // reduction, kernel_size=1))  # 1*1卷积与此效果相同
        self.Excitation.add_module('ReLU', nn.ReLU())
        self.Excitation.add_module('FC2', nn.Conv2d(in_channel // reduction, in_channel, kernel_size=1))
        self.Excitation.add_module('Sigmoid', nn.Sigmoid())

    def forward(self, x):
        y = self.Squeeze(x)
        ouput = self.Excitation(y)
        return x * (ouput.expand_as(x))


class Conv_BN_HSwish(nn.Module):
    """
    This equals to
    def conv_3x3_bn(inp, oup, stride):
        return nn.Sequential(
            nn.Conv2d(inp, oup, 3, stride, 1, bias=False),
            nn.BatchNorm2d(oup),
            h_swish()
        )
    """

    def __init__(self, c1, c2, stride):
        super(Conv_BN_HSwish, self).__init__()
        self.conv = nn.Conv2d(c1, c2, 3, stride, 1, bias=False)
        self.bn = nn.BatchNorm2d(c2)
        self.act = nn.Hardswish()

    def forward(self, x):
        return self.act(self.bn(self.conv(x)))


class MobileNetV3_InvertedResidual(nn.Module):
    def __init__(self, inp, oup, hidden_dim, kernel_size, stride, use_se, use_hs):
        super(MobileNetV3_InvertedResidual, self).__init__()
        assert stride in [1, 2]

        self.identity = stride == 1 and inp == oup

        if inp == hidden_dim:
            self.conv = nn.Sequential(
                # dw
                nn.Conv2d(hidden_dim, hidden_dim, kernel_size, stride, (kernel_size - 1) // 2, groups=hidden_dim,
                          bias=False),
                nn.BatchNorm2d(hidden_dim),
                nn.Hardswish() if use_hs else nn.ReLU(),
                # Squeeze-and-Excite
                SeBlock(hidden_dim) if use_se else nn.Sequential(),
                # pw-linear
                nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),
                nn.BatchNorm2d(oup),
            )
        else:
            self.conv = nn.Sequential(
                # pw
                nn.Conv2d(inp, hidden_dim, 1, 1, 0, bias=False),
                nn.BatchNorm2d(hidden_dim),
                nn.Hardswish() if use_hs else nn.ReLU(),
                # dw
                nn.Conv2d(hidden_dim, hidden_dim, kernel_size, stride, (kernel_size - 1) // 2, groups=hidden_dim,
                          bias=False),
                nn.BatchNorm2d(hidden_dim),
                # Squeeze-and-Excite
                SeBlock(hidden_dim) if use_se else nn.Sequential(),
                nn.Hardswish() if use_hs else nn.ReLU(),
                # pw-linear
                nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),
                nn.BatchNorm2d(oup),
            )

    def forward(self, x):
        y = self.conv(x)
        if self.identity:
            return x + y
        else:
            return y

yolo.py中添加如下代码：

本人更多YOLOv5实战内容导航🍀🌟🚀

1. 手把手带你调参Yolo v5 (v6.2)（推理）🌟强烈推荐

2. 手把手带你调参Yolo v5 (v6.2)（训练）🚀

3. 手把手带你调参Yolo v5 (v6.2)（验证）

4. 如何快速使用自己的数据集训练Yolov5模型

5. 手把手带你Yolov5 (v6.2)添加注意力机制(一)（并附上30多种顶会Attention原理图）🌟强烈推荐🍀新增8种

6. 手把手带你Yolov5 (v6.2)添加注意力机制(二)（在C3模块中加入注意力机制）

7. Yolov5如何更换激活函数？

8. Yolov5如何更换BiFPN?

9. Yolov5 (v6.2)数据增强方式解析

10. Yolov5更换上采样方式( 最近邻 / 双线性 / 双立方 / 三线性 / 转置卷积)

11. Yolov5如何更换EIOU / alpha IOU / SIoU？

12. Yolov5更换主干网络之《旷视轻量化卷积神经网络ShuffleNetv2》

13. YOLOv5应用轻量级通用上采样算子CARAFE

14. 空间金字塔池化改进 SPP / SPPF / SimSPPF / ASPP / RFB / SPPCSPC / SPPFCSPC🚀

15. 用于低分辨率图像和小物体的模块SPD-Conv

16. GSConv+Slim-neck 减轻模型的复杂度同时提升精度🍀

17. 头部解耦 | 将YOLOX解耦头添加到YOLOv5 | 涨点杀器🍀

18. Stand-Alone Self-Attention | 搭建纯注意力FPN+PAN结构🍀

19. YOLOv5模型剪枝实战🚀

20. YOLOv5知识蒸馏实战🚀

21. YOLOv7知识蒸馏实战🚀

22. 改进YOLOv5 | 引入密集连接卷积网络DenseNet思想 | 搭建密集连接模块🍀

参考文献：

https://github.com/Gumpest/YOLOv5-Multibackbone-Compression