第J4周：DenseNet与ResNet结合探索

🍨 本文为🔗365天深度学习训练营中的学习记录博客
🍖 原作者：K同学啊

本次尝试根据DenseNet与ResNet的特征来构建一个新的模型结构，目前的思路：将ResNet的残差结构加入到DenseNet中，也就是说把DenseNet的Dense Block当作一个整体，通过残差块连接与输入相加，使用第J3周的数据集进行对比实验，由于只改变了网络结构，其他的代码未作变动，这里只附上更改后的代码。

代码

python">from keras import layers, Model
import kerasdef _DenseLayer(x, growth_rate, bn_size, drop_rate):y = layers.BatchNormalization()(x)y = layers.ReLU()(y)y = layers.Conv2D(bn_size * growth_rate, kernel_size=1, use_bias=False)(y)y = layers.BatchNormalization()(y)y = layers.ReLU()(y)y = layers.Conv2D(growth_rate, kernel_size=3, padding='same', use_bias=False)(y)# Dropout（可选）if drop_rate > 0:y = layers.Dropout(drop_rate)(y)return layers.concatenate([x, y])  # 拼接输入和输出，实现密集连接def _ResidualDenseBlock(x, num_layers, growth_rate, bn_size, drop_rate):dense_input = x  # 保存Dense Block的输入，用于残差连接for i in range(num_layers):x = _DenseLayer(x, growth_rate, bn_size, drop_rate)# 将Dense Block的输入和输出通道数进行匹配if dense_input.shape[-1] != x.shape[-1]:  dense_input = layers.Conv2D(x.shape[-1], kernel_size=1, padding='same', use_bias=False)(dense_input)x = layers.Add()([x, dense_input])return xdef _Transition(x, reduction):x = layers.BatchNormalization()(x)x = layers.ReLU()(x)x = layers.Conv2D(int(x.shape[-1] * reduction), kernel_size=1, use_bias=False)(x)x = layers.AveragePooling2D(pool_size=2, strides=2)(x)return xdef ResidualDenseNet121(input_shape=(224, 224, 3), growth_rate=32, num_init_features=64, drop_rate=0.0, num_classes=1000):img_input = layers.Input(shape=input_shape)# 初始卷积和池化层x = layers.Conv2D(num_init_features, kernel_size=7, strides=2, padding='same')(img_input)x = layers.BatchNormalization()(x)x = layers.ReLU()(x)x = layers.MaxPooling2D((3, 3), strides=2, padding='same')(x)# 添加带有残差连接的Dense Block和Transition层x = _ResidualDenseBlock(x, num_layers=6, growth_rate=growth_rate, bn_size=4, drop_rate=drop_rate)x = _Transition(x, reduction=0.5)x = _ResidualDenseBlock(x, num_layers=12, growth_rate=growth_rate, bn_size=4, drop_rate=drop_rate)x = _Transition(x, reduction=0.5)x = _ResidualDenseBlock(x, num_layers=24, growth_rate=growth_rate, bn_size=4, drop_rate=drop_rate)x = _Transition(x, reduction=0.5)x = _ResidualDenseBlock(x, num_layers=16, growth_rate=growth_rate, bn_size=4, drop_rate=drop_rate)# 最后的批量归一化和全局池化x = layers.BatchNormalization()(x)x = layers.ReLU()(x)x = layers.GlobalAveragePooling2D()(x)# 输出分类层x = layers.Dense(num_classes, activation='softmax')(x)model = Model(inputs=img_input, outputs=x)return model# 构建模型
model = ResidualDenseNet121(input_shape=(224, 224, 3), num_classes=1000)
model.summary()

运行结果：

python">Epoch 1/20
29/29 [==============================] - 222s 255ms/step - loss: 5.2172 - accuracy: 0.4447 - val_loss: 7.5063 - val_accuracy: 0.3628
Epoch 2/20
29/29 [==============================] - 2s 86ms/step - loss: 1.8295 - accuracy: 0.7677 - val_loss: 5.7636 - val_accuracy: 0.4690
Epoch 3/20
29/29 [==============================] - 2s 82ms/step - loss: 0.8690 - accuracy: 0.8562 - val_loss: 4.5134 - val_accuracy: 0.4779
Epoch 4/20
29/29 [==============================] - 2s 82ms/step - loss: 0.5140 - accuracy: 0.8717 - val_loss: 2.6580 - val_accuracy: 0.5575
Epoch 5/20
29/29 [==============================] - 2s 83ms/step - loss: 0.3090 - accuracy: 0.9137 - val_loss: 2.1694 - val_accuracy: 0.7257
Epoch 6/20
29/29 [==============================] - 2s 84ms/step - loss: 0.1874 - accuracy: 0.9558 - val_loss: 1.7057 - val_accuracy: 0.6283
Epoch 7/20
29/29 [==============================] - 2s 84ms/step - loss: 0.0973 - accuracy: 0.9912 - val_loss: 1.3680 - val_accuracy: 0.6195
Epoch 8/20
29/29 [==============================] - 2s 81ms/step - loss: 0.0743 - accuracy: 0.9845 - val_loss: 0.9615 - val_accuracy: 0.7611
Epoch 9/20
29/29 [==============================] - 2s 85ms/step - loss: 0.0393 - accuracy: 1.0000 - val_loss: 1.2206 - val_accuracy: 0.7434
Epoch 10/20
29/29 [==============================] - 2s 85ms/step - loss: 0.0198 - accuracy: 1.0000 - val_loss: 0.6961 - val_accuracy: 0.7965
Epoch 11/20
29/29 [==============================] - 2s 83ms/step - loss: 0.0137 - accuracy: 1.0000 - val_loss: 0.8251 - val_accuracy: 0.7788
Epoch 12/20
29/29 [==============================] - 2s 84ms/step - loss: 0.0098 - accuracy: 1.0000 - val_loss: 0.4899 - val_accuracy: 0.8673
Epoch 13/20
29/29 [==============================] - 2s 84ms/step - loss: 0.0081 - accuracy: 1.0000 - val_loss: 0.4835 - val_accuracy: 0.8496
Epoch 14/20
29/29 [==============================] - 2s 85ms/step - loss: 0.0064 - accuracy: 1.0000 - val_loss: 0.4743 - val_accuracy: 0.8673
Epoch 15/20
29/29 [==============================] - 2s 86ms/step - loss: 0.0055 - accuracy: 1.0000 - val_loss: 0.4286 - val_accuracy: 0.8850
Epoch 16/20
29/29 [==============================] - 2s 85ms/step - loss: 0.0049 - accuracy: 1.0000 - val_loss: 0.4103 - val_accuracy: 0.8850
Epoch 17/20
29/29 [==============================] - 2s 83ms/step - loss: 0.0044 - accuracy: 1.0000 - val_loss: 0.4074 - val_accuracy: 0.9027
Epoch 18/20
29/29 [==============================] - 2s 85ms/step - loss: 0.0040 - accuracy: 1.0000 - val_loss: 0.4042 - val_accuracy: 0.9027
Epoch 19/20
29/29 [==============================] - 2s 83ms/step - loss: 0.0037 - accuracy: 1.0000 - val_loss: 0.3963 - val_accuracy: 0.9027
Epoch 20/20
29/29 [==============================] - 3s 87ms/step - loss: 0.0034 - accuracy: 1.0000 - val_loss: 0.3932 - val_accuracy: 0.9115

Acc与Loss图

在这里插入图片描述

J3周的结果

python">................................................................................................
Epoch 18/20
29/29 ━━━━━━━━━━━━━━━━━━━━ 4s 148ms/step - accuracy: 1.0000 - loss: 0.0046 - val_accuracy: 0.9027 - val_loss: 0.3119
Epoch 19/20
29/29 ━━━━━━━━━━━━━━━━━━━━ 4s 152ms/step - accuracy: 1.0000 - loss: 0.0062 - val_accuracy: 0.9027 - val_loss: 0.3093
Epoch 20/20
29/29 ━━━━━━━━━━━━━━━━━━━━ 4s 148ms/step - accuracy: 1.0000 - loss: 0.0050 - val_accuracy: 0.9027 - val_loss: 0.3102