【笔记】summary函数: summary文件主要调用一系列模块激活hook,用以搭建model(总的layer)中的每一层网络的正向传播框架,使用model(*x)完成网络的正向传播

news/2024/12/2 18:03:03/

流程:首先把model中的每一层网络激活hook,完成正向传播的框架搭建,搭建的过程中顺带将每一层的信息进行统计。然后输出input,完成model的正向传播。

x是input

type(x)
Out: listtype(x[0])
Out: torch.Tensortype(*x)
Out: torch.Tensorx[0].size()
Out: torch.Size([2, 3, 418, 418])(*x).size()
error:...

x

x
Out: 
[tensor([[[[0.3102, 0.8596, 0.1304,  ..., 0.6594, 0.8860, 0.5789],[0.8040, 0.2697, 0.2976,  ..., 0.5797, 0.4450, 0.5615],[0.2392, 0.5074, 0.0021,  ..., 0.6517, 0.7355, 0.6797],...,[0.3715, 0.2288, 0.7578,  ..., 0.0383, 0.3627, 0.5528],[0.1345, 0.3320, 0.5228,  ..., 0.4717, 0.4596, 0.7026],[0.1294, 0.0535, 0.3622,  ..., 0.0159, 0.0584, 0.2387]],[[0.7831, 0.7086, 0.5249,  ..., 0.9599, 0.1168, 0.2670],[0.5323, 0.9834, 0.9716,  ..., 0.2078, 0.9068, 0.4957],[0.0579, 0.6982, 0.6790,  ..., 0.3966, 0.2998, 0.8798],...,[0.6805, 0.0573, 0.6803,  ..., 0.1519, 0.4711, 0.7284],[0.9284, 0.0585, 0.2972,  ..., 0.7272, 0.9505, 0.2277],[0.4118, 0.6793, 0.3267,  ..., 0.3202, 0.8033, 0.5302]],[[0.1242, 0.6022, 0.3829,  ..., 0.9428, 0.2495, 0.6285],[0.3000, 0.0764, 0.7873,  ..., 0.9511, 0.7078, 0.0534],[0.5872, 0.4522, 0.9887,  ..., 0.1104, 0.2189, 0.2736],...,[0.1302, 0.7966, 0.5835,  ..., 0.7391, 0.1846, 0.4527],[0.5570, 0.7166, 0.3877,  ..., 0.1735, 0.9202, 0.5621],[0.5085, 0.6321, 0.1718,  ..., 0.0255, 0.3761, 0.8534]]],[[[0.3739, 0.1188, 0.4226,  ..., 0.6244, 0.2109, 0.9015],[0.1861, 0.7112, 0.1474,  ..., 0.7150, 0.9982, 0.2875],[0.4693, 0.2805, 0.1399,  ..., 0.0569, 0.6584, 0.2678],...,[0.1254, 0.3277, 0.9279,  ..., 0.4351, 0.6777, 0.7394],[0.9116, 0.8686, 0.2146,  ..., 0.3590, 0.0505, 0.5434],[0.4119, 0.8424, 0.3614,  ..., 0.3036, 0.2965, 0.1025]],[[0.1698, 0.3996, 0.4214,  ..., 0.4053, 0.1676, 0.8579],[0.2363, 0.2986, 0.4431,  ..., 0.0400, 0.9726, 0.8652],[0.3392, 0.2502, 0.0449,  ..., 0.5490, 0.1409, 0.7616],...,[0.8507, 0.5658, 0.5749,  ..., 0.5579, 0.7436, 0.0212],[0.8636, 0.6931, 0.7503,  ..., 0.2917, 0.9506, 0.2678],[0.4792, 0.2392, 0.3151,  ..., 0.8005, 0.4148, 0.6095]],[[0.2562, 0.2324, 0.1086,  ..., 0.6738, 0.6497, 0.9233],[0.4929, 0.7130, 0.7814,  ..., 0.4813, 0.5589, 0.3298],[0.3308, 0.5899, 0.0266,  ..., 0.5381, 0.2375, 0.5328],...,[0.8276, 0.9921, 0.5365,  ..., 0.1358, 0.3378, 0.5708],[0.5240, 0.9950, 0.0833,  ..., 0.7758, 0.6101, 0.0930],[0.9067, 0.3900, 0.4336,  ..., 0.4563, 0.3291, 0.6772]]]],device='cuda:0')]

torchsummary.py

import torch
import torch.nn as nn
from torch.autograd import Variablefrom collections import OrderedDict
import numpy as npdef summary(model, input_size, batch_size=-1, device="cuda"):def register_hook(module):def hook(module, input, output):class_name = str(module.__class__).split(".")[-1].split("'")[0]module_idx = len(summary)m_key = "%s-%i" % (class_name, module_idx + 1)summary[m_key] = OrderedDict()summary[m_key]["input_shape"] = list(input[0].size())summary[m_key]["input_shape"][0] = batch_sizeif isinstance(output, (list, tuple)):summary[m_key]["output_shape"] = [[-1] + list(o.size())[1:] for o in output]else:summary[m_key]["output_shape"] = list(output.size())summary[m_key]["output_shape"][0] = batch_sizeparams = 0if hasattr(module, "weight") and hasattr(module.weight, "size"):params += torch.prod(torch.LongTensor(list(module.weight.size())))summary[m_key]["trainable"] = module.weight.requires_gradif hasattr(module, "bias") and hasattr(module.bias, "size"):params += torch.prod(torch.LongTensor(list(module.bias.size())))summary[m_key]["nb_params"] = paramsif (not isinstance(module, nn.Sequential)and not isinstance(module, nn.ModuleList)and not (module == model)):hooks.append(module.register_forward_hook(hook))device = device.lower()assert device in ["cuda","cpu",], "Input device is not valid, please specify 'cuda' or 'cpu'"if device == "cuda" and torch.cuda.is_available():dtype = torch.cuda.FloatTensorelse:dtype = torch.FloatTensor# multiple inputs to the networkif isinstance(input_size, tuple):input_size = [input_size]# batch_size of 2 for batchnormx = [torch.rand(2, *in_size).type(dtype) for in_size in input_size]# print(type(x[0]))# create propertiessummary = OrderedDict()hooks = []# register hookmodel.apply(register_hook)# make a forward pass# print(x.shape)model(*x)# remove these hooksfor h in hooks:h.remove()print("----------------------------------------------------------------")line_new = "{:>20}  {:>25} {:>15}".format("Layer (type)", "Output Shape", "Param #")print(line_new)print("================================================================")total_params = 0total_output = 0trainable_params = 0for layer in summary:# input_shape, output_shape, trainable, nb_paramsline_new = "{:>20}  {:>25} {:>15}".format(layer,str(summary[layer]["output_shape"]),"{0:,}".format(summary[layer]["nb_params"]),)total_params += summary[layer]["nb_params"]total_output += np.prod(summary[layer]["output_shape"])if "trainable" in summary[layer]:if summary[layer]["trainable"] == True:trainable_params += summary[layer]["nb_params"]print(line_new)# assume 4 bytes/number (float on cuda).total_input_size = abs(np.prod(input_size) * batch_size * 4. / (1024 ** 2.))total_output_size = abs(2. * total_output * 4. / (1024 ** 2.))  # x2 for gradientstotal_params_size = abs(total_params.numpy() * 4. / (1024 ** 2.))total_size = total_params_size + total_output_size + total_input_sizeprint("================================================================")print("Total params: {0:,}".format(total_params))print("Trainable params: {0:,}".format(trainable_params))print("Non-trainable params: {0:,}".format(total_params - trainable_params))print("----------------------------------------------------------------")print("Input size (MB): %0.2f" % total_input_size)print("Forward/backward pass size (MB): %0.2f" % total_output_size)print("Params size (MB): %0.2f" % total_params_size)print("Estimated Total Size (MB): %0.2f" % total_size)print("----------------------------------------------------------------")# return summary

/home/wangbin/anaconda3/envs/deep_learning/bin/python3.7 /media/wangbin/F/深度学习_程序/yolo_practice/resnet_yolo.py
----------------------------------------------------------------Layer (type)               Output Shape         Param #
================================================================Conv2d-1         [-1, 64, 209, 209]           9,408BatchNorm2d-2         [-1, 64, 209, 209]             128ReLU-3         [-1, 64, 209, 209]               0MaxPool2d-4         [-1, 64, 105, 105]               0Conv2d-5         [-1, 64, 105, 105]           4,096BatchNorm2d-6         [-1, 64, 105, 105]             128ReLU-7         [-1, 64, 105, 105]               0Conv2d-8         [-1, 64, 105, 105]          36,864BatchNorm2d-9         [-1, 64, 105, 105]             128ReLU-10         [-1, 64, 105, 105]               0Conv2d-11        [-1, 256, 105, 105]          16,384BatchNorm2d-12        [-1, 256, 105, 105]             512Conv2d-13        [-1, 256, 105, 105]          16,384BatchNorm2d-14        [-1, 256, 105, 105]             512ReLU-15        [-1, 256, 105, 105]               0Bottleneck-16        [-1, 256, 105, 105]               0Conv2d-17         [-1, 64, 105, 105]          16,384BatchNorm2d-18         [-1, 64, 105, 105]             128ReLU-19         [-1, 64, 105, 105]               0Conv2d-20         [-1, 64, 105, 105]          36,864BatchNorm2d-21         [-1, 64, 105, 105]             128ReLU-22         [-1, 64, 105, 105]               0Conv2d-23        [-1, 256, 105, 105]          16,384BatchNorm2d-24        [-1, 256, 105, 105]             512ReLU-25        [-1, 256, 105, 105]               0Bottleneck-26        [-1, 256, 105, 105]               0Conv2d-27         [-1, 64, 105, 105]          16,384BatchNorm2d-28         [-1, 64, 105, 105]             128ReLU-29         [-1, 64, 105, 105]               0Conv2d-30         [-1, 64, 105, 105]          36,864BatchNorm2d-31         [-1, 64, 105, 105]             128ReLU-32         [-1, 64, 105, 105]               0Conv2d-33        [-1, 256, 105, 105]          16,384BatchNorm2d-34        [-1, 256, 105, 105]             512ReLU-35        [-1, 256, 105, 105]               0Bottleneck-36        [-1, 256, 105, 105]               0Conv2d-37        [-1, 128, 105, 105]          32,768BatchNorm2d-38        [-1, 128, 105, 105]             256ReLU-39        [-1, 128, 105, 105]               0Conv2d-40          [-1, 128, 53, 53]         147,456BatchNorm2d-41          [-1, 128, 53, 53]             256ReLU-42          [-1, 128, 53, 53]               0Conv2d-43          [-1, 512, 53, 53]          65,536BatchNorm2d-44          [-1, 512, 53, 53]           1,024Conv2d-45          [-1, 512, 53, 53]         131,072BatchNorm2d-46          [-1, 512, 53, 53]           1,024ReLU-47          [-1, 512, 53, 53]               0Bottleneck-48          [-1, 512, 53, 53]               0Conv2d-49          [-1, 128, 53, 53]          65,536BatchNorm2d-50          [-1, 128, 53, 53]             256ReLU-51          [-1, 128, 53, 53]               0Conv2d-52          [-1, 128, 53, 53]         147,456BatchNorm2d-53          [-1, 128, 53, 53]             256ReLU-54          [-1, 128, 53, 53]               0Conv2d-55          [-1, 512, 53, 53]          65,536BatchNorm2d-56          [-1, 512, 53, 53]           1,024ReLU-57          [-1, 512, 53, 53]               0Bottleneck-58          [-1, 512, 53, 53]               0Conv2d-59          [-1, 128, 53, 53]          65,536BatchNorm2d-60          [-1, 128, 53, 53]             256ReLU-61          [-1, 128, 53, 53]               0Conv2d-62          [-1, 128, 53, 53]         147,456BatchNorm2d-63          [-1, 128, 53, 53]             256ReLU-64          [-1, 128, 53, 53]               0Conv2d-65          [-1, 512, 53, 53]          65,536BatchNorm2d-66          [-1, 512, 53, 53]           1,024ReLU-67          [-1, 512, 53, 53]               0Bottleneck-68          [-1, 512, 53, 53]               0Conv2d-69          [-1, 128, 53, 53]          65,536BatchNorm2d-70          [-1, 128, 53, 53]             256ReLU-71          [-1, 128, 53, 53]               0Conv2d-72          [-1, 128, 53, 53]         147,456BatchNorm2d-73          [-1, 128, 53, 53]             256ReLU-74          [-1, 128, 53, 53]               0Conv2d-75          [-1, 512, 53, 53]          65,536BatchNorm2d-76          [-1, 512, 53, 53]           1,024ReLU-77          [-1, 512, 53, 53]               0Bottleneck-78          [-1, 512, 53, 53]               0Conv2d-79          [-1, 256, 53, 53]         131,072BatchNorm2d-80          [-1, 256, 53, 53]             512ReLU-81          [-1, 256, 53, 53]               0Conv2d-82          [-1, 256, 27, 27]         589,824BatchNorm2d-83          [-1, 256, 27, 27]             512ReLU-84          [-1, 256, 27, 27]               0Conv2d-85         [-1, 1024, 27, 27]         262,144BatchNorm2d-86         [-1, 1024, 27, 27]           2,048Conv2d-87         [-1, 1024, 27, 27]         524,288BatchNorm2d-88         [-1, 1024, 27, 27]           2,048ReLU-89         [-1, 1024, 27, 27]               0Bottleneck-90         [-1, 1024, 27, 27]               0Conv2d-91          [-1, 256, 27, 27]         262,144BatchNorm2d-92          [-1, 256, 27, 27]             512ReLU-93          [-1, 256, 27, 27]               0Conv2d-94          [-1, 256, 27, 27]         589,824BatchNorm2d-95          [-1, 256, 27, 27]             512ReLU-96          [-1, 256, 27, 27]               0Conv2d-97         [-1, 1024, 27, 27]         262,144BatchNorm2d-98         [-1, 1024, 27, 27]           2,048ReLU-99         [-1, 1024, 27, 27]               0Bottleneck-100         [-1, 1024, 27, 27]               0Conv2d-101          [-1, 256, 27, 27]         262,144BatchNorm2d-102          [-1, 256, 27, 27]             512ReLU-103          [-1, 256, 27, 27]               0Conv2d-104          [-1, 256, 27, 27]         589,824BatchNorm2d-105          [-1, 256, 27, 27]             512ReLU-106          [-1, 256, 27, 27]               0Conv2d-107         [-1, 1024, 27, 27]         262,144BatchNorm2d-108         [-1, 1024, 27, 27]           2,048ReLU-109         [-1, 1024, 27, 27]               0Bottleneck-110         [-1, 1024, 27, 27]               0Conv2d-111          [-1, 256, 27, 27]         262,144BatchNorm2d-112          [-1, 256, 27, 27]             512ReLU-113          [-1, 256, 27, 27]               0Conv2d-114          [-1, 256, 27, 27]         589,824BatchNorm2d-115          [-1, 256, 27, 27]             512ReLU-116          [-1, 256, 27, 27]               0Conv2d-117         [-1, 1024, 27, 27]         262,144BatchNorm2d-118         [-1, 1024, 27, 27]           2,048ReLU-119         [-1, 1024, 27, 27]               0Bottleneck-120         [-1, 1024, 27, 27]               0Conv2d-121          [-1, 256, 27, 27]         262,144BatchNorm2d-122          [-1, 256, 27, 27]             512ReLU-123          [-1, 256, 27, 27]               0Conv2d-124          [-1, 256, 27, 27]         589,824BatchNorm2d-125          [-1, 256, 27, 27]             512ReLU-126          [-1, 256, 27, 27]               0Conv2d-127         [-1, 1024, 27, 27]         262,144BatchNorm2d-128         [-1, 1024, 27, 27]           2,048ReLU-129         [-1, 1024, 27, 27]               0Bottleneck-130         [-1, 1024, 27, 27]               0Conv2d-131          [-1, 256, 27, 27]         262,144BatchNorm2d-132          [-1, 256, 27, 27]             512ReLU-133          [-1, 256, 27, 27]               0Conv2d-134          [-1, 256, 27, 27]         589,824BatchNorm2d-135          [-1, 256, 27, 27]             512ReLU-136          [-1, 256, 27, 27]               0Conv2d-137         [-1, 1024, 27, 27]         262,144BatchNorm2d-138         [-1, 1024, 27, 27]           2,048ReLU-139         [-1, 1024, 27, 27]               0Bottleneck-140         [-1, 1024, 27, 27]               0Conv2d-141          [-1, 512, 27, 27]         524,288BatchNorm2d-142          [-1, 512, 27, 27]           1,024ReLU-143          [-1, 512, 27, 27]               0Conv2d-144          [-1, 512, 14, 14]       2,359,296BatchNorm2d-145          [-1, 512, 14, 14]           1,024ReLU-146          [-1, 512, 14, 14]               0Conv2d-147         [-1, 2048, 14, 14]       1,048,576BatchNorm2d-148         [-1, 2048, 14, 14]           4,096Conv2d-149         [-1, 2048, 14, 14]       2,097,152BatchNorm2d-150         [-1, 2048, 14, 14]           4,096ReLU-151         [-1, 2048, 14, 14]               0Bottleneck-152         [-1, 2048, 14, 14]               0Conv2d-153          [-1, 512, 14, 14]       1,048,576BatchNorm2d-154          [-1, 512, 14, 14]           1,024ReLU-155          [-1, 512, 14, 14]               0Conv2d-156          [-1, 512, 14, 14]       2,359,296BatchNorm2d-157          [-1, 512, 14, 14]           1,024ReLU-158          [-1, 512, 14, 14]               0Conv2d-159         [-1, 2048, 14, 14]       1,048,576BatchNorm2d-160         [-1, 2048, 14, 14]           4,096ReLU-161         [-1, 2048, 14, 14]               0Bottleneck-162         [-1, 2048, 14, 14]               0Conv2d-163          [-1, 512, 14, 14]       1,048,576BatchNorm2d-164          [-1, 512, 14, 14]           1,024ReLU-165          [-1, 512, 14, 14]               0Conv2d-166          [-1, 512, 14, 14]       2,359,296BatchNorm2d-167          [-1, 512, 14, 14]           1,024ReLU-168          [-1, 512, 14, 14]               0Conv2d-169         [-1, 2048, 14, 14]       1,048,576BatchNorm2d-170         [-1, 2048, 14, 14]           4,096ReLU-171         [-1, 2048, 14, 14]               0Bottleneck-172         [-1, 2048, 14, 14]               0Conv2d-173          [-1, 256, 14, 14]         524,288BatchNorm2d-174          [-1, 256, 14, 14]             512Conv2d-175          [-1, 256, 14, 14]         589,824BatchNorm2d-176          [-1, 256, 14, 14]             512Conv2d-177          [-1, 256, 14, 14]          65,536BatchNorm2d-178          [-1, 256, 14, 14]             512Conv2d-179          [-1, 256, 14, 14]         524,288BatchNorm2d-180          [-1, 256, 14, 14]             512
detnet_bottleneck-181          [-1, 256, 14, 14]               0Conv2d-182          [-1, 256, 14, 14]          65,536BatchNorm2d-183          [-1, 256, 14, 14]             512Conv2d-184          [-1, 256, 14, 14]         589,824BatchNorm2d-185          [-1, 256, 14, 14]             512Conv2d-186          [-1, 256, 14, 14]          65,536BatchNorm2d-187          [-1, 256, 14, 14]             512
detnet_bottleneck-188          [-1, 256, 14, 14]               0Conv2d-189          [-1, 256, 14, 14]          65,536BatchNorm2d-190          [-1, 256, 14, 14]             512Conv2d-191          [-1, 256, 14, 14]         589,824BatchNorm2d-192          [-1, 256, 14, 14]             512Conv2d-193          [-1, 256, 14, 14]          65,536BatchNorm2d-194          [-1, 256, 14, 14]             512
detnet_bottleneck-195          [-1, 256, 14, 14]               0Conv2d-196           [-1, 30, 14, 14]          69,120BatchNorm2d-197           [-1, 30, 14, 14]              60
================================================================
Total params: 26,728,060
Trainable params: 26,728,060
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 2.00
Forward/backward pass size (MB): 1038.47
Params size (MB): 101.96
Estimated Total Size (MB): 1142.43
----------------------------------------------------------------Process finished with exit code 0


http://www.ppmy.cn/news/290165.html

相关文章

启天微型计算机可以开vt吗,如何开启笔记本的Virtualization Technology虚拟化技术功能...

现象描述 笔记本是否支持虚拟化功能?是否能开启?如何操作? 现象分析开启虚拟化技术支持,需几个方面的条件支持:芯片组自身支持、BIOS提供支持、处理器自身支持、操作系统支持。操作系统方面,主流操作系统均…

云时代阿里服务器的演进与展望

在整个基础设施中,服务器是面向最终用户提供计算和存储资源,且在数据中心的整体 TCO 中占比最大的至关重要的角色。服务器从诞生到现在,每一次技术的突破和换代,都是朝着更开放、更易用、更优 TCO、更易交付的方向演进。 本文选自…

07_创建tensor,从numpy创建,从List创建,设置默认类型,rand/rand_like,randint,full,arange,linspace/logspace,linspace等等

1.7.创建Tensor 1.7.1.Import from numpy 1.7.2.Import from List 1.7.3.set default type 1.7.4.rand/rand_like,randint 1.7.5.full() 1.7.6.arange() 1.7.7.linspace/logspace 1.7.7.1.torch.linspace() 1.7.7.2.torch.logspace() 1.7.8.ones/zeros/eye 1.7.8.1.ones 1.7.8.…

GM7150 低功耗 NTSC/PAL 视频解码器 PIN TO PIN 完全替代TVP5150

GM7150 低功耗 NTSC/PAL 视频解码器 PIN TO PIN 完全替代TVP5150 1 概述 GM7150 是一款9 位视频输入预处理芯片,该芯片采用CMOS 工艺,通过I2C 总线与PC 或DSP 相连构成应用系统。它内部包含1个模拟处理通道,能实现CVBS、S-Video视频信号源选…

电源系统优化设计,低压差稳压器(LDO)如何选型?

目录 1、压差 2、裕量电压 3、静态电流和接地电流 4、关断电流 5、效率 6、直流负载调整率 7、直流输入电压调整率 8、直流精度 9、负载瞬态响应 10、线路瞬态响应 11、电源抑制 12、PSRR 与频率的关系 13、PSRR 与负载电流的关系 14、PSRR与LDO裕量的关系 15、…

Nvidia驱动支持的linux版本,完善支持 NVIDIA显卡Linux驱动275.19正式版

虽然NVIDIA已经在开发280系列驱动,并在月初发布了一个测试版,但是近日依然发布了一个新的275系列驱动的稳定版本----275.19。275.19增加了对GeForce GT 540M的支持,还修正了几个较为严重的Bug。 具体更新如下: 1、增加了对GeForce…

Java 设计模式最佳实践:三、行为模式

原文:Design Patterns and Best Practices in Java 协议:CC BY-NC-SA 4.0 贡献者:飞龙 本文来自【ApacheCN Java 译文集】,采用译后编辑(MTPE)流程来尽可能提升效率。 本章的目的是学习行为模式。行为模式是…

转接口IC GM7150BN/ GM7150BC:CVBS转BT656芯片 低功耗NTSC/PAL 视频解码器

1 概述 GM7150 是一款9 位视频输入预处理芯片,该芯片采用CMOS 工艺,通过I2C 总线与PC 或DSP 相连构成应用系统。 它内部包含1 个模拟处理通道,能实现CVBS、S-Video 视频信号源选择、A/D 转换、自动钳位、自动增益控制(AGC&a…