基于YOLO8的WholeBody估计

news/2024/10/31 5:28:48/

目录

  • 前言
  • 一、制作YOLO格式的数据集
  • 二、开始训练


前言

YOLO很强大,强大到可以做多个联合检测的任务,不仅准确,而且速度还快.例如,YOLOPose不可一世.那么,是否可以基于YOLPose实现更多的task呢?比如,基于YOLO的全身关键点检测,包括人脸人体手足的所有关键点的一起检测呢?这里探讨实现的可能性.


全人体关键点检测效果如下所示:
在这里插入图片描述

一、制作YOLO格式的数据集

参考COCO-wholebody的标注,我们需要将他们转为YOLO格式的数据集.
如果采用原始的coco2yolo,会遇到这样的问题:

val: WARNING ⚠️ /home/wqt/Datasets/coco/images/val2017/000000369503.jpg: ignoring corrupt image/label: non-normalized or out of bounds coordinate labels
......
......
val: WARNING ⚠️ /home/wqt/Datasets/coco/images/val2017/000000497344.jpg: ignoring corrupt image/label: non-normalized or out of bounds coordinate labels

即存在大量的图片标注是超过图像的范围,而yolo归一化坐标在[0, 1]之间.所以有必要对格式进行修正,即当坐标点不在图像宽高范围内,我们将之调整到边界上,并设置visitiable=0,即不可见的.
具体可参考coco2yolo.py工程的文件如下:

"""
2021/1/24
COCO 格式的数据集转化为 YOLO 格式的数据集,源代码采取遍历方式,太慢,
这里改进了一下时间复杂度,从O(nm)改为O(n+m),但是牺牲了一些内存占用
--json_path 输入的json文件路径
--save_path 保存的文件夹名字,默认为当前目录下的labels。
"""import os 
import json
from tqdm import tqdm
import argparse
import numpyparser = argparse.ArgumentParser()
parser.add_argument('--json_path', default='./instances_val2017.json',type=str, help="input: coco format(json)")
parser.add_argument('--save_path', default='./labels', type=str, help="specify where to save the output dir of labels")
arg = parser.parse_args()def convert(size, box):dw = 1. / (size[0])dh = 1. / (size[1])x = box[0] + box[2] / 2.0y = box[1] + box[3] / 2.0w = box[2]h = box[3]x = x * dww = w * dwy = y * dhh = h * dhreturn (x, y, w, h)def convertKpts(size, kpts):dw = 1. / (size[0])dh = 1. / (size[1])kpts_str = ''for x in range(len(kpts)):if x % 3 == 0:w_factor = kpts[x]/ (size[0])if w_factor > 1:w_factor = 1kpts_str = kpts_str + str(w_factor) + ' 'h_factor = kpts[x+1]/ (size[1])if h_factor > 1:h_factor = 1kpts_str = kpts_str + str(h_factor) + ' 'conf_factor = kpts[x+2]if w_factor == 1 or h_factor == 1:conf_factor = 0kpts_str = kpts_str + str(conf_factor) + ' 'return kpts_strif __name__ == '__main__':json_file =   arg.json_path # COCO Object Instance 类型的标注ana_txt_save_path = arg.save_path  # 保存的路径data = json.load(open(json_file, 'r'))if not os.path.exists(ana_txt_save_path):os.makedirs(ana_txt_save_path)id_map = {} # coco数据集的id不连续!重新映射一下再输出!for i, category in enumerate(data['categories']): id_map[category['id']] = i# 通过事先建表来降低时间复杂度max_id = 0for img in data['images']:max_id = max(max_id, img['id'])# 注意这里不能写作 [[]]*(max_id+1),否则列表内的空列表共享地址img_ann_dict = [[] for i in range(max_id+1)] for i, ann in enumerate(data['annotations']):img_ann_dict[ann['image_id']].append(i)for img in tqdm(data['images']):filename = img["file_name"]img_width = img["width"]img_height = img["height"]img_id = img["id"]head, tail = os.path.splitext(filename)ana_txt_name = head + ".txt"  # 对应的txt名字,与jpg一致f_txt = open(os.path.join(ana_txt_save_path, ana_txt_name), 'w')'''for ann in data['annotations']:if ann['image_id'] == img_id:box = convert((img_width, img_height), ann["bbox"])f_txt.write("%s %s %s %s %s\n" % (id_map[ann["category_id"]], box[0], box[1], box[2], box[3]))'''# 这里可以直接查表而无需重复遍历for ann_id in img_ann_dict[img_id]:ann = data['annotations'][ann_id]if ann["bbox"] is None:continuebox = convert((img_width, img_height), ann["bbox"])# id (1) + bbox (4)f_txt.write("%s %s %s %s %s " % (id_map[ann["category_id"]], box[0], box[1], box[2], box[3]))#keypoints(51) + face kpts (204) + lefthand_kpts () + righthand_kpts () + foot_kpts ()keypoints = convertKpts((img_width, img_height),ann['keypoints'])face_kpts = convertKpts((img_width, img_height),ann['face_kpts'])lefthand_kpts = convertKpts((img_width, img_height), ann['lefthand_kpts'])righthand_kpts = convertKpts((img_width, img_height), ann['righthand_kpts'])foot_kpts = convertKpts((img_width, img_height), ann['foot_kpts'])f_txt.write(keypoints + foot_kpts + face_kpts + lefthand_kpts + righthand_kpts)# obj['keypoints'] + obj['foot_kpts'] + obj['face_kpts'] + obj['lefthand_kpts'] + obj['righthand_kpts']f_txt.write('\n') f_txt.close()

二、开始训练

对GPU不足的人特别不友好,一个epoch下来,几乎要半个小时;没办法,batch-size只能最大设置为16.

看下样本示例:
在这里插入图片描述
这样本标注的非常密集,或微小的目标也在标注范围内.

接下来就是漫长的训练过程来.
训练配置如下:

task=pose, mode=train, model=/home/wqt/NewProjects/ultralyticsWholeBody/runs/pose/train9/weights/best.pt, data=coco8-pose.yaml, epochs=100, patience=50, batch=16, imgsz=640, save=True, save_period=20, cache=False, device=, workers=8, project=None, name=/home/wqt/NewProjects/ultralyticsWholeBody/runs/pose/train, exist_ok=False, pretrained=True, optimizer=SGD, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=0, resume=False, amp=True, fraction=1.0, profile=False, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, show=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, vid_stride=1, line_width=None, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, boxes=True, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0, cfg=None, v5loader=False, tracker=botsort.yaml, save_dir=/home/wqt/NewProjects/ultralyticsWholeBody/runs/pose/train10

模型配置如下:

from  n    params  module                                       arguments                     0                  -1  1       928  ultralytics.nn.modules.conv.Conv             [3, 32, 3, 2]                 1                  -1  1     18560  ultralytics.nn.modules.conv.Conv             [32, 64, 3, 2]                2                  -1  1     29056  ultralytics.nn.modules.block.C2f             [64, 64, 1, True]             3                  -1  1     73984  ultralytics.nn.modules.conv.Conv             [64, 128, 3, 2]               4                  -1  2    197632  ultralytics.nn.modules.block.C2f             [128, 128, 2, True]           5                  -1  1    295424  ultralytics.nn.modules.conv.Conv             [128, 256, 3, 2]              6                  -1  2    788480  ultralytics.nn.modules.block.C2f             [256, 256, 2, True]           7                  -1  1   1180672  ultralytics.nn.modules.conv.Conv             [256, 512, 3, 2]              8                  -1  1   1838080  ultralytics.nn.modules.block.C2f             [512, 512, 1, True]           9                  -1  1    656896  ultralytics.nn.modules.block.SPPF            [512, 512, 5]                 10                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          11             [-1, 6]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           12                  -1  1    591360  ultralytics.nn.modules.block.C2f             [768, 256, 1]                 13                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          14             [-1, 4]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           15                  -1  1    148224  ultralytics.nn.modules.block.C2f             [384, 128, 1]                 16                  -1  1    147712  ultralytics.nn.modules.conv.Conv             [128, 128, 3, 2]              17            [-1, 12]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           18                  -1  1    493056  ultralytics.nn.modules.block.C2f             [384, 256, 1]                 19                  -1  1    590336  ultralytics.nn.modules.conv.Conv             [256, 256, 3, 2]              20             [-1, 9]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           21                  -1  1   1969152  ultralytics.nn.modules.block.C2f             [768, 512, 1]                 22        [15, 18, 21]  1  10115986  ultralytics.nn.modules.head.Pose             [1, [133, 3], [128, 256, 512]]
YOLOv8s-pose summary: 250 layers, 19135538 parameters, 19135522 gradients

数据集与训练参数如下:

Transferred 397/397 items from pretrained weights
AMP: running Automatic Mixed Precision (AMP) checks with YOLOv8n...
AMP: checks passed ✅
optimizer: SGD(lr=0.01) with parameter groups 63 weight(decay=0.0), 73 weight(decay=0.0005), 72 bias
train: Scanning /home/wqt/Datasets/coco/labels/train2017.cache... 64115 images, 0 backgrounds, 0 corrupt: 
val: Scanning /home/wqt/Datasets/coco/labels/val2017.cache... 2693 images, 0 backgrounds, 0 corrupt
Plotting labels to /home/wqt/NewProjects/ultralyticsWholeBody/runs/pose/train10/labels.jpg... 
Image sizes 640 train, 640 val
Using 8 dataloader workers
Logging results to /home/wqt/NewProjects/ultralyticsWholeBody/runs/pose/train10
Starting training for 100 epochs...

http://www.ppmy.cn/news/815376.html

相关文章

FMD0-A15B20-G24比例换向阀放大器

FMD0-A08B15-G24、FMD0-A15B20-G24、FMD0-A08B20-G24、FMD0-A15B20-G24比例换向阀是由连个二位三通比例换向阀插装组合而成,两个插件分别对应油口A和B,A/B口流量可以分别选择,以相同信号不同流量的输出;该系列阀件P到A或P到B 节流口大小受比例…

原子力显微镜的特点

在做原子力显微镜AFM测试时,科学指南针检测平台工作人员在与很多同学沟通中了解到,好多同学对AFM测试不太了解,针对此,科学指南针检测平台团队组织相关同事对网上海量知识进行整理,希望可以帮助到科研圈的伙伴们&#…

storm显微镜成像原理_超分辨率显微镜技术原理概述

原标题:超分辨率显微镜技术原理概述 如今,科学家们已经研发了多种超分辨率技术,远远超出了衍射J限,能够观察到分子尺度的细节。SRM技术可以将细胞结构解析为亚细胞水平,从而获取有关细胞组分的3D结构的信息,并可以观察到单分子共定位。 下面我们来简要概述了时下几种流行…

共聚焦显微镜各部分名称和功能

以共聚焦技术为原理,用于对各种精密器件及材料表面进行微纳米级测量的共聚焦显微镜,可对材料样品的表面形貌直接进行成像,横向分辨率可达1nm,Z轴分辨率可达0.5nm。不仅可对样品表面形貌进行测量,提供表征微观形貌的轮廓…

原子力显微镜在材料科学研究中的应用(一)

在做原子力显微镜AFM测试时,科学指南针检测平台工作人员在与很多同学沟通中了解到,好多同学对AFM测试不太了解,针对此,科学指南针检测平台团队组织相关同事对网上海量知识进行整理,希望可以帮助到科研圈的伙伴们&#…

球差透射电镜和透射电镜的测试能力区别在哪?

透射电子显微镜是指可以看清分辨率达到纳米级的物体精细结构观测仪器,而球差透射电镜的球差是像差的一种,是影响TEM分辨率的主要原因之一。相比TEM透射电镜,由于球差电镜有效削减了像差所以分辨率得到显著提高,分辨率的提高意味着…

原子力显微镜功能技术

在做原子力显微镜AFM测试时,科学指南针检测平台工作人员在与很多同学沟通中了解到,好多同学对AFM测试原子力显微镜功能技术不太了解,针对此,科学指南针检测平台团队组织相关同事对网上海量知识进行整理,希望可以帮助到…

【已解决】体视显微镜左右成像大小不一致

体视显微镜左右成像大小不一致 问题描述 设备型号:舜宇SZMN7045-TRB4(三目体视显微镜) 参数: 机型:SZMN7045TR-B4 纯光学玻璃镜片 全金属机身 标准配置大视野高眼点广角WF10X/20mm镀蓝紫膜目镜 物镜配置 0.7X-4.5X连续…