假如目标检测-终榜R25-安全AI挑战者计划第五期：伪造图像的对抗攻击

关联比赛: 【长期赛】安全AI挑战者计划第五期：伪造图像的对抗攻击

一、主要任务

比赛采用的数据为大量伪造的证书文档类图像。任务是通过提供的训练集学习出有效的检测算法，对测试集的伪造图像进行篡改定位。

二、算法思路

2.1 核心方法

转换为目标检测问题，将篡改区域定义目标类别class-cheat

2.2 数据处理

原始图像尺寸参差不齐，统一使用颜色125填充至1550*1550。

举例：

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702098711-bc4f0f72-6727-491d-9c58-c5535c665bcd.png "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702135902-afbe4505-39b8-42a1-90fd-6cdbb83af1a2.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

颜色125部分定义为第二类别class-pmiss

对mask进行解析，如下红色为class-pmiss，蓝绿色为class-cheat。

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702135902-afbe4505-39b8-42a1-90fd-6cdbb83af1a2.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702291222-d600aa30-5e0a-42de-8fb2-40712918192e.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

据此构建COCO数据集，用于mmdetection目标检测。

2.3 数据增强

原有训练数据集较少，且部分图像来自同一张图像，为避免过拟合。使用了以下方法以数据增强。

原始

cv旋转0

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702599933-e56d8fb4-a831-45d8-9330-7ad1cae00e40.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702582708-6a72d4f1-8c39-4215-9e5a-c0fed8ac4e19.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

cv旋转1

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702681927-b7b52193-ee1a-4ba0-b1d4-dd8760e7c8ce.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702660640-33ea9031-c44b-4c88-bef2-53c2dd151716.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

cv旋转-1

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702713437-523b516d-a169-45d2-a9e1-1654e45d4faa.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702720493-b08dc4c0-4d72-463c-97f7-c34a73629cfe.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

M2（随机生成1/2边长正方形作为遮挡，遮挡部分定义为class-pmiss）

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702830823-2ba26cec-fd0b-49d5-945b-22098765d884.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606702847142-b50fc374-ccae-48b2-8d22-07e252267c78.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

MF（随机生成1/16边长正方形作为遮挡，遮挡部分定义为class-pmiss）

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703028461-afaa1515-d6d6-4c6a-a527-61445e37bd19.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703015741-68b3f5b4-035a-47ee-8c3d-814d6091741e.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

blur（调整蓝色色调，肉眼不可见）

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703063951-dc966676-86bf-4888-9f7e-036957283ad0.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703076706-b9b10fbf-4cc7-45d8-92e1-7b6d690556f5.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

chicken（逛超市看到鸡块想到的鸡块法，也就是随机删减）

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703131717-304ba926-697e-4e86-9198-2667719c2318.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703121285-8d5f271d-4e41-4d7d-ad3a-6a40bffa5040.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

jely（jely提出的1/4随机拼凑）

![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703245900-46a54db8-9f71-452b-a781-3fcbef1704a2.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)![image.png](https://cdn.nlark.com/yuque/0/2020/png/2666308/1606703253022-353bf1f5-b22d-454a-83ef-e664b17165b2.png?x-oss-process=image%2Fresize%2Cw_1500 "image.png" =300x)

时间有限，未逐一验证是否都有效，但本方案最终使用了除blur外全部。

详见 clean_data_code.py

2.4 数据分配

_n = random.choice(list("0000000012"))

随机分配，训练：测试：验证 = 8：1：1.

详见 clean_data_code.py

三、复现流程

3.1 环境依赖

为了方便复现，在nvidia-docker上构建镜像。

3.1.1 docker依赖

FROM registry.cn-shanghai.aliyuncs.com/tcc-public/pytorch:1.1.0-cuda10.0-py3

核心参数：cuda10.0
其他参数同此docker镜像

3.1.2 pip依赖

## RUN apt-get update 
RUN apt-get install libglib2.0-dev libsm6 libxrender1 libxext-dev -y   ## 在构建镜像时安装依赖包 
RUN pip install -i https://mirrors.aliyun.com/pypi/simple -r requirements/build.txt 
RUN pip install -i https://mirrors.aliyun.com/pypi/simple -r requirements/runtime.txt