Title 

题目

Automatic Inter-frame Patient Motion Correction  for Dynamic Cardiac PET Using Deep Learning

利用深度学习进行动态心脏PET的自动帧间患者运动校正

01

文献速递介绍

OSITRON正电子发射断层扫描（PET）心肌灌注成像已被证明相较于其他非侵入性成像模式，能够提高冠状动脉疾病（CAD）的诊断准确性。使用动态PET进行心肌血流（MBF）和心肌流量储备（MFR）的绝对定量已被证明在诊断和预后价值方面优于传统的相对心肌灌注成像。在典型的动态PET中，从注射放射性标记示踪剂开始，连续几分钟内获取动态图像序列，直至心肌完全灌注。通常在重建的动态图像上定义感兴趣区域，以采样心肌组织和左心室（LV）腔的时间活动曲线（TAC）。这些TAC可以通过动力学建模进一步处理，以量化MBF。

动态成像期间的患者运动，通常包括呼吸运动、心脏运动和自主体运动，可能会导致MBF估计中的错误。具体来说，患者运动可能表现为帧间运动，这可能导致连续动态图像帧之间心脏对准不良，并导致由于ROI采样不一致而导致的扭曲的TAC。另一方面，帧内运动可能导致图像模糊以及左心室腔内测得的图像衍生输入函数（IDIF）的不准确性。由于患者运动引起的PET和基于CT的衰减图之间的不匹配，还可能引入额外的错误。

过去已经对PET中的呼吸和心脏运动校正方法进行了研究。一些研究使用外部运动跟踪标记物或传感器来跟踪呼吸运动，以较高的时间分辨率进行跟踪。心电图（ECG）仍然是跟踪心脏运动的黄金标准，因为它具有较高的时间分辨率。然而，这样的跟踪系统通常需要额外的设置时间，并且可能并不总是可用。相比之下，不需要外部设备的数据驱动运动检测方法更受青睐，以促进运动校正的更容易的临床转化。然而，很少有研究集中在82Rb心脏动态PET成像中帧间运动的校正，因为其具有挑战性。

82Rb的快速变化的生物分布导致示踪剂的分布随时间的推移而动态变化，这可能导致图像配准困难。图像配准的准确性通常依赖于要与之配准的两个图像之间的相似性。由于示踪剂的快速动力学，一个动态帧中的示踪剂分布可能与另一个帧中的示踪剂分布有很大不同，这可能导致不准确甚至失败的图像配准。这在早期动态帧（血池相）特别明显，因此现有的运动校正研究主要集中在后期动态帧（心肌组织帧）。为了解决这个挑战，Hunter等人提出通过投影一致性条件进行衰减发射对齐来对动态心脏PET-CT的患者体运动进行校正。该方法在早期和后期帧上表现出同样好的效果，但仅在模拟和幻像数据上进行了评估。Lee等人开发了一种用于动态82Rb心脏PET的自动运动校正框架。首先使用自动算法识别右心室血池（RVBP）相、左心室血池（LVBP）相和组织相，然后将每个动态帧刚性对准以匹配固定的后期组织相累积帧以实现运动校正。在注册中使用了归一化梯度场，而不是图像强度，以考虑血池相期间示踪剂的快速动力学。Lee的结果与临床研究中手动运动校正结果一致，该方法适用于整个动态序列。然而，由于活动在血池和心肌组织中均有，因此Lee方法在过渡帧（在LV血液相和组织相之间）的性能存疑，导致边界无法识别，因此无法可靠地计算出梯度来进行注册。RV帧也没有经过验证，因为缺乏手动运动校正。由于组织相帧被累积以提供参考图像，所以参考图像可能因帧间运动而模糊，并且可能导致不准确的运动估计。

深度学习已经在许多医学成像任务中展现出了其良好的性能，包括图像增强、图像配准、图像分割、图像生成和计算机辅助诊断。然而，在序列图像上使用深度

Abstract

摘要

Patient motion during dynamic PET imaging can induce errors in myocardial blood flow (MBF) estimation. Motion correction for dynamic cardiac PET is challenging because the rapid tracer kinetics of 82Rb leads to substantial tracer distribution change across different dynamic frames over time, which can cause difficulties for image registration-based motion correction, particularly for early dynamic frames. In this paper, we developed an automatic deep learning-based motion correction (DeepMC) method for dynamic cardiac PET. In this study we focused on the detection and correction of inter-frame rigid translational motion caused by voluntary body movement and pattern change of respiratory motion. A bidirectional-3D LSTM network was developed to fully utilize both local and nonlocal temporal information in the 4D dynamic image data for motion detection. The network was trained and evaluated over motion-free patient scans with simulated motion so that the motion ground-truths are available, where one million samples based on 65 patient scans were used in training, and 600 samples based on 20 patient scans were used in evaluation. The proposed method was also evaluated using additional 10 patient datasets with real motion. We demonstrated that the proposed DeepMC obtained superior performance compared to conventional registration-based methods and other convolutional neural networks (CNN), in terms of motion estimation and MBF quantification accuracy. Once trained, DeepMC is much faster than the registration-based methods and can be easily integrated into the clinical workflow. In the future work, additional investigation is needed to evaluate this approach in a clinical context with realistic patient motion.

动态PET成像中的患者运动可能会导致心肌血流（MBF）估计中的错误。动态心脏PET的运动校正具有挑战性，因为82Rb的快速示踪剂动力学导致不同动态帧之间示踪剂分布的显著变化，这可能会导致基于图像配准的运动校正出现困难，特别是对于早期动态帧。在本文中，我们开发了一种基于深度学习的自动运动校正（DeepMC）方法，用于动态心脏PET。在这项研究中，我们专注于检测和校正由自主体运动和呼吸运动模式变化引起的帧间刚性平移运动。我们开发了一个双向3D LSTM网络，充分利用了4D动态图像数据中的本地和非本地时间信息进行运动检测。该网络在无运动的患者扫描中进行了训练和评估，以便获得运动的基本真相，其中训练使用了基于65个患者扫描的100万个样本，评估使用了基于20个患者扫描的600个样本。提出的方法还使用了额外的10个带有真实运动的患者数据集进行评估。我们证明了所提出的DeepMC在运动估计和MBF量化准确性方面相对于传统的基于配准的方法和其他卷积神经网络（CNN）具有优越的性能。一旦训练完成，DeepMC比基于配准的方法快得多，并且可以轻松集成到临床工作流程中。在未来的工作中，需要进一步研究以评估该方法在具有真实患者运动的临床环境中的应用情况。

Method

方法

A. Dataset

A total of 160 anonymized clinical 82Rb PET rest and regadenoson-induced stress studies were included from Yale New Haven Hospital from December 2019 to January 2020. The PET data were acquired using a Discovery 690 PET/CT scanner (GE Healthcare, Waukesha, WI). The data anonymization for this research was approved by the Institutional Review Board of Yale University. 82Rb was delivered via a programed infusion using a commercial 82Rb generator (Bracco Diagnostics Inc.) with a weight-based targeted injection dose of 20-35 mCi depending on patient body mass index. The total acquisition time for each scan was 7 min, but only the first 6 min 10 s data were included in the rebinning process according to the clinical setting. Listmode data were rebinned into dynamic image sequence of 14×5s, 6×10s, 3×20s,3×30s, and 1×90s timeframes. Images were reconstructed using OSEM with 2 iterations and 24 subsets, resulting in 128× 128× 47 voxels of size 3.125× 3.125× 3.270 mm. Images were filtered with Butterworth filter with a cutoff frequency of 21 mm-1 and an order of 5. Corrections for isotope decay, photon attenuation and scatter, random and prompt-gamma coincidences, detector efficiency, and deadtime were all applied to reconstruct quantitative images of activity concentration (Bq/mL), according to our standard clinical practice. Note that each dynamic frame was reconstructed independently. Therefore, the scatter estimation was derived from each individual frame’s emission data, instead of from an initial static reconstruction. CT and 82Rb PET image were manually registered using the vendor ACQC package.

A. 数据集

从2019年12月到2020年1月，从耶鲁大学纽黑文医院收集了共计160个匿名的临床82Rb PET静息和雷加诺唑诱导的应激研究。PET数据是使用Discovery 690 PET/CT扫描仪（GE Healthcare，Waukesha，WI）获取的。本研究的数据匿名化经过耶鲁大学的机构审查委员会批准。82Rb是通过使用商业82Rb发生器（Bracco Diagnostics Inc.）进行程序化注入获得的，注射剂量根据患者体重指数为20-35 mCi。每次扫描的总采集时间为7分钟，但根据临床设置，只有前6分钟10秒的数据包括在重新分箱过程中。Listmode数据被重新分箱成14×5秒、6×10秒、3×20秒、3×30秒和1×90秒的动态图像序列。图像使用OSEM进行重建，迭代次数为2次，子集数为24个，结果为128×128×47个大小为3.125×3.125×3.270 mm的体素。图像经过Butterworth滤波器滤波，截止频率为21 mm-1，阶数为5。对同位素衰变、光子衰减和散射、随机和提示-γ 符合、探测器效率和死时间进行了校正，以重建定量活度浓度图像（Bq/mL），根据我们的标准临床实践。需要注意的是，每个动态帧都是独立重建的。因此，散射估计是根据每个单独帧的发射数据得出的，而不是从初始静态重建中得出的。CT和82Rb PET图像是使用供应商ACQC软件包手动注册的。

Conclusion

结论

This work presents an automatic motion correction framework for dynamic cardiac PET using deep learning. The proposedLSTM-based network can detect and correct inter-frame rigid motion for both early frames with fast tracer distribution change and later frames with slower tracer kinetics. Evaluation with patient data demonstrated the effectiveness of motion correction and improved accuracy of MBF quantification.

这项工作介绍了一种利用深度学习的自动动态心脏PET运动校正框架。提出的基于LSTM的网络能够检测和纠正早期帧和后期帧中的刚性帧间运动，前者具有快速示踪剂分布变化，后者具有较慢的示踪剂动力学。通过患者数据的评估表明，这种运动校正方法的有效性以及MBF定量的准确性得到了改善。

Figure

图

Fig. 1. An illustration of the three types of motion in simulation for network training. This illustration only shows 1-D motion displacement patterns with the motion magnitudes changing over frames, whereas the motion directions are not reflected in this figure.

图1. 模拟网络训练中三种类型运动的示意图。这个示意图只显示了一维运动位移模式，运动幅度随着帧数变化，而运动方向在这个图中没有反映出来。

Fig. 2. (a) The architecture of the motion prediction and EQ frame prediction network. The input frame images are only displayed as 2D images, although they are actually 3D images, with dual-channels (red) for motion prediction network and single-channel (purple) for EQ frame prediction network. The motion prediction network and the EQ frame prediction network share the same network structure except for the last fully connected (FC) layer. They are trained separately. (b) A diagram of the entire motion prediction workflow.

图2. (a) 运动预测和EQ帧预测网络的架构。输入的帧图像只显示为2D图像，尽管它们实际上是3D图像，具有用于运动预测网络的双通道（红色）和用于EQ帧预测网络的单通道（紫色）。运动预测网络和EQ帧预测网络除了最后一个全连接（FC）层之外，它们共享相同的网络结构。它们被分开训练。（b）整个运动预测工作流程的图表。

Fig. 3. An example of the early frames and late frames after intensity and temporal normalization. The whole sequence was shifted towards the early phase by three frames, therefore the first three frames with zero activity were discarded, and the last frame was duplicated three times at the end of the late frames (purple). The last frame was also put at the end of the early frames to provide a reference (red). All the images are shown in the coronal view. The frame number after temporal normalization was shown for each frame. The EQ frame was pointed out in blue color.

图3. 早期帧和晚期帧经过强度和时间归一化的示例。整个序列向早期相移了三帧，因此丢弃了前三帧的零活动，并且将最后一帧在晚期帧末尾重复三次（紫色）。最后一帧也放置在早期帧的末尾以提供参考（红色）。所有图像均以冠状视图显示。每帧经过时间归一化后的帧号也显示在每帧旁边。EQ 帧用蓝色指出。

Fig. 4. The distribution of the motion magnitude (mm) and the frame index of the motion-affected frames for the 600 image sequences in the evaluation.

图4. 在评估中，600个图像序列中受运动影响的帧的运动幅度（毫米）和帧索引的分布

Fig. 5. Two examples (selected from the 600 evaluation cases) of the image frames with simulated motion (No MC) and after motion correction (DeepMC). The two examples correspond to two different patients. The images were displayed after intensity normalization. The top example shows the early frames with rectangle motion (motion magnitude 16.1mm), and the bottom example shows the late frames with triangle motion (peak motion magnitude 12.5mm). In both examples, the LV myocardium and LV blood pool ROIs drawn on the last frame are displayed on each image frame.

图5. 两个示例（从600个评估案例中选取），显示模拟运动（无运动校正）前后的图像帧（DeepMC进行了运动校正）。这两个示例对应两个不同的患者。图像显示经过强度归一化处理。顶部示例显示具有矩形运动（运动幅度16.1毫米）的早期帧，底部示例显示具有三角形运动（峰值运动幅度12.5毫米）的晚期帧。在两个示例中，左心室心肌和左心室血池ROI在每个图像帧上都显示出来。

Fig. 6. The average motion estimation error (mm) by frame for the different methods on the 600 evaluation image sequences. The first two data points of MC-IR with large motion errors are shown in a zoomed region for better illustration.

图6. 不同方法在600个评估图像序列上按帧计算的平均运动估计误差（毫米）。为了更好地说明，MC-IR的前两个数据点的大运动误差显示在了一个放大区域内。

Fig. 7. The scatter plots of the MBF estimation between the motion-corrected and motion-free groups. Each one of the parallel clusters (20 in total) represents the results from the 30 motion replicates derived from each of the 20 motionfree patient scans.

图7. MBF估计的散点图，显示了运动校正组和无运动组之间的差异。每个平行的聚类（共20个）代表了来自每个20个无运动患者扫描中的30个运动复制品的结果。

Fig. 8. The Bland-Altman plots of the MBF estimation between the motioncorrected and motion-free groups. Each one of the parallel clusters (20 in total) represents the results from the 30 motion replicates derived from each of the 20 motion-free patient scans. Note the scale difference of the y-axis for differentgroups.

图8. MBF估计的 Bland-Altman 图，显示了运动校正组和无运动组之间的差异。每个平行的聚类（共20个）代表了来自每个20个无运动患者扫描中的30个运动复制品的结果。请注意不同组之间y轴的刻度差异。

Fig. 9. Examples of the image frames from three subjects before and after motion correction selected from three patient scans. The LV myocardium and blood pool ROIs drawn on the last frame were overlaid with the images (the blood pool ROI was not shown on the selected slice for Subject C). The white arrows pointed out the mismatches caused by motion when no motion correction was applied (noMC). The green arrows pointed out the residual mismatches after motion correction in some of the results. The motion magnitudes detected by different motion correction methods were also shown beneath the images.

图9. 选自三个患者扫描的图像帧示例，在进行运动校正前后的情况。在最后一帧上绘制了左心室心肌和血池ROI，并叠加在图像上（对于C患者的选择切片未显示血池ROI）。白色箭头指出了在未应用运动校正时（noMC）由运动引起的不匹配情况。绿色箭头指出了部分结果在运动校正后仍存在的残余不匹配情况。图像下方还显示了不同运动校正方法检测到的运动幅度。

Fig. 10. Scatter plots of the percentage difference between the MBFs measured on the images before and after motion correction using DeepMC.

图10. 使用DeepMC进行运动校正前后测得的MBF之间的百分比差异的散点图。

Table

表

Table I. A summary of the characteristics of the included scans.

表格 I. 包含扫描的特征总结。

Table II. Ablation study results in terms of mean and max motion estimation errors across all the 600 evaluation samples. MC: motion correction; IN: intensity normalization; TN: temporal normalization; IMC: iterative motion correction.

表格 II. 在所有600个评估样本中，基于平均和最大运动估计误差的消融研究结果。MC：运动校正；IN：强度归一化；TN：时间归一化；IMC：迭代运动校正。

Table III. A comprehensive summary of the evaluation results. For MBF bias, indicates the results’ mean is significantly different from zero based on paired t-test (5% significance level). Bold font indicates the best performer in each group.

表格 III. 评估结果的综合总结。对于MBF偏差， 表示结果的平均值在配对 t 检验（5% 显著性水平）中与零显著不同。粗体字表示每个组中的最佳表现者。

able IV. Summary of the MBFs and fitting residuals measured on the 10 cases with real motion for different motion correction methods. The results measured on both the whole LV myocardium (Global) and the RCA ROIs were reported.

表格 IV. 不同运动校正方法测得的10个真实运动案例的MBF和拟合残差总结。报告了对整个左心室心肌（全局）和右冠状动脉（RCA）ROIs测得的结果。