Background: During X-ray imaging, pulmonary movements can cause many image artifacts. To tackle this issue, several studies, including mathematical algorithms and 2D-3D image registration methods, have been presented. Recently, the application of deep artificial neural networks has been considered for image generation and prediction. Objective: In this study, a convolutional long short-term memory (ConvLSTM) neural network is used to predict spatiotemporal 4DCT images. Material and Methods: In this analytical analysis study, two ConvLSTM structures, consisting of stacked ConvLSTM models along with the hyperparameter optimizer algorithm and a new design of the ConvLSTM model are proposed. The hyperparameter optimizer algorithm in the conventional ConvLSTM includes the number of layers, number of filters, kernel size, epoch number, optimizer, and learning rate. The two ConvLSTM structures were also evaluated through six experiments based on Root Mean Square Error (RMSE) and structural similarity index (SSIM). Results: Comparing the two networks demonstrates that the new design of the ConvLSTM network is faster, more accurate, and more reliable in comparison to the tuned-stacked ConvLSTM model. For all patients, the estimated RMSE and SSIM were 3.17 and 0.988, respectively, and a significant improvement can be observed in comparison to the previous studies. Conclusion: Overall, the results of the new design of the ConvLSTM network show excellent performances in terms of RMSE and SSIM. Also, the generated CT images with the new design of the ConvLSTM model show a good consistency with the corresponding references regarding registration accuracy and robustness. |
- De Vos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M, Išgum I. A deep learning framework for unsupervised affine and deformable image registration. Med Image Anal. 2019;52:128-43. doi: 10.1016/j.media.2018.11.010. PubMed PMID: 30579222.
- Oh S, Kim S. Deformable image registration in radiation therapy. Radiat Oncol J. 2017;35(2):101-11. doi: 10.3857/roj.2017.00325. PubMed PMID: 28712282. PubMed PMCID: PMC5518453.
- Rigaud B, Simon A, Castelli J, Lafond C, Acosta O, Haigron P, Cazoulat G, et al. Deformable image registration for radiation therapy: principle, methods, applications and evaluation. Acta Oncol. 2019;58(9):1225-37. doi: 10.1080/0284186X.2019.1620331. PubMed PMID: 31155990.
- Samadi Miandoab P, Esmaili Torshabi A, Parandeh S. Calculation of inter-and intra-fraction motion errors at external radiotherapy using a markerless strategy based on image registration combined with correlation model. Iranian Journal of Medical Physics. 2019;16(3):224-31. doi: 10.22038/ IJMP.2018.30477.1348.
- Watkins WT, Li R, Lewis J, Park JC, Sandhu A, Jiang SB, et al. Patient-specific motion artifacts in 4DCT. Med Phys. 2010;37(6):2855-61. doi: 10.1118/1.3432615. PubMed PMID: 20632597.
- Kwong Y, Mel AO, Wheeler G, Troupis JM. Four-dimensional computed tomography (4DCT): A review of the current status and applications. J Med Imaging Radiat Oncol. 2015;59(5):545-54. doi: 10.1111/1754-9485.12326. PubMed PMID: 26041442.
- Keall PJ, Vedam SS, George R, Williamson JF. Respiratory regularity gated 4D CT acquisition: concepts and proof of principle. Australas Phys Eng Sci Med. 2007;30(3):211-20. doi: 10.1007/ BF03178428. PubMed PMID: 18044305.
- Sarrut D, Clippe S. Fast DRR generation for intensity-based 2D/3D image registration in radiotherapy. LIRIS UMR. 2003;5205.
- Xie Y, Xing L, Gu J, Liu W. Tissue feature-based intra-fractional motion tracking for stereoscopic x-ray image guided radiotherapy. Phys Med Biol. 2013;58(11):3615-30. doi: 10.1088/0031- 9155/58/11/3615. PubMed PMID: 23648334.
- Miandoab PS, Torshabi AE, Nankali S. Extraction of respiratory signal based on image clustering and intensity parameters at radiotherapy with external beam: A comparative study. J Biomed Phys Eng. 2016;6(4):253-64. PubMed PMID: 28144595. PubMed PMCID: PMCPMC5219576.
- Kai J, Fujii F, Shiinoki T. Prediction of Lung Tumor Motion Based on Recurrent Neural Network. International Conference on Mechatronics and Automation (ICMA); Changchun, China: IEEE; 2018. p. 1093-9. doi: 10.1109/ICMA.2018.8484575.
- Nabavi S, Abdoos M, Moghaddam ME, Mohammadi M. Respiratory Motion Prediction Using Deep Convolutional Long Short-Term Memory Network. J Med Signals Sens. 2020;10(2):69-75. doi: 10.4103/jmss.JMSS_38_19. PubMed PMID: 32676442. PubMed PMCID: PMC7359959.
- Van De Leemput SC, Prokop M, Van Ginneken B, Manniesing R. Stacked Bidirectional Convolutional LSTMs for Deriving 3D Non-Contrast CT From Spatiotemporal 4D CT. IEEE Trans Med Imaging. 2020;39(4):985-96. doi: 10.1109/ TMI.2019.2939044. PubMed PMID: 31484111.
- Fu Y, Lei Y, Wang T, Higgins K, Bradley JD, Curran WJ, et al. LungRegNet: An unsupervised deformable image registration method for 4D-CT lung. Med Phys. 2020;47(4):1763-74. doi: 10.1002/ mp.14065. PubMed PMID: 32017141. PubMed PMCID: PMC7165051.
- Lee SW, Kim HY. Stock market forecasting with super-high dimensional time-series data using ConvLSTM, trend sampling, and specialized data augmentation. Expert Systems with Applications. 2020;161:113704. doi: 10.1016/j. eswa.2020.113704. X.
- Yuan Z, Zhou X, Yang T. Hetero-convlstm: A deep learning approach to traffic accident prediction on heterogeneous spatio-temporal data. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; New York: Association for Computing Machinery; 2018. p. 84-92. doi: 10.1145/3219819.3219922.
- Ray A, Rajeswar S, Chaudhury S. Text recognition using deep BLSTM networks. International conference on advances in pattern recognition (ICAPR); Kolkata, India: IEEE; 2015. doi: 10.1109/ ICAPR.2015.7050699.
- Balci B, Saadati D, Shiferaw D. Handwritten text recognition using deep learning. CS23 1n: Convolutional Neural Networks for Visual Recognition; Stanford University: Spring; 2017. p. 752-9.
- Kim S, Hong S, Joh M, Song SK. Deeprain: Convlstm network for precipitation prediction using multichannel radar data [Internet]. arXiv [Preprint]. 2017 [cited 2017 November 7]. Available from: https://arxiv.org/abs/1711.02316.
- Salman AG, Heryadi Y, Abdurahman E, Suparta W. Single Layer & Multi-layer Long Short-Term Memory (LSTM) Model with Intermediate Variables for Weather Forecasting. Procedia Computer Science. 2018;135:89-98. doi: 10.1016/j. procs.2018.08.153.
- Zhang K, Geng X, Yan XH. Prediction of 3-D ocean temperature by multilayer convolutional LSTM. IEEE Geoscience and Remote Sensing Letters. 2020;17(8):1303-7. doi: 10.1109/ lgrs.2019.2947170.
- Dehkordi AN, Sina S, Khodadadi F. A Comparison of Deep Learning and Pharmacokinetic Model Selection Methods in Segmentation of High-Grade Glioma. Frontiers in Biomedical Technologies. 2021;8(1);50-60. doi: 10.18502/fbt.v8i1.5858.
- Zapata-Impata BS, Gil P, Torres F. Learning Spatio Temporal Tactile Features with a ConvLSTM for the Direction Of Slip Detection. Sensors (Basel). 2019;19(3):523. doi: 10.3390/s19030523. PubMed PMID: 30691197. PubMed PMCID: PMC6387284.
- Wang D, Yang Y, Ning S. Deepstcl: A deep spatio-temporal convlstm for travel demand prediction. International joint conference on neural networks (IJCNN); Brazil: IEEE; 2018.
- Vandemeulebroucke J, Sarrut D, Clarysse P. The POPI-model, a point-validated pixel-based breathing thorax model. XVth international conference on the use of computers in radiation therapy; Toronto, Ontario, Canada: ICCR; 2007. p. 195-99.
- Xingjian SH, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC. Convolutional LSTM network: A machine learning approach for precipitation nowcasting [Internet]. arXiv [Preprint]. 2015 [cited 2015 June 13]. Available from: https://arxiv.org/abs/1506.04214.
- Yu T, Zhu H. Hyper-parameter optimization: A review of algorithms and applications. ArXiv:200305689.
- Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13(4):600-12. doi: 10.1109/tip.2003.819861. PubMed PMID: 15376593.
- Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. 14th international joint conference on Artificial intelligence; Montreal, Canada: Ijcai; 1995. p. 1137- 43.
- Purdie TG, Moseley DJ, Bissonnette JP, et al. Respiration correlated cone-beam computed tomography and 4DCT for evaluating target motion in Stereotactic Lung Radiation Therapy. Acta Oncol. 2006;45(7):915-22. doi: 10.1080/02841860600907345. PubMed PMID: 16982558.
- Samadi-Miyandoab P, Torshabi A, Nankali S. 2D and 3D Optical Flow Based Interpolation of the 4DCT ImageSequences in the External Beam Radiotherapy. Frontiers in Biomedical Technologies. 2015;2(2):93-102.
|