Diabetic Foot Ulcer Detection: Combining Deep Learning Models for Improved Localization

Diabetic Foot Ulcer Detection: Combining Deep Learning Models for Improved Localization Rusab Sarmun Muhammad E. H. Chowdhury http://orcid.org/0000-0003-0744-8206 M. Murugappan Ahmed Aqel Maymouna Ezzuddin Syed Mahfuzur Rahman Amith Khandakar Sanzida Akter Rashad Alfkey Anwarul Hasan Abstract

Diabetes mellitus (DM) can cause chronic foot issues and severe infections, including Diabetic Foot Ulcers (DFUs) that heal slowly due to insufficient blood flow. A recurrence of these ulcers can lead to 84% of lower limb amputations and even cause death. High-risk diabetes patients require expensive medications, regular check-ups, and proper personal hygiene to prevent DFUs, which affect 15–25% of diabetics. Accurate diagnosis, appropriate care, and prompt response can prevent amputations and fatalities through early and reliable DFU detection from image analysis. We propose a comprehensive deep learning-based system for detecting DFUs from patients’ feet images by reliably localizing ulcer points. Our method utilizes innovative model ensemble techniques—non-maximum suppression (NMS), Soft-NMS, and weighted bounding box fusion (WBF)—to combine predictions from state-of-the-art object detection models. The performances of diverse cutting-edge model architectures used in this study complement each other, leading to more generalized and improved results when combined in an ensemble. Our WBF-based approach combining YOLOv8m and FRCNN-ResNet101 achieves a mean average precision (mAP) score of 86.4% at the IoU threshold of 0.5 on the DFUC2020 dataset, significantly outperforming the former benchmark by 12.4%. We also perform external validation on the IEEE DataPort Diabetic Foot dataset which has demonstrated robust and reliable model performance on the qualitative analysis. In conclusion, our study effectively developed an innovative diabetic foot ulcer (DFU) detection system using an ensemble model of deep neural networks (DNNs). This AI-driven tool serves as an initial screening aid for medical professionals, augmenting the diagnostic process by enhancing sensitivity to potential DFU cases. While recognizing the presence of false positives, our research contributes to improving patient care through the integration of human medical expertise with AI-based solutions in DFU management.
04 01 2024 05 2024 1413 1431 10267 1 10.1007/springer_crossmark_policy link.springer.com false 29 March 2023 3 March 2024 1 April 2024 20 April 2024 Update The original version of this article was updated to correct the author name of the last author. This study is conducted on two publicly accessible datasets. The main training dataset is made available by Prof. Moi Hoon Yap as Diabetic Foot Ulcer Challenge 2020 (DFUC2020) Dataset while external validation set was available from IEEE DataPort. Therefore, ethical approval is not applicable for this study. As this study uses two publicly available datasets, informed consent is not applicable for this study. The authors declare no competing interests. Qatar National Research Fund http://dx.doi.org/10.13039/100008982 NPRP12S-0227-190164 International Research Collaboration Co-Fund (IRCC) grant IRCC-2021-001 Qatar University http://dx.doi.org/10.13039/501100004252 https://creativecommons.org/licenses/by/4.0 https://creativecommons.org/licenses/by/4.0 10.1007/s12559-024-10267-3 20240601110136782 https://link.springer.com/10.1007/s12559-024-10267-3 https://link.springer.com/content/pdf/10.1007/s12559-024-10267-3.pdf https://link.springer.com/content/pdf/10.1007/s12559-024-10267-3.pdf https://link.springer.com/article/10.1007/s12559-024-10267-3/fulltext.html Journal of Paramedic Practice R Higginson 11 6 1 2019 10.12968/jpar.2019.11.6.CPD1 Higginson R, Burrows P, Jones B. Continuing professional development: diabetes and associated diabetic emergencies. Journal of Paramedic Practice. 2019;11(6):1–5. Diabetes Res Clin Pract P Saeedi 157 2019 10.1016/j.diabres.2019.107843 Saeedi P, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: results from the International Diabetes Federation Diabetes Atlas. Diabetes Res Clin Pract. 2019;157: 107843. Diabetes Metab Syndr S Noor 9 3 192 2015 10.1016/j.dsx.2015.04.007 Noor S, Zubair M, Ahmad J. Diabetic foot ulcer—a review on pathophysiology, classification and microbial etiology. Diabetes Metab Syndr. 2015;9(3):192–9. J Diabetes Sci Technol AL Killeen 14 1 37 2020 10.1177/1932296819848769 Killeen AL, Brock KM, Dancho JF, Walters JL. Remote temperature monitoring in patients with visual impairment due to diabetes mellitus: a proposed improvement to current standard of care for prevention of diabetic foot ulcers. J Diabetes Sci Technol. 2020;14(1):37–45. J Med Internet Res AM Reyzelman 20 12 2018 10.2196/12460 Reyzelman AM, et al. Continuous temperature-monitoring socks for home use in patients with diabetes: observational study. J Med Internet Res. 2018;20(12): e12460. N Engl J Med DG Armstrong 376 24 2367 2017 10.1056/NEJMra1615439 Armstrong DG, Boulton AJ, Bus SA. Diabetic foot ulcers and their recurrence. N Engl J Med. 2017;376(24):2367–75. Wound Repair and Regeneration CE Ananian 26 3 274 2018 10.1111/wrr.12645 Ananian CE, et al. A multicenter, randomized, single-blind trial comparing the efficacy of viable cryopreserved placental membrane to human fibroblast-derived dermal substitute for the treatment of chronic diabetic foot ulcers. Wound Repair and Regeneration. 2018;26(3):274–83. Foot Ankle Int D Boutoille 29 11 1074 2008 10.3113/FAI.2008.1074 Boutoille D, Féraille A, Maulaz D, Krempf M. Quality of life with diabetes-associated foot complications: comparison between lower-limb amputation and chronic foot ulceration. Foot Ankle Int. 2008;29(11):1074–8. Diabetes Metab Res Rev P Cavanagh 28 107 2012 10.1002/dmrr.2245 Cavanagh P, Attinger C, Abbas Z, Bal A, Rojas N, Xu ZR. Cost of treating diabetic foot ulcers in five different countries. Diabetes Metab Res Rev. 2012;28:107–11. 10.1007/978-3-030-94907-5_1 Yap MH, Kendrick C, Reeves ND, Goyal M, Pappachan JM, Cassidy B. “Development of diabetic foot ulcer datasets: an overview,” Diabetic Foot Ulcers Grand Challenge: Second Challenge, DFUC 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings. 2022:1–18. J Diabetes Sci Technol MH Yap 12 1 169 2018 10.1177/1932296817713761 Yap MH, et al. A new mobile application for standardizing diabetic foot images. J Diabetes Sci Technol. 2018;12(1):169–73. J Diabetes Sci Technol MH Yap 10 2 612 2016 10.1177/1932296815611425 Yap MH, et al. Computer vision algorithms in the detection of diabetic foot ulceration: a new paradigm for diabetic foot care? J Diabetes Sci Technol. 2016;10(2):612–3. 10.1109/SMC.2017.8122675 Goyal M, Yap MH, Reeves ND, Rajbhandari S, Spragg J. “Fully convolutional networks for diabetic foot ulcer segmentation,” in 2017 IEEE int conf syst man cybern (SMC), 2017:618–623. Infrared Phys Technol D Hernandez-Contreras 73 149 2015 10.1016/j.infrared.2015.09.022 Hernandez-Contreras D, Peregrina-Barreto H, Rangel-Magdaleno J, Ramirez-Cortes J, Renero-Carrillo F. Automatic classification of thermal patterns in diabetic foot based on morphological pattern spectrum. Infrared Phys Technol. 2015;73:149–57. Sensors A Khandakar 22 5 1793 2022 10.3390/s22051793 Khandakar A, et al. Thermal change index-based diabetic foot thermogram image classification using machine learning techniques. Sensors. 2022;22(5):1793. Comput Biol Med A Khandakar 137 2021 10.1016/j.compbiomed.2021.104838 Khandakar A, et al. A machine learning model for early detection of diabetic foot using thermogram images. Comput Biol Med. 2021;137: 104838. ICT Express SK Das 8 2 271 2022 10.1016/j.icte.2021.08.022 Das SK, Roy P, Mishra AK. DFU_SPNet: a stacked parallel convolution layers based CNN to improve Diabetic Foot Ulcer classification. ICT Express. 2022;8(2):271–5. Multimedia Tools and Applications L Alzubaidi 79 21 15655 2020 10.1007/s11042-019-07820-w Alzubaidi L, Fadhel MA, Oleiwi SR, Al-Shamma O, Zhang J. DFU_QUTNet: diabetic foot ulcer classification using novel deep convolutional neural network. Multimedia Tools and Applications. 2020;79(21):15655–77. IEEE Trans Biomed Eng L Wang 64 9 2098 2016 10.1109/TBME.2016.2632522 Wang L, Pedersen PC, Agu E, Strong DM, Tulu B. Area determination of diabetic foot ulcer images using a cascaded two-stage SVM-based classification. IEEE Trans Biomed Eng. 2016;64(9):2098–109. IEEE Transactions on Emerging Topics in Computational Intelligence M Goyal 4 5 728 2018 10.1109/TETCI.2018.2866254 Goyal M, Reeves ND, Davison AK, Rajbhandari S, Spragg J, Yap MH. Dfunet: convolutional neural networks for diabetic foot ulcer classification. IEEE Transactions on Emerging Topics in Computational Intelligence. 2018;4(5):728–39. Simonyan K, Zisserman A, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint. 2014. arXiv:1409.1556. 10.1109/CVPR.2015.7298594 Szegedy C. et al., “Going deeper with convolutions,” in Proceedings of the IEEE conf comp vis pattern recognition. 2015:1-9. 10.1016/j.sintl.2022.100210 Thotad PN, Bharamagoudar GR, Anami BS, “Diabetic foot ulcer detection using deep learning approaches,” Sens Int. 2022:100210. Howard AG et al., “Mobilenets: efficient convolutional neural networks for mobile vision applications” arXiv preprint. 2017. arXiv:1704.04861. Dai J, Li Y, He K, Sun J. “R-fcn: object detection via region-based fully convolutional networks,” Adv neural inf process systems. 2016;29. IEEE J Biomed Health Inform M Goyal 23 4 1730 2018 10.1109/JBHI.2018.2868656 Goyal M, Reeves ND, Rajbhandari S, Yap MH. Robust methods for real-time diabetic foot ulcer detection and localization on mobile devices. IEEE J Biomed Health Inform. 2018;23(4):1730–41. 10.1109/CVPR.2019.00953 Zhu X, Hu H, Lin S, Dai J. “Deformable convnets v2: more deformable, better results,” in Proceedings of the IEEE/CVF conf comp vis pattern recog, 2019, pp. 9308–9316. Comput Biol Med MH Yap 135 2021 10.1016/j.compbiomed.2021.104596 Yap MH, et al. Deep learning in diabetic foot ulcers detection: a comprehensive evaluation. Comput Biol Med. 2021;135: 104596. Rogers LC, Lavery LA, Joseph WS, Armstrong DG. “All feet on deck—the role of podiatry during the COVID-19 pandemic: preventing hospitalizations in an overburdened healthcare system, reducing amputation and death in people with diabetes,” J Am Podiatr Med Assoc 2020;1(aop):0000–0000. Wounds a compend clin res pract LC Rogers 32 7 178 2020 Rogers LC, et al. Wound center without walls: the new model of providing care during the COVID-19 pandemic. Wounds a compend clin res pract. 2020;32(7):178. 10.1109/CVPRW50498.2020.00203 Wang C-Y, Liao H-YM, Wu Y-H, Chen P-Y, Hsieh J-W, Yeh I-H. “CSPNet: a new backbone that can enhance learning capability of CNN,” in Proceedings of the IEEE/CVF conf comp vis pattern recognit workshops, 2020, pp. 390–391. 10.1109/ICCV.2019.00929 Wang K, Liew JH, Zou Y, Zhou D, Feng J. “Panet: Few-shot image semantic segmentation with prototype alignment,” in Proceedings of the IEEE/CVF Int Conf Comp Vis 2019, pp. 9197–9206. Redmon J, Farhadi A. “Yolov3: an incremental improvement,” arXiv preprint 2018. arXiv:1804.02767 IEEE Access T Rahman 10 53359 2022 10.1109/ACCESS.2022.3173424 Rahman T, et al. HipXNet: deep learning approaches to detect aseptic loos-ening of hip implants using X-ray images. IEEE Access. 2022;10:53359–73. 10.1109/CVPR52729.2023.00721 Wang C-Y, Bochkovskiy A, Liao H-YM. “YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proceedings of the IEEE/CVF Conf Comp Vis Pattern Recognit 2023, pp. 7464–7475. Sensors M Hussain 22 18 6927 2022 10.3390/s22186927 Hussain M, Al-Aqrabi H, Munawar M, Hill R, Alsboui T. Domain feature mapping with YOLOv7 for automated edge-based pallet racking inspections. Sensors. 2022;22(18):6927. Appl Sci W Zhu 12 23 11959 2022 10.3390/app122311959 Zhu W, et al. CPAM: cross patch attention module for complex texture tile block defect detection. Appl Sci. 2022;12(23):11959. Glenn J, Ayush C, Jing Q. YOLO by Ultralytics (version 8.0.0). https://github.com/ultralytics/ultralytics. Accessed 14 Feb 2023. T-Y Lin 740 2014 Euro conf comp vis Lin T-Y, et al. Microsoft coco: common objects in context. In: Euro conf comp vis. Springer; 2014. p. 740–55. Terven J, Cordova-Esparza D. “A comprehensive review of YOLO: From YOLOv1 and beyond. arXiv 2023,” arXiv preprint arXiv:2304.00501, 2023. Jacob Solawetz F. “What is YOLOv8? The ultimate guide.” https://blog.roboflow.com/whats-new-in-yolov8/. Accessed 14 Feb 2023. Korean J Radiol Q-Q Zhou 21 7 869 2020 10.3348/kjr.2019.0651 Zhou Q-Q, et al. Automatic detection and classification of rib fractures on thoracic CT using convolutional neural network: accuracy and feasibility. Korean J Radiol. 2020;21(7):869. Ren S, He K, Girshick R, Sun J. “Faster r-cnn: towards real-time object detection with region proposal networks,” Adv neural inform process syst 2015;vol. 28. K He 770 2016 Proceedings of the IEEE conf comp vis pattern recognit He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conf comp vis pattern recognit. 2016. p. 770–778. M Tan 10781 2020 Proceedings of the IEEE/CVF con comp vis pattern recognit Tan M, Pang R, Le QV. Efficientdet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF con comp vis pattern recognit. 2020. p. 10781–10790. TY Lin 2117 2017 Proceedings of the IEEE con comp vis pattern recognit Lin TY, Dollár P, Girshick R, He R, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE con comp vis pattern recognit. 2017. p. 2117–2125. touch REV Endocrinology B Cassidy 17 1 5 2021 10.17925/EE.2021.17.1.5 Cassidy B, et al. The DFUC 2020 dataset: analysis towards diabetic foot ulcer detection. touch REV Endocrinology. 2021;17(1):5. Zhu H. Diabetic foot. In: Zhu HE, editor. IEEE DataPort: IEEE DataPort 2020. Accessed 14 Feb 2023. R Padilla 237 2020 int con syst sig image process (IWSSIP),IEEE Padilla R, Netto SL, Da Silva EA. A survey on performance metrics for object-detection algorithms. In: int con syst sig image process (IWSSIP),IEEE. 2020. p. 237–242. 10.1109/ICCV.2017.593 Bodla N, Singh B, Chellappa R, Davis LS. “Soft-NMS--improving object detection with one line of code,” in Proceedings of the IEEE int con comp vis 2017:pp. 5561–5569. Image Vis Comput R Solovyev 107 2021 10.1016/j.imavis.2021.104117 Solovyev R, Wang W, Gabruseva T. Weighted boxes fusion: ensembling boxes from different object detection models. Image Vis Comput. 2021;107: 104117. Diabetic Foot Ulcer Challenge 2020. https://dfu2020.grand-challenge.org/evaluation/challenge/leaderboard/. Accessed 18 Feb 2023. Reis D, Kupec J, Hong J, Daoudi A. “Real-time flying object detection with YOLOv8,” 2023. arXiv preprint arXiv:2305.09972. J Big data S Srivastava 8 1 1 2021 10.1186/s40537-021-00434-w Srivastava S, Divekar AV, Anilkumar C, Naik I, Kulkarni V, Pattabiraman V. Comparative analysis of deep learning image detection algorithms. J Big data. 2021;8(1):1–27. 10.1109/ICCE-Asia49877.2020.9277040 Kim J-A, Sung J-Y, Park S-H. “Comparison of Faster-RCNN, YOLO, and SSD for real-time vehicle type recognition,” in 2020 IEEE int conf consum elec Asia (ICCE-Asia) 2020;IEEE:pp. 1–4.