Risk-Aware Hierarchical Deep Learning Framework for Clinically-Aligned Skin Lesion Classification
Abstract
Effective clinical intervention for malignant skin lesions requires accurately and expeditiously identifying these malignancies; however, many current AI/ML-based solutions treat the diagnosis of skin diseases as a flat multi-class classification problem, and therefore fail to take into account clinical decision-making priorities and the risk of malignancy. To address this shortcoming, a clinically aligned hierarchical deep learning algorithm was presented that separates the diagnosis of skin diseases into the screening for malignancy and the specific classification of a dermatological disease. Specifically, the algorithm utilized EfficientNet-B4 and consisted of two distinct decision-making stages with an optimization step incorporated into the second stage utilizing a probability threshold to minimize the rate of malignant false negatives. In addition, to facilitate interpretability and transparency, the proposed framework incorporated Gradient-weighted Class Activation Mapping (Grad-CAM). The system was implemented as a complete end-to-end decision-support system leveraging FastAPI for the backend and a modern web-based user interface constructed using React, TypeScript, and Tailwind CSS. The complete model was evaluated using standard performance metrics and subsequently validated on an unseen external database containing images of skin lesions acquired under a wide range of conditions. These results demonstrated that the proposed system exhibited improved generalization beyond the model’s training database. The proposed approach prioritized clinical safety, interpretability, and deployability, thereby providing a practical decision-support tool for dermatologists involved in malignant skin lesion screening. The proposed framework achieved an internal ROC-AUC of 99.3% and a malignant recall of 97.2% on the internal validation set. External validation on HAM10000 yielded a malignant recall of 99.3% without threshold re-tuning.References
[1] A. Esteva, B. Kuprel, R. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, "Dermatologist-level classification of skin cancer with deep neural networks," Nature, vol. 542, pp. 115–118, 2017.
[2] P. Tschandl, C. Rosendahl, and H. Kittler, "The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions," Scientific Data, vol. 5, p. 180161, 2018.
[3] N. C. F. Codella et al., "Skin lesion analysis toward melanoma detection: A challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC)," in Proc. IEEE 15th Int. Symp. Biomed. Imag. (ISBI), Washington, DC, USA, 2018, pp. 168–172.
[4] M. Tan and Q. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks," in Proc. Int. Conf. Mach. Learn. (ICML), 2019, pp. 6105–6114.
[5] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-CAM: Visual explanations from deep networks via gradient-based localization," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2017, pp. 618–626.
[6] D. Saraswat et al., "Explainable AI for healthcare 5.0: Opportunities and challenges," IEEE Access, vol. 10, pp. 84486–84517, 2022.
[7] N. Codella et al., "Explainability for skin lesion analysis towards algorithmic fairness," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2019.
[8] M. Harahap, A. Husein, S. Kwok, V. Wizley, J. Leonardi, D. Ong, D. Ginting, and B. Silitonga, "Skin cancer classification using EfficientNet architecture," Bulletin of Electrical Engineering and Informatics, vol. 13, pp. 2716-2728, 2024.
[9] H. H. Wang, Y. H. Wang, C. W. Liang, and Y. C. Li, "Assessment of deep learning using nonimaging information and sequential medical records to develop a prediction model for nonmelanoma skin cancer," JAMA Dermatology, vol. 155, no. 11, pp. 1277–1283, 2019.
[10] M. Alruwaili and M. Mohamed, "An Integrated Deep Learning Model with EfficientNet and ResNet for Accurate Multi-Class Skin Disease Classification," Diagnostics, vol. 15, no. 5, p. 551, 2025.
[11] A. Hekler et al., "Superior skin cancer classification by the combination of human and artificial intelligence," European Journal of Cancer, vol. 120, pp. 114–121, 2019.
[12] N. Gessert, M. Nielsen, M. Shaikh, R. Werner, and A. Schlaefer, "Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data," MethodsX, vol. 7, p. 100864, 2020.
[13] T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2006.
[14] C. Elkan, "The foundations of cost-sensitive learning," in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), vol. 17, no. 1, 2001, pp. 973–978.
[15] Z. Yu et al., "Hierarchical skin lesion image classification with prototypical decision tree," NPJ Digital Medicine, vol. 8, no. 1, p. 26, 2025.
[16] C. Barata, J. S. Marques, and M. E. Celebi, "Deep attention model for the hierarchical diagnosis of skin lesions," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2019.
[17] M. Badr, A. Elkasaby, M. Alrahmawy, and S. El-Metwally, "A multi-model deep learning architecture for diagnosing multi-class skin diseases," J. Digit. Imaging Inform. Med., vol. 38, pp. 1776–1795, 2025.
[18] T. J. Brinker et al., "Deep neural networks are superior to dermatologists in melanoma image classification," European Journal of Cancer, vol. 119, pp. 11–17, 2019.
[19] M. Combalia, N. C. F. Codella, V. Rotemberg, B. Helba, V. Vilaplana, O. Reiter, C. Carrera, A. Barreiro, A. C. Halpern, S. Puig, and J. Malvehy, "BCN20000: Dermoscopic lesions in the wild," arXiv preprint arXiv:1908.02288, 2019.
[20] R. L. Siegel, K. D. Miller, N. S. Wagle, and A. Jemal, "Cancer statistics, 2023," CA: A Cancer Journal for Clinicians, vol. 73, no. 1, pp. 17–48, 2023.
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
