HybridMedNet: An Efficient Deep Learning Framework Based on CNN–Transformer Architecture for Benign and Malignant Thyroid Nodule Classification Using Ultrasound Images

aiub ketab

doi:10.71229/qkcz0952

Authors

Aiub Ketab Kaeed College Computer Science Department, Diyala University, Diyala, Iraq.

DOI:

https://doi.org/10.71229/qkcz0952

Keywords:

Deep learning, hybrid CNN–Transformer, thyroid nodule classification, ultrasound imaging, medical decision support, squeeze-and-excitation

Abstract

Thyroid nodule assessment often involves ultrasound imaging, but differentiating benign from malignant nodules remains challenging due to image noise, acquisition variations, and subtle visual differences. Convolutional neural networks (CNNs) effectively learn local image features, but their restricted receptive fields may limit their ability to model long-range spatial relationships. Vision Transformers provide a global perspective by modeling the entire image, but typically require larger training sets, making them less suitable for data-scarce medical imaging applications. To address these limitations, this paper introduces HybridMedNet, a compact CNN–Transformer architecture for thyroid nodule classification in ultrasound images. The proposed architecture integrates a convolutional stem, channel-wise feature recalibration using Residual Squeeze-and-Excitation blocks, a six-layer Transformer encoder for global representation learning, and learned attention pooling for classification. Experiments used the official 7:1:2 train–validation–test split of the TN5000 thyroid ultrasound dataset. The model was trained with AdamW using cosine learning-rate scheduling, warmup, Mixup, label smoothing, class-weighted loss, and test-time augmentation. HybridMedNet was compared with ResNet-18, ResNet-50, and MobileNet-V2 using the same dataset partition. On the 1,000-image test set, HybridMedNet achieved an accuracy of 0.90, macro-F1 score of 0.90, and AUC of 0.95, outperforming the evaluated baselines with approximately 3.3 million trainable parameters. Grad-CAM visualizations showed that the model primarily focused on thyroid gland regions. These results demonstrate the feasibility of combining local convolutional representation with global Transformer-based reasoning for compact thyroid ultrasound classification. Further evaluation using multiple random seeds, independent datasets, statistical significance testing, and clinically aligned multi-class classification is required to establish generalizability and clinical applicability.

References

[1] F. N. Tessler, W. D. Middleton, E. G. Grant, J. K. Hoang, L. L. Berland, S. A. Teefey, et al., “ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee,” Journal of the American College of Radiology, vol. 14, no. 5, pp. 587–595, 2017, doi: 10.1016/j.jacr.2017.01.046.

[2] J. Kim, M.-H. Kim, D.-J. Lim, et al., “Deep Learning Technology for Classification of Thyroid Nodules Using Multi-View Ultrasound Images: Potential Benefits and Challenges in Clinical Application,” Endocrinology and Metabolism, 2024, doi: 10.3803/EnM.2024.2058.

[3] J. Weng, et al., “Deep Learning for Classification of Thyroid Nodules on Ultrasound: Validation on an Independent Dataset,” arXiv:2207.13765, 2023.

[4] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” in Proc. International Conference on Learning Representations (ICLR), 2021.

[5] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90.

[6] F. Shamshad, S. Khan, S. W. Zamir, M. H. Khan, M. Hayat, F. S. Khan, and H. Fu, “Transformers in Medical Imaging: A Survey,” Medical Image Analysis, vol. 88, p. 102802, 2023, doi: 10.1016/j.media.2023.102802.

[7] H. Zhang, Q. Liu, X. Han, L. Niu, and W. Sun, “TN5000: An Ultrasound Image Dataset for Thyroid Nodule Detection and Classification,” Scientific Data, vol. 12, no. 1, p. 1437, 2025, doi: 10.1038/s41597-025-05757-4.

[8] L. Wang, X. Zhou, X. Nie, X. Lin, J. Li, H. Zheng, et al., “A Multi-Scale Densely Connected Convolutional Neural Network for Automated Thyroid Nodule Classification,” Frontiers in Neuroscience, vol. 16, p. 878718, 2022, doi: 10.3389/fnins.2022.878718.

[9] O. A. Ajilisa, V. P. Jagathy Raj, and M. K. Sabu, “A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning,” Diagnostics, vol. 13, no. 14, p. 2463, 2023, doi: 10.3390/diagnostics13142463.

[10] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008.

[11] H. Wu, B. Xiao, N. Codella, M. Liu, X. Dai, L. Yuan, and L. Zhang, “CvT: Introducing Convolutions to Vision Transformers,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 22–31.

[12] Z. Dai, H. Liu, Q. V. Le, and M. Tan, “CoAtNet: Marrying Convolution and Attention for All Data Sizes,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 34, 2021, pp. 3965–3977.

[13] I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” in Proc. International Conference on Learning Representations (ICLR), 2019.

[14] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond Empirical Risk Minimization,” in Proc. International Conference on Learning Representations (ICLR), 2018.

[15] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2818–2826, doi: 10.1109/CVPR.2016.308.

[16] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248–255, doi: 10.1109/CVPR.2009.5206848.

[17] J. Hu, L. Shen, and G. Sun, “Squeeze-and-Excitation Networks,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2018, pp. 7132–7141, doi: 10.1109/CVPR.2018.00745.

[18] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4510–4520, doi: 10.1109/CVPR.2018.00474.

[19] J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “MedMNIST v2 — A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification,” Scientific Data, vol. 10, no. 1, p. 41, 2023, doi: 10.1038/s41597-022-01721-8.

[20] E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson, “Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach,” Biometrics, vol. 44, no. 3, pp. 837–845, 1988, doi: 10.2307/2531595.

[21] J. A. Hanley and B. J. McNeil, “The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve,” Radiology, vol. 143, no. 1, pp. 29–36, 1982, doi: 10.1148/radiology.143.1.7063747.

[22] M. Yavuz and N. Yumuşak, “ROI+Context Graph Neural Networks for Thyroid Nodule Classification: Baselines, Cross-Validation Protocol, and Reproducibility,” Electronics, vol. 15, no. 1, p. 151, 2026, doi: 10.3390/electronics15010151.

[23] G. N. Sujini and S. Sivadi, “Automated Thyroid Nodule Classification in Ultrasound Imaging Using a Hybrid Vision Transformer and Wasserstein GAN with Gradient Penalty,” Scientific Reports, vol. 15, p. 40786, 2025, doi: 10.1038/s41598-025-24651-y.

[24] K. Bahmane, S. Bhattacharya, and A. B. Chaouki, “Evaluation of a Hybrid CNN Model for Automatic Detection of Malignant and Benign Lesions,” Medicina, vol. 61, p. 2036, 2025, doi: 10.3390/medicina61112036.