Affiliations 

  • 1 Xinxiang Medical University, No. 601, Jinshui Avenue, Xinxiang, 453003, CHINA
  • 2 DEPARTMENT OF BIOMEDICAL IMAGING, University of Malaya, FACULTY OF MEDICINE, UNIVERSITY OF MALAYA, Kuala Lumpur, 50603, MALAYSIA
  • 3 University of Malaya, Kuala Lumpur, Kuala Lumpur, Wilayah Persekutuan, 50603, MALAYSIA
  • 4 University of Malaya, Center of Image and Signal Processing, Kuala Lumpur, Wilayah Persekutuan, 50603, MALAYSIA
  • 5 Xinxiang Medical University, No. 601, Jinshui Avenue, Xinxiang, Henan, 453000, CHINA
  • 6 Xinxiang Medical University, No. 601, Jinshui Avenue, Hongqi District, Xinxiang City, Henan Province, Xinxiang, Henan, 453003, CHINA
  • 7 college of Biomedical Engineering, Xinxiang Medical University, No. 601, Jinshui Avenue, Xinxiang, 453003, CHINA
Phys Med Biol, 2024 Feb 19.
PMID: 38373345 DOI: 10.1088/1361-6560/ad2a95

Abstract

OBJECTIVE: Generally, due to a lack of explainability, radiomics based on deep learning has been perceived as a black-box solution for radiologists. Automatic generation of diagnostic reports is a semantic approach to enhance the explanation of deep learning radiomics (DLR).

APPROACH: In this paper, we propose a novel model called radiomics-reporting network (Radioport), which incorporates text attention. This model aims to improve the interpretability of deep learning radiomics in mammographic calcification diagnosis. Firstly, it employs convolutional neural networks (CNN) to extract visual features as radiomics for multi-category classification based on Breast Imaging Reporting and Data System (BI-RADS). Then, it builds a mapping between these visual features and textual features to generate diagnostic reports, incorporating an attention module for improved clarity.

MAIN RESULTS: To demonstrate the effectiveness of our proposed model, we conducted experiments on a breast calcification dataset comprising mammograms and diagnostic reports. The results demonstrate that our model can: (i) semantically enhance the interpretability of deep learning radiomics; and, (ii) improve the readability of generated medical reports.

SIGNIFICANCE: Our interpretable textual model can explicitly simulate the mammographic calcification diagnosis process.

* Title and MeSH Headings from MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine.