Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features

Shulong Li, Panpan Xu, Bin Li, Liyuan Chen, Zhiguo Zhou, Hongxia Hao, Yingying Duan, Michael Folkert, Jianhua Ma, Shiying Huang, Steve Jiang, Jing Wang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

To predict lung nodule malignancy with a high sensitivity and specificity for low dose CT (LDCT) lung cancer screening, we propose a fusion algorithm that combines handcrafted features (HF) into the features learned at the output layer of a 3D deep convolutional neural network (CNN). First, we extracted twenty-nine HF, including nine intensity features, eight geometric features, and twelve texture features based on grey-level co-occurrence matrix (GLCM). We then trained 3D CNNs modified from three 2D CNN architectures (AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at the output layer. For each 3D CNN, the CNN features combined with the 29 HF were used as the input for the support vector machine (SVM) coupled with the sequential forward feature selection (SFS) method to select the optimal feature subset and construct the classifiers. The fusion algorithm takes full advantage of the HF and the highest level CNN features learned at the output layer. It can overcome the disadvantage of the HF that may not fully reflect the unique characteristics of a particular lesion by combining the intrinsic CNN features. Meanwhile, it also alleviates the requirement of a large scale annotated dataset for the CNNs based on the complementary of HF. The patient cohort includes 431 malignant nodules and 795 benign nodules extracted from the LIDC/IDRI database. For each investigated CNN architecture, the proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and specificity scores among all competitive classification models.

Original languageEnglish (US)
Number of pages1
JournalPhysics in medicine and biology
Volume64
Issue number17
DOIs
StatePublished - Sep 4 2019

Fingerprint

Lung
Sensitivity and Specificity
Neoplasms
Early Detection of Cancer
Area Under Curve
Lung Neoplasms
Databases
Datasets
Support Vector Machine

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging

Cite this

Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features. / Li, Shulong; Xu, Panpan; Li, Bin; Chen, Liyuan; Zhou, Zhiguo; Hao, Hongxia; Duan, Yingying; Folkert, Michael; Ma, Jianhua; Huang, Shiying; Jiang, Steve; Wang, Jing.

In: Physics in medicine and biology, Vol. 64, No. 17, 04.09.2019.

Research output: Contribution to journalArticle

Li, Shulong ; Xu, Panpan ; Li, Bin ; Chen, Liyuan ; Zhou, Zhiguo ; Hao, Hongxia ; Duan, Yingying ; Folkert, Michael ; Ma, Jianhua ; Huang, Shiying ; Jiang, Steve ; Wang, Jing. / Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features. In: Physics in medicine and biology. 2019 ; Vol. 64, No. 17.
@article{0dad3bd666a34a2396442edfe532dc75,
title = "Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features",
abstract = "To predict lung nodule malignancy with a high sensitivity and specificity for low dose CT (LDCT) lung cancer screening, we propose a fusion algorithm that combines handcrafted features (HF) into the features learned at the output layer of a 3D deep convolutional neural network (CNN). First, we extracted twenty-nine HF, including nine intensity features, eight geometric features, and twelve texture features based on grey-level co-occurrence matrix (GLCM). We then trained 3D CNNs modified from three 2D CNN architectures (AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at the output layer. For each 3D CNN, the CNN features combined with the 29 HF were used as the input for the support vector machine (SVM) coupled with the sequential forward feature selection (SFS) method to select the optimal feature subset and construct the classifiers. The fusion algorithm takes full advantage of the HF and the highest level CNN features learned at the output layer. It can overcome the disadvantage of the HF that may not fully reflect the unique characteristics of a particular lesion by combining the intrinsic CNN features. Meanwhile, it also alleviates the requirement of a large scale annotated dataset for the CNNs based on the complementary of HF. The patient cohort includes 431 malignant nodules and 795 benign nodules extracted from the LIDC/IDRI database. For each investigated CNN architecture, the proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and specificity scores among all competitive classification models.",
author = "Shulong Li and Panpan Xu and Bin Li and Liyuan Chen and Zhiguo Zhou and Hongxia Hao and Yingying Duan and Michael Folkert and Jianhua Ma and Shiying Huang and Steve Jiang and Jing Wang",
year = "2019",
month = "9",
day = "4",
doi = "10.1088/1361-6560/ab326a",
language = "English (US)",
volume = "64",
journal = "Physics in Medicine and Biology",
issn = "0031-9155",
publisher = "IOP Publishing Ltd.",
number = "17",

}

TY - JOUR

T1 - Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features

AU - Li, Shulong

AU - Xu, Panpan

AU - Li, Bin

AU - Chen, Liyuan

AU - Zhou, Zhiguo

AU - Hao, Hongxia

AU - Duan, Yingying

AU - Folkert, Michael

AU - Ma, Jianhua

AU - Huang, Shiying

AU - Jiang, Steve

AU - Wang, Jing

PY - 2019/9/4

Y1 - 2019/9/4

N2 - To predict lung nodule malignancy with a high sensitivity and specificity for low dose CT (LDCT) lung cancer screening, we propose a fusion algorithm that combines handcrafted features (HF) into the features learned at the output layer of a 3D deep convolutional neural network (CNN). First, we extracted twenty-nine HF, including nine intensity features, eight geometric features, and twelve texture features based on grey-level co-occurrence matrix (GLCM). We then trained 3D CNNs modified from three 2D CNN architectures (AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at the output layer. For each 3D CNN, the CNN features combined with the 29 HF were used as the input for the support vector machine (SVM) coupled with the sequential forward feature selection (SFS) method to select the optimal feature subset and construct the classifiers. The fusion algorithm takes full advantage of the HF and the highest level CNN features learned at the output layer. It can overcome the disadvantage of the HF that may not fully reflect the unique characteristics of a particular lesion by combining the intrinsic CNN features. Meanwhile, it also alleviates the requirement of a large scale annotated dataset for the CNNs based on the complementary of HF. The patient cohort includes 431 malignant nodules and 795 benign nodules extracted from the LIDC/IDRI database. For each investigated CNN architecture, the proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and specificity scores among all competitive classification models.

AB - To predict lung nodule malignancy with a high sensitivity and specificity for low dose CT (LDCT) lung cancer screening, we propose a fusion algorithm that combines handcrafted features (HF) into the features learned at the output layer of a 3D deep convolutional neural network (CNN). First, we extracted twenty-nine HF, including nine intensity features, eight geometric features, and twelve texture features based on grey-level co-occurrence matrix (GLCM). We then trained 3D CNNs modified from three 2D CNN architectures (AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at the output layer. For each 3D CNN, the CNN features combined with the 29 HF were used as the input for the support vector machine (SVM) coupled with the sequential forward feature selection (SFS) method to select the optimal feature subset and construct the classifiers. The fusion algorithm takes full advantage of the HF and the highest level CNN features learned at the output layer. It can overcome the disadvantage of the HF that may not fully reflect the unique characteristics of a particular lesion by combining the intrinsic CNN features. Meanwhile, it also alleviates the requirement of a large scale annotated dataset for the CNNs based on the complementary of HF. The patient cohort includes 431 malignant nodules and 795 benign nodules extracted from the LIDC/IDRI database. For each investigated CNN architecture, the proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and specificity scores among all competitive classification models.

UR - http://www.scopus.com/inward/record.url?scp=85071788094&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071788094&partnerID=8YFLogxK

U2 - 10.1088/1361-6560/ab326a

DO - 10.1088/1361-6560/ab326a

M3 - Article

C2 - 31307017

AN - SCOPUS:85071788094

VL - 64

JO - Physics in Medicine and Biology

JF - Physics in Medicine and Biology

SN - 0031-9155

IS - 17

ER -