Unsupervised super resolution network for hyperspectral histologic imaging

Hyperspectral imaging (HSI) has many advantages in microscopic applications, including high sensitivity and specificity for cancer detection on histological slides. However, acquiring hyperspectral images of a whole slide with a high image resolution and a high image quality can take a long scanning time and require a very large data storage. One potential solution is to acquire and save low-resolution hyperspectral images and reconstruct the high-resolution ones only when needed. The purpose of this study is to develop a simple yet effective unsupervised super resolution network for hyperspectral histologic imaging with the guidance of RGB digital histology images. High-resolution hyperspectral images of hemoxylin & eosin (H&E) stained slides were obtained at 10× magnification and down-sampled 2×, 4×, and 5× to generate low-resolution hyperspectral data. High-resolution digital histologic RGB images of the same field of view (FOV) were cropped and registered to the corresponding high-resolution hyperspectral images. A neural network based on a modified U-Net architecture, which takes the low-resolution hyperspectral images and high-resolution RGB images as inputs, was trained with unsupervised methods to output high-resolution hyperspectral data. The generated high-resolution hyperspectral images have similar spectral signatures and improved image contrast than the original high-resolution hyperspectral images, which indicates that the super resolution network with RGB guidance can improve the image quality. The proposed method can reduce the acquisition time and save storage space taken up by hyperspectral images without compromising image quality, which will potentially promote the use of hyperspectral imaging technology in digital pathology and many other clinical applications.