Learned NIR&VISCAM: multi-spectral fusion for large depth-of-field computational imaging

Tingdong Kou; Junfei Shen; Qican Zhang; Wenjing Chen

doi:10.1117/12.2642517

20 December 2022 Learned NIR&VISCAM: multi-spectral fusion for large depth-of-field computational imaging

Tingdong Kou, Junfei Shen, Qican Zhang, Wenjing Chen

Proceedings Volume 12315, Optical Design and Testing XII; 123150S (2022) https://doi.org/10.1117/12.2642517
Event: SPIE/COS Photonics Asia, 2022, Online Only

Abstract

Large DOF (depth-of-field) with high SNR (signal-noise-ratio) imaging plays an important role in many applications such as unmanned driving to medical imaging. However, there is always a trade-off between DOF and SNR in traditional optical design. In this paper, we propose a NIR&VISCAM (NIR&VIS Camera) that combines multi-spectral optical design and deep learning to realize large DOF and high SNR imaging. Specifically, a multi-spectral optical imaging system based on the HVS (human visual system) is designed to provide colorful but small DOF VIS (visible) image and large DOF NIR (near-infrared) image. To achieve DOF extension, we build a fusion network NIR&VISNet consisting of a VIS encoder for color extraction, a NIR encoder for spatial details extraction and a decoder for information fusion. We establish a prototype to capture real-scene dataset containing 1000 sets and test our method on a variety of test samples. The experimental results demonstrate that our NIR&VISCAM can effectively produce large DOF images with high quality. Moreover, compared to the classic image fusion methods, our designed algorithm achieves the optimal performance in DOF extension and color fidelity. With the prominent performance in large DOF and high SNR imaging, this novel and portable system is promising for vision applications such as smartphone photography, industry detection, and life medical.

Citation Download Citation

Tingdong Kou, Junfei Shen, Qican Zhang, and Wenjing Chen "Learned NIR&VISCAM: multi-spectral fusion for large depth-of-field computational imaging", Proc. SPIE 12315, Optical Design and Testing XII, 123150S (20 December 2022); https://doi.org/10.1117/12.2642517

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Near infrared

Image fusion

Cameras

Signal to noise ratio

Computational imaging

Optical imaging

Computer programming

Show All Keywords

Keywords/Phrases

Search In:

Publication Years