Regular Articles

Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing

[+] Author Affiliations
Yan Dong, Chunyu Lin, Yao Zhao

Beijing Jiaotong University, Institute of Information Science, Beijing Key Laboratory of Advanced Information Science and Network, Beijing, China

Chao Yao

Beijing University of Posts and Telecommunications, Institute of Sensing Technology and Business, Beijing, China

J. Electron. Imaging. 26(4), 043004 (Jul 11, 2017). doi:10.1117/1.JEI.26.4.043004
History: Received January 24, 2017; Accepted June 19, 2017
Text Size: A A A

Abstract.  In texture-plus-depth format of three-dimensional visual data, both texture and depth maps are required to synthesize a desired view via depth-image-based rendering. However, the depth maps captured or estimated always exist with low resolution compared to their corresponding texture images. We introduce a joint edge-guided convolutional neural network that upsamples the resolution of a depth map on the premise of synthesized view quality. The network takes the low-resolution depth map as an input using a joint edge feature extracted from the depth map and the registered texture image as a reference, and then produces a high-resolution depth map. We further use local constraints that preserve smooth regions and sharp edges so as to improve the quality of the depth map and synthesized view. Finally, a global looping optimization is performed with virtual view quality as guidance in the recovery process. We train our model using pairs of depth maps and texture images and then make tests on other depth maps and video sequences. The experimental results demonstrate that our scheme outperforms existing methods both in the quality of the depth maps and synthesized views.

Figures in this Article
© 2017 SPIE and IS&T

Citation

Yan Dong ; Chunyu Lin ; Yao Zhao and Chao Yao
"Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing", J. Electron. Imaging. 26(4), 043004 (Jul 11, 2017). ; http://dx.doi.org/10.1117/1.JEI.26.4.043004


Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement


 

  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.