Depth-layer-based multiview image synthesis and coding for interactive z- and x-dimension view switching

Yu Mao; Gene Cheung; Yusheng Ji

doi:10.1117/12.2004254

21 February 2013 Depth-layer-based multiview image synthesis and coding for interactive z- and x-dimension view switching

Yu Mao, Gene Cheung, Yusheng Ji

Author Affiliations +

Proceedings Volume 8666, Visual Information Processing and Communication IV; 86660C (2013) https://doi.org/10.1117/12.2004254
Event: IS&T/SPIE Electronic Imaging, 2013, Burlingame, California, United States

Abstract

In an interactive multiview image navigation system, a user requests switches to adjacent views as he observes the static 3D scene from different viewpoints. In response, the server transmits encoded data to enable client- side decoding and rendering of the requested viewpoint images. It is clear that there exists correlation between consecutive requested viewpoint images that can be exploited to lower transmission rate. In previous works, this is done using a pixel-based synthesis and coding approach for view-switch along the x-dimension (horizontal camera motion): given texture and depth maps of the previous view, texture pixels are individually shifted horizontally to the newly requested view, each according to its disparity value, via depth-image-based rendering (DIBR). Unknown pixels in the disoccluded region in the new view (pixels not visible in the previous view) are either inpainted, or intra-coded and transmitted by server for reconstruction at decoder. In this paper, to enable efficient view-switch along the z-dimension (camera motion into / out of the scene), we propose an alternative layer-based synthesis and coding approach. Specifically, we first divide each multiview image into depth layers, where adjacent pixels with similar depth values are grouped to the same layer. During a view-switch into the scene, spatial region in a layer is enlarged via super-resolution, where the scale factor is determined by the distance between the layer and the camera. On the other hand, during a view-switch out of the scene, spatial region in a layer is shrunk via low-pass filtering and down-sampling. Due to high quality reconstruction of depth layers in the new view via rescaling, coding and transmission of a depth layer in the new view by server is necessary only in the rare case when the quality of layer-based reconstruction is poor, saving transmission rate. Experiments show that our layer-based approach can reduce bit-rate by up to 35% compared to previous pixel-based approach.

Citation Download Citation

Yu Mao, Gene Cheung, and Yusheng Ji "Depth-layer-based multiview image synthesis and coding for interactive z- and x-dimension view switching", Proc. SPIE 8666, Visual Information Processing and Communication IV, 86660C (21 February 2013); https://doi.org/10.1117/12.2004254

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available