JEI Letters

Image resolution upscaling in the wavelet domain using directional cycle spinning

[+] Author Affiliations
Alptekin Temizel

Visioprime, 30 St. Johns Road, Woking, GU21 7SA United Kingdom

Theo Vlachos

University of Surrey, Centre for Vision, Speech, and Signal Processing, Guildford, GU2 7XH United Kingdom

J. Electron. Imaging. 14(4), 040501 (October 13, 2005). doi:10.1117/1.2061247
History: Received June 14, 2005; Revised July 10, 2005; Accepted July 12, 2005; Published October 13, 2005; Online October 13, 2005
Text Size: A A A

Open Access Open Access

Cycle spinning, a technique mainly used for wavelet denoising, has also been shown to be successful toward image resolution upscaling in the wavelet domain. We propose a directional variant of the cycle spinning methodology. We obtain estimates of local edge orientation from a wavelet decomposition of the available low-resolution image and use this information to influence the choice of cycle spinning parameters that are employed for resolution upscaling. Our experimental results show that the proposed method outperforms competing methods for a wide range of images offering modest but consistent improvements both in objective as well as subjective terms. Lower computational complexity compared to the conventional cycle spinning is also demonstrated.

Figures in this Article

A common feature of wavelet domain image resolution upscaling algorithms is the assumption that the image to be enhanced is the low-pass filtered subband of a wavelet-transformed high-resolution (HR) image. Then the detail wavelet coefficients in subbands containing high-pass spatial frequency information are estimated and the HR image is obtained by inverse wavelet transform. In Ref. 1, detail coefficients are estimated using the evolution of wavelet transform extrema in coarser subbands. A similar but less computationally expensive approach is advocated in Ref. 2. In Ref. 3 a technique is proposed that takes into account the hidden Markov tree (HMT) approach. The HMT-based methods have been further developed not to require any training data set.4

The decimated wavelet transform is not shift-invariant and, as a result, inaccurate representation of wavelet coefficients—for example, due to quantization or non-exact estimation—introduces cyclostationarity into the image, which manifests itself as ringing in the neighborhood of discontinuities. Cycle-spinning (CS), introduced in Ref. 5, has been shown to be an effective method against ringing when used for denoising purposes. In Refs. 67, it was shown that CS, applied as a postprocessing operation, yields improvements in the framework of JPEG and JPEG2000. Recently, it has been shown that it is also a powerful alternative to competing methods, when applied to image resolution upscaling problems in the wavelet domain.8

Since its introduction, the basic CS algorithm has hardly evolved apart from an iterative variant in Ref. 9. This involves iterated applications of a processing chain consisting of forward translation, wavelet denoising, and backward translation. Although this scheme was shown to improve upon the basic CS in wavelet denoising problems, it doesn’t lend itself to wavelet domain resolution upscaling. In the resolution upscaling context, the upscaled image, which is the output of the previous iteration, would need to be downsampled before being fed as input into the next, nullifying the effects of the preceding iteration.

We introduce the concept of directional cycle spinning. In particular we take into account local edge orientation information derived from a wavelet decomposition of the available low-resolution (LR) image to influence key parameters of the CS algorithm.

Image resolution upscaling involving CS typically consists of two steps, namely, wavelet-domain zero padding (WZP) followed by the application of a CS operator.

In the first step, an initial approximation to the unknown HR image is generated using WZP: Using the available LR image x of size m×n, the unknown HR image y is reconstructed by using zero padding of high-frequency (HF) subbands followed by inverse wavelet transform:Display Formula

1ŷ0=W1[x0m,n0m,n0m,n]
where 0m,n is an all-zero matrix of size m×n and W1 is the inverse discrete wavelet transform. This implies a simplified image formation process whereby the point spread function is associated with the low-pass wavelet filter kernel used for the above transformation. The HR approximation obtained as above commonly exhibits artifacts such as smoothing and ringing. The ringing emerges at the vicinity of discontinuities as alternating undershoots and overshoots of the intensity level. CS has been shown to be an effective means to reduce these artifacts by averaging out the translation dependence.5 For the image resolution upscaling purposes, a variant of the CS methodology is applied as a second step as follows:8 First a number of LR images x̂i,j are generated from ŷ0 by (1) spatial shifting, (2) wavelet transformation, and (3) discarding the HF coefficients: x̂i,j=DWSi,jŷ0 where D represents discarding of HF coefficients, W denotes wavelet transform, and Si,j is a shift operator applying horizontal and vertical shifts of (i,j) in the range i,j{k,k+1,,k1,k}. Then, Eq. 1 is applied to all x̂i,j yielding Nŷi,j images, where N=(2k+1)(2k+1). Finally, these intermediate HR images are re-aligned and averaged to give the final HR reconstructed image: ŷ=(1N)i=kkj=kkSi,j1ŷi,j where Si,j1 is the inverse of the shifting operator Si,j.

In conventional CS as described above, all possible shifts within a range that defines a local neighborhood are used. Nevertheless, a closer inspection of artifacts in a typical HR image generated using WZP reveals that ringing artifacts not only occur in the vicinity of strong edges but, more importantly, they are strongly correlated with the orientation of those edges. In particular, for an edge of a given orientation, ringing is more pronounced in the normal direction. This is due to the fact that ringing artifacts materialize as a result of undershoots and overshoots of sharp intensity level changes. This observation suggests that CS should be predominantly applied across edges (i.e., vertically in the vicinity of a horizontal edge and vice versa) avoiding other orientations and thus preventing unnecessary smoothing of image detail.

To test this conjecture, we have used the synthetic image shown in Fig. 1. We consider this synthetic image as the ground truth HR image we seek to reconstruct. Then we generate an LR version of it by low-pass filtering followed by downsampling. Finally we obtain approximations to the original HR image using WZP followed by CS applied at different orientations: nondirectional (NCS), horizontal (HCS), vertical (VCS), or diagonal (DCS). Figure 2 indicates the patterns used for CS (shift locations used are shown as shaded elements) with the no shift location occupying (0,0). For example HR reconstructions using HCS and VCS are obtained respectively usingDisplay Formula

2Ch=i=kkŷi,02k+1andCν=j=kkŷ0,j2k+1
where ŷi,j are the intermediate HR images corresponding to a shift of (i,j) as explained in Sec. 2.

Graphic Jump LocationF1 :

PSNR (dB) values for the different quadrants of the synthetic image using different CS directions.

Graphic Jump LocationF2 :

Shifts used for different directional CS types.

PSNR values of the HR reconstruction for each quadrant and for each CS orientation are shown in Fig. 1 while the best value is shown in boldface. Our results confirm that the best reconstruction always occurs by CS in the direction of the normal to dominant edges while in the absence of such edges nondirectional CS is preferable. The gain is more pronounced in HCS and VCS, while DCS brings a more modest improvement.

Our approach uses a local estimate of image directional activity to influence the orientation in which CS is applied. This estimate is obtained by using a wavelet decomposition of the available LR image x.

Let LH0 and HL0 denote respectively the HF horizontal and vertical detail subbands of x and whr,s and wνr,s denote the wavelet coefficients at position (r,s) of LH0 and HL0 respectively. We partition x into nonoverlapping blocks and estimate horizontal and vertical activity measures for the p’th block as Ahp=r,swhr,s and Aνp=r,swνr,s respectively; summation is over all wavelet coefficients contained in block p. Finally, block p in the HR image is reconstructed by using these activity values as weights to the corresponding reconstructions generated using Eq. 2:Display Formula

3ŷp=AhpCνp+AνpChpAhp+Aνp.
It has been found that the extra computational complexity brought by application of diagonal CS hardly justifies the relatively insignificant gain it provides and hence excluded from the algorithm.

The proposed method has been tested on a number of well-known test images. An HR version of these images (512×512) was used as ground truth for performance evaluation purposes. The wavelet transform was implemented using the well-known Daubechies 97 filters, the maximum shift k was set to 5, and block sizes were 8×8. The proposed method has been compared with other wavelet based methods as well as an edge directed10 and a fractal-based11 method. PSNR values are tabulated in Table 1 for both 2× and 4× enlargement factors. Figures 34 respectively show the enlarged images and amplified residual images for subjective comparisons with bilinear interpolation and WZP. Figure 5 shows a cross section of an edge (taken from Fig. 1) and reconstructed versions as well as the absolute error of these reconstructions.

Table Grahic Jump Location
PSNR (dB) values for 2×(256×256to512×512) and 4×(128×128to512×512) enlargement.
Graphic Jump LocationF3 :

Extract from original Lena (a) and images for 4× reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Graphic Jump LocationF4 :

Extract from original Lena (a) and amplified error images for 4× reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Graphic Jump LocationF5 :

Cross section of an edge and its reconstructed versions using various methods (a). Absolute error of the reconstructed edges (b).

It can be seen that directional CS adapts better to edge orientation and avoids jagged edge (staircase) artifacts. Overall our results show that the directional CS outperforms the competing methods and offers modest but consistent improvements over baseline CS. Additionally, the proposed scheme offers a reduction in computational complexity relative to conventional CS because only shifts in the normal to salient edges are required, involving 2k WZP calculations compared with k2 in the conventional method.

A directional variant of the CS methodology was considered for image resolution upscaling in the wavelet domain. The proposed method estimates local edge orientation from a wavelet decomposition of the available LR image and uses this information to influence CS parameters. Our experimental results confirm that the proposed method outperforms competing methods for a wide range of images both in objective and subjective terms.

Chang  S. G., , Cvetkovic  Z., , and Vetterli  M., “ Resolution enhancement of images using wavelet transform extrema extrapolation. ,”  Proc. ICASSP. 4, , 2379–2382  ((1995)).
Carey  W. K., , Chuang  D. B., , and Hemami  S. S., “ Regularity-preserving image interpolation. ,” IEEE Trans. Image Process..  1057-7149 8, (9 ), 1295–1297  ((1999)).
Kinebuchi  K., , Muresan  D. D., , and Parks  T. W., “ Image interpolation using wavelet-based hidden Markov trees. ,”  Proc. ICASSP. 3, , 7–11  ((2001)).
Woo  D. H., , Eom  I. K., , and Kim  Y. S., “ Image interpolation based on inter-scale dependency in wavelet domain. ,”  Proc. ICIP. 3, , 1687–1690  ((2004)).
Coifman  R. R., and Donoho  D. L., “ Translation-invariant denoising. ,”  Lecture Notes in Statistics: Wavelets and Statistics. , pp. 125–150 ,  Springer Verlag , New York ((1995)).
Nosratinia  A., “ Denoising of JPEG images by re-application of JPEG. ,” J. VLSI Signal Proc. Syst. Signal, Image, Video Technol..  0922-5773 27, (1 ), 69–79  ((2001)).
Nosratinia  A., “ Post-processing of JPEG-2000 images to remove compression artifacts. ,” IEEE Signal Process. Lett..  1070-9908 10, (10 ), 296–299  ((2003)).
Temizel  A., and Vlachos  T., “ Wavelet domain image resolution enhancement using cycle-spinning. ,” Electron. Lett..  0013-5194 41, (3 ), 119–121  ((2005)).
Fletcher  A. K., , Ramchandran  K., , and Goyal  V. K., “ Wavelet denoising by recursive cycle spinning. ,”  ICIP. 2, , 873–876  ((2002)).
Li  X., and Orchard  M. T., “ New edge-directed interpolation. ,” IEEE Trans. Image Process..  1057-7149 10, (10 ), 1521–1527  ((2001)).
LizardTech, “ Genuine Fractals Print Pro Trial Version, ver. 3.5.1 build 787, 2004. ,” http://www.genuinefractals.com/products/gf/ (accessed 4 Nov. (2004)).
© 2005 SPIE and IS&T

Citation

Alptekin Temizel and Theo Vlachos
"Image resolution upscaling in the wavelet domain using directional cycle spinning", J. Electron. Imaging. 14(4), 040501 (October 13, 2005). ; http://dx.doi.org/10.1117/1.2061247


Figures

Graphic Jump LocationF1 :

PSNR (dB) values for the different quadrants of the synthetic image using different CS directions.

Graphic Jump LocationF2 :

Shifts used for different directional CS types.

Graphic Jump LocationF3 :

Extract from original Lena (a) and images for 4× reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Graphic Jump LocationF4 :

Extract from original Lena (a) and amplified error images for 4× reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Graphic Jump LocationF5 :

Cross section of an edge and its reconstructed versions using various methods (a). Absolute error of the reconstructed edges (b).

Tables

Table Grahic Jump Location
PSNR (dB) values for 2×(256×256to512×512) and 4×(128×128to512×512) enlargement.

References

Chang  S. G., , Cvetkovic  Z., , and Vetterli  M., “ Resolution enhancement of images using wavelet transform extrema extrapolation. ,”  Proc. ICASSP. 4, , 2379–2382  ((1995)).
Carey  W. K., , Chuang  D. B., , and Hemami  S. S., “ Regularity-preserving image interpolation. ,” IEEE Trans. Image Process..  1057-7149 8, (9 ), 1295–1297  ((1999)).
Kinebuchi  K., , Muresan  D. D., , and Parks  T. W., “ Image interpolation using wavelet-based hidden Markov trees. ,”  Proc. ICASSP. 3, , 7–11  ((2001)).
Woo  D. H., , Eom  I. K., , and Kim  Y. S., “ Image interpolation based on inter-scale dependency in wavelet domain. ,”  Proc. ICIP. 3, , 1687–1690  ((2004)).
Coifman  R. R., and Donoho  D. L., “ Translation-invariant denoising. ,”  Lecture Notes in Statistics: Wavelets and Statistics. , pp. 125–150 ,  Springer Verlag , New York ((1995)).
Nosratinia  A., “ Denoising of JPEG images by re-application of JPEG. ,” J. VLSI Signal Proc. Syst. Signal, Image, Video Technol..  0922-5773 27, (1 ), 69–79  ((2001)).
Nosratinia  A., “ Post-processing of JPEG-2000 images to remove compression artifacts. ,” IEEE Signal Process. Lett..  1070-9908 10, (10 ), 296–299  ((2003)).
Temizel  A., and Vlachos  T., “ Wavelet domain image resolution enhancement using cycle-spinning. ,” Electron. Lett..  0013-5194 41, (3 ), 119–121  ((2005)).
Fletcher  A. K., , Ramchandran  K., , and Goyal  V. K., “ Wavelet denoising by recursive cycle spinning. ,”  ICIP. 2, , 873–876  ((2002)).
Li  X., and Orchard  M. T., “ New edge-directed interpolation. ,” IEEE Trans. Image Process..  1057-7149 10, (10 ), 1521–1527  ((2001)).
LizardTech, “ Genuine Fractals Print Pro Trial Version, ver. 3.5.1 build 787, 2004. ,” http://www.genuinefractals.com/products/gf/ (accessed 4 Nov. (2004)).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.