1.

Introduction

A common feature of wavelet domain image resolution upscaling algorithms is the assumption that the image to be enhanced is the low-pass filtered subband of a wavelet-transformed high-resolution (HR) image. Then the detail wavelet coefficients in subbands containing high-pass spatial frequency information are estimated and the HR image is obtained by inverse wavelet transform. In Ref. 1, detail coefficients are estimated using the evolution of wavelet transform extrema in coarser subbands. A similar but less computationally expensive approach is advocated in Ref. 2. In Ref. 3 a technique is proposed that takes into account the hidden Markov tree (HMT) approach. The HMT-based methods have been further developed not to require any training data set.⁴

The decimated wavelet transform is not shift-invariant and, as a result, inaccurate representation of wavelet coefficients—for example, due to quantization or non-exact estimation—introduces cyclostationarity into the image, which manifests itself as ringing in the neighborhood of discontinuities. Cycle-spinning (CS), introduced in Ref. 5, has been shown to be an effective method against ringing when used for denoising purposes. In Refs. 6, 7, it was shown that CS, applied as a postprocessing operation, yields improvements in the framework of JPEG and JPEG2000. Recently, it has been shown that it is also a powerful alternative to competing methods, when applied to image resolution upscaling problems in the wavelet domain.⁸

Since its introduction, the basic CS algorithm has hardly evolved apart from an iterative variant in Ref. 9. This involves iterated applications of a processing chain consisting of forward translation, wavelet denoising, and backward translation. Although this scheme was shown to improve upon the basic CS in wavelet denoising problems, it doesn’t lend itself to wavelet domain resolution upscaling. In the resolution upscaling context, the upscaled image, which is the output of the previous iteration, would need to be downsampled before being fed as input into the next, nullifying the effects of the preceding iteration.

We introduce the concept of directional cycle spinning. In particular we take into account local edge orientation information derived from a wavelet decomposition of the available low-resolution (LR) image to influence key parameters of the CS algorithm.

2.

Image Resolution Upscaling Using Cycle Spinning

Image resolution upscaling involving CS typically consists of two steps, namely, wavelet-domain zero padding (WZP) followed by the application of a CS operator.

In the first step, an initial approximation to the unknown HR image is generated using WZP: Using the available LR image $\mathbf{x}$ of size $m\times n$ , the unknown HR image $\mathbf{y}$ is reconstructed by using zero padding of high-frequency (HF) subbands followed by inverse wavelet transform:

3.

Directional Cycle Spinning

In conventional CS as described above, all possible shifts within a range that defines a local neighborhood are used. Nevertheless, a closer inspection of artifacts in a typical HR image generated using WZP reveals that ringing artifacts not only occur in the vicinity of strong edges but, more importantly, they are strongly correlated with the orientation of those edges. In particular, for an edge of a given orientation, ringing is more pronounced in the normal direction. This is due to the fact that ringing artifacts materialize as a result of undershoots and overshoots of sharp intensity level changes. This observation suggests that CS should be predominantly applied across edges (i.e., vertically in the vicinity of a horizontal edge and vice versa) avoiding other orientations and thus preventing unnecessary smoothing of image detail.

To test this conjecture, we have used the synthetic image shown in Fig. 1 . We consider this synthetic image as the ground truth HR image we seek to reconstruct. Then we generate an LR version of it by low-pass filtering followed by downsampling. Finally we obtain approximations to the original HR image using WZP followed by CS applied at different orientations: nondirectional (NCS), horizontal (HCS), vertical (VCS), or diagonal (DCS). Figure 2 indicates the patterns used for CS (shift locations used are shown as shaded elements) with the no shift location occupying (0,0). For example HR reconstructions using HCS and VCS are obtained respectively using

Fig. 1

PSNR (dB) values for the different quadrants of the synthetic image using different CS directions.

Fig. 2

Shifts used for different directional CS types.

PSNR values of the HR reconstruction for each quadrant and for each CS orientation are shown in Fig. 1 while the best value is shown in boldface. Our results confirm that the best reconstruction always occurs by CS in the direction of the normal to dominant edges while in the absence of such edges nondirectional CS is preferable. The gain is more pronounced in HCS and VCS, while DCS brings a more modest improvement.

4.

Algorithm Description

Our approach uses a local estimate of image directional activity to influence the orientation in which CS is applied. This estimate is obtained by using a wavelet decomposition of the available LR image $\mathbf{x}$ .

Let $L{H}_0$ and $H{L}_0$ denote respectively the HF horizontal and vertical detail subbands of $\mathbf{x}$ and $w_h^{r,s}$ and $w_{\nu }^{r,s}$ denote the wavelet coefficients at position $(r,s)$ of $L{H}_0$ and $H{L}_0$ respectively. We partition $\mathbf{x}$ into nonoverlapping blocks and estimate horizontal and vertical activity measures for the $p$ ’th block as $A_h^p={\sum }_{r,s}\mid w_h^{r,s}\mid$ and $A_{\nu }^p={\sum }_{r,s}\mid w_{\nu }^{r,s}\mid$ respectively; summation is over all wavelet coefficients contained in block $p$ . Finally, block $p$ in the HR image is reconstructed by using these activity values as weights to the corresponding reconstructions generated using Eq. ²:

5.

Experimental Results

The proposed method has been tested on a number of well-known test images. An HR version of these images $(512\times 512)$ was used as ground truth for performance evaluation purposes. The wavelet transform was implemented using the well-known Daubechies $9∕7$ filters, the maximum shift $k$ was set to 5, and block sizes were $8\times 8$ . The proposed method has been compared with other wavelet based methods as well as an edge directed¹⁰ and a fractal-based¹¹ method. PSNR values are tabulated in Table 1 for both $2\times$ and $4\times$ enlargement factors. Figures 3 and 4 respectively show the enlarged images and amplified residual images for subjective comparisons with bilinear interpolation and WZP. Figure 5 shows a cross section of an edge (taken from Fig. 1) and reconstructed versions as well as the absolute error of these reconstructions.

Fig. 3

Extract from original Lena (a) and images for $4\times$ reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Fig. 4

Extract from original Lena (a) and amplified error images for $4\times$ reconstructions: bilinear interpolation (b), WZP (c), and directional CS (d).

Fig. 5

Cross section of an edge and its reconstructed versions using various methods (a). Absolute error of the reconstructed edges (b).

Table 1

PSNR (dB) values for 2× (256×256to512×512) and 4× (128×128to512×512) enlargement.

It can be seen that directional CS adapts better to edge orientation and avoids jagged edge (staircase) artifacts. Overall our results show that the directional CS outperforms the competing methods and offers modest but consistent improvements over baseline CS. Additionally, the proposed scheme offers a reduction in computational complexity relative to conventional CS because only shifts in the normal to salient edges are required, involving $2k$ WZP calculations compared with $k^2$ in the conventional method.

6.

Conclusion

A directional variant of the CS methodology was considered for image resolution upscaling in the wavelet domain. The proposed method estimates local edge orientation from a wavelet decomposition of the available LR image and uses this information to influence CS parameters. Our experimental results confirm that the proposed method outperforms competing methods for a wide range of images both in objective and subjective terms.