JEI Letters

Visual-oriented morphological foreground content grayscale frames interpolation method

[+] Author Affiliations
Nicolae Vizireanu, Radu Udrea

Politehnica University Bucharest, Telecommunications Department, Iuliu Maniu 1-3, Sector 6, Bucharest, 70000, Romania

J. Electron. Imaging. 18(2), 020502 (May 26, 2009). doi:10.1117/1.3134142
History: Received January 01, 2009; Revised March 04, 2009; Accepted April 06, 2009; Published May 26, 2009
Text Size: A A A

Open Access Open Access

A new and improved visual-oriented grayscale frames interpolation method consists of partially changing, step by step using growing structuring elements, the morphological S transforms of the foreground content of an input frame with the morphological S transforms of the foreground content of an output frame. Better performance comes at the expense of not very great computational complexity. Computer simulations illustrate the results.

Figures in this Article

The grayscale frames interpolation problem is the process of creating intermediary grayscale frames between two given frames.1 The interpolated grayscale frame (Xi), 0iM, M2 depends on the content of an input grayscale frame and an output grayscale frame. For i=0, the interpolated grayscale frame is equal to the input grayscale frame X0; for i=M, the interpolated grayscale frame is equal to the output grayscale frame XM.2

The simplest method involves linear interpolation.34 In Ref. 5, a dynamically elastic surface interpolation scheme was proposed. A hybrid approach that combined elastic interpolation, spline theory, and the surface consistency theorem was proposed to produce further improvement.67 Grevera and Udupa8 and Herman et al. 9 interpolated the distance and proposed shape-based methods by encoding the segmented image with distance codes. Guo et al. developed a morphology-based interpolation method and successfully resolved the problems in objects with holes and large offsets.10 Lee et al. proposed another morphology-based scheme that was simpler in computational complexity.11 Goshtasby et al. selected feature points from successive frames to control the gray-level interpolation.12 In Ref. 13, feature points were used that were based on a fuzzy measure of the boundaries and medial axis transforms. In Ref. 14, a morphological skeleton interpolation was proposed in which interpolated sets were generated from a succession of skeletons derived from the matching of two neighboring set skeletons. Berier and Neely computed the interpolation with feature line-segment control.15 An example of object-based methods that used features is shape-based interpolation,16 which was extended by allowing registration between frames17 and using feature guidance.18 For grayscale images, mathematical morphology provides a well-founded theory.19

The new interpolation grayscale frame method presented in this paper is based on the morphological S transform (MST) and consists of partially changing, step by step using growing structuring elements (SEs) the morphological S transforms of the foreground content of an input grayscale frame with the MSTs of the foreground content of an output grayscale frame. Better performance comes at the expense of not very great computational complexity.

This paper is organized as follows. Section 2 defines the grayscale MST. Section 3 presents the grayscale frames interpolation algorithm. Section 4 provides experimental results. The conclusions of this study are drawn in Sec. 5.

The MST of a grayscale image X can be calculated by means of morphological operations.19 For any SE B, we haveDisplay Formula

1(n+1)B=nBB,n=0,1,2,and0B=(0,0).

The morphological S(X,nB) transform of a grayscale image X by SE nB is given byDisplay Formula

2S(X,nB)=XnB(XnB)B.

A grayscale image can be represented using the morphological formulaDisplay Formula

3X=(S[X,(N2)B]+{S[X,(N1)B]+S(X,NB)B}B)B,
where N is the maximum value defined byDisplay Formula
4S(X,nB)=,n>NandS(X,NB).

The new and improved grayscale frame interpolation method presented in this paper consists of partially changing, step by step using growing SEs nB for n=N,,1,0 with 0B=(0,0), the MST of the foreground content of a grayscale input frame with the MST of the foreground content of an output grayscale frame.

Based on the grayscale input frame X0 and grayscale output frame XM, the foreground contents are defined asDisplay Formula

5X0f=X0.*sign(X0XM)andXMf=XM.*sign(X0XM),
whereDisplay Formula
6sign(x)={1,ifx00,ifx=0}.
Using the morphological S transform and SE B, the foreground content X0f of input grayscale frame X0 and the foreground content XMf of output grayscale frame XM can be perfectly reconstructed using 3.

The interpolated foreground content Xif of grayscale interpolated frame Xi, for 1iN and N2 for N=max(N0,NM), obtained by using the morphological formulaDisplay Formula

7Xif(X0f,XMf,B)=(S[X0f,(Ni)B]++{S[XMf,(N1)B]+S(XMf,NB)B}B)BB
for 1imin(N0,NM)1. Finally, the grayscale interpolated frame Xi is given byDisplay Formula
8Xi(X0,XM,B)=[X0X0.*sign(X0XM)]+Xif

Using morphological formula 7, we can build a large number, min(N0,NM), of not necessarily different interpolated grayscale frames.

We used the proposed algorithm for grayscale frame interpolation by constructing a frame using growing SEs and different i values. We performed the experiments using grayscale images.

An example is presented in Fig. 1. Test input and output grayscale frames are presented in Figs. 1, and the “missing” frame Xref is in Fig. 1. This frame will be used as reference to compare our results. Our frames have dimensions [256, 256].

Graphic Jump LocationF1 :

Input, output, and reference grayscale frames for the interpolation example: (a) grayscale input frame Xi, (b) grayscale output frame XM, and (c) grayscale reference frame Xref.

Using formula 5, we obtained the foreground contents for input and output frame X0f and XMf for input and output frame X0 and XM [Figs. 2]. The background content for both the input and output frames (content the frames have in common), Xb [Fig. 2], is given by Display Formula

9Xb=X0X0.*sign(X0XM).

Graphic Jump LocationF2 :

Foreground and background contents for the input frame and output frames: (a) foreground contents for input frame X0f, (b) foreground contents for output frame XMf, and (c) background content of both frames (content the frames have in common) Xb.

Using the morphological S transform and crosses or squares as SE B, we obtained N0c=38, NMc=38, and Nc=38 for the crosses and N0s=30, NMs=30, and Ns=30 for the squares. Interpolated foreground content Xif, with 1i38 (for the crosses) and 1i30 (for the squares), was obtained using morphological formula 7. Finally, grayscale interpolated frame Xi was given by formula 8. For the crosses, we built 34 interpolated grayscale frames, but only 21 were different. For the squares, we built 30 interpolated grayscale frames, but only 17 were different. Using a standard computer and an optimized C routine implementation, we obtained a grayscale interpolated frame at 40ms. For size(Xref)=[AB], we can define error function ri byDisplay Formula

10ri=sum(sum{sign[abs(XrefXi)]})(A*B).

Some error function results for the crosses (ric) and squares (ris) SEs are presented in Tables 12.

Table Grahic Jump Location
Error function ric.
Table Grahic Jump Location
Error function ris.

Interpolated frames using different SEs are presented in Figs. 34.

Graphic Jump LocationF3 :

Interpolated frame (Xi, i=12) using crosses as the SE.

Graphic Jump LocationF4 :

Interpolated frame (Xi, i=10) using squares as the SE.

After simulations using different SEs and different i values, we observed that one of the interpolated frames offered by our method was similar to the original reference frame.

This paper has addressed the grayscale frames generalized interpolation by means of mathematical morphology.

After describing the algorithm, we provided experimental results that were very encouraging. We applied the proposed scheme to real-world data. For many situations, the proposed scheme was experimentally shown to successfully resolve complex interpolation problems. Some preprocessing could improve the results. The algorithm is fully morphological and can be applied quickly. This entire process is efficient without significant computational complexity. However, it is difficult to state rigorously that our method produces theoretically correct interpolated results for any frames.

This research was founded by UEFISCSU, contract PII, IDEI, no. 100/2007, D.N. VIZIREANU.

Iwanowski  M., and Serra  J., “ Morphological interpolation and color images. ,” in  Proc. of Intern. Conf. on Image Proc.. ,  Venice ,  Italy  ((1999)).
Huang  A.-M., and Nguyen  T. Q., “ A multistage motion vector processing method for motion-compensated frame interpolation. ,” IEEE Trans. Image Process..  1057-7149 17, (5 ), 694–708  ((2008)).
Liang  C. C., , Chen  C. T., , and Lin  W. C., “ Intensity interpolation for reconstructing 3-D images from serial cross-sections. ,” in  Proc. IEEE Eng. Med. Bio. Soc. 10th Int. Conf.. ,  New Orleans, LA , Paper CH2566-8 , pp. 1389–1390  ((1988)).
Liang  C. C., , Chen  C. T., , and Lin  W. C. “ Intensity interpolation for branching in reconstructing three-dimensional objects from serial cross-sections. ,” Proc. SPIE.  0277-786X 1445, , 456-467  ((1991)).
Lin  W.-C., , Liang  C.-C., , and Chen  C.-T., “ Dynamic elastic interpolation for 3-D image reconstruction from cross sections. ,” IEEE Trans. Med. Imaging.  0278-0062 7, (9 ), 225–232  ((1988)).
Chen  S.-Y., and Lin  W.-C., “ Automated surface interpolation technique for 3-D object reconstruction from serial cross sections. ,” Comput. Med. Imaging Graph..  0895-6111 15, (4 ), 265–276  ((1991)).
Liu  Y.-H., , Sun  Y.-N., , Mao  C.-W., , and Lin  C.-J., “ Edge-shrinking interpolation for images. ,” Comput. Vis. Graph. Image Process..  0734-189X 21, (2 ), 91–101  ((1997)).
Grevera  G. J., and Udupa  J. K., “ Shape-based interpolation of multi-dimensional grey-level images. ,” IEEE Trans. Med. Imaging.  0278-0062 15, (12 ) 881–892  ((1996)).
Herman  G. T., , Zheng  J., , and Bucholtz  C. A., “ Shape-based interpolation. ,” IEEE Comput. Graphics Appl..  0272-1716 12, , 69–79  (May (1992)).
Guo  J.-F., , Cai  Y.-L., , and Wang  Y.-P., “ Morphology-based interpolation for 3D image reconstruction. ,” Comput. Med. Imaging Graph..  0895-6111 19, (3 ), 267–279  ((1995)).
Lee  T.-Y., and Wang  W.-H., “ Morphology-based three-dimensional interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 19, (7 ), 711–721  ((2000)).
Goshtasby  A., , Tuner  D. A., , and Ackerman  L. V., “ Matching tomographic slices for interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 11, (4 ), 507–516  ((1992)).
Puff  D. T., , Eberly  D., , and Pizer  S. M., “ Object-based interpolation via cores. ,” Proc. SPIE.  0277-786X 2167, , 143–150  ((1994)).
Chatzis  V., and Pitas  I., “ Interpolation of 3-D binary images based on morphological skeletonization. ,” IEEE Trans. Med. Imaging.  0278-0062 19, (7 ), 699–710  ((2000)).
Berier  T., and Neely  S., “ Feature-based image metamorphosis. ,” in  Proc. Comput. Graphics (SIGGRAPH’92). 26, (2 ), 35–42  ((1992)).
Raya  S. P., and Udupa  J. K., “ Shape-based interpolation of multidimensional objects. ,” IEEE Trans. Med. Imaging.  0278-0062 9, (3 ), 32–42  ((1990)).
Higgins  W. E., , Morice  C., , and Ritman  E. L., “ Shape-based interpolation of tree-like structures in three-dimensional images. ,” IEEE Trans. Med. Imaging.  0278-0062 12, (9 ), 439–450  ((1993)).
Lee  T.-Y., and Lin  C.-H., “ Feature-guided shape-based image interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 21, (12 ), 1479–1489  ((2002)).
Dougherty  E. R., and Lotufo  R. A.,  Hands-on Morphological Image Processing. ,  SPIE Press ,  Bellingham, WA  ((2003)).
© 2009 SPIE and IS&T

Citation

Nicolae Vizireanu and Radu Udrea
"Visual-oriented morphological foreground content grayscale frames interpolation method", J. Electron. Imaging. 18(2), 020502 (May 26, 2009). ; http://dx.doi.org/10.1117/1.3134142


Figures

Graphic Jump LocationF3 :

Interpolated frame (Xi, i=12) using crosses as the SE.

Graphic Jump LocationF2 :

Foreground and background contents for the input frame and output frames: (a) foreground contents for input frame X0f, (b) foreground contents for output frame XMf, and (c) background content of both frames (content the frames have in common) Xb.

Graphic Jump LocationF1 :

Input, output, and reference grayscale frames for the interpolation example: (a) grayscale input frame Xi, (b) grayscale output frame XM, and (c) grayscale reference frame Xref.

Graphic Jump LocationF4 :

Interpolated frame (Xi, i=10) using squares as the SE.

Tables

Table Grahic Jump Location
Error function ris.
Table Grahic Jump Location
Error function ric.

References

Iwanowski  M., and Serra  J., “ Morphological interpolation and color images. ,” in  Proc. of Intern. Conf. on Image Proc.. ,  Venice ,  Italy  ((1999)).
Huang  A.-M., and Nguyen  T. Q., “ A multistage motion vector processing method for motion-compensated frame interpolation. ,” IEEE Trans. Image Process..  1057-7149 17, (5 ), 694–708  ((2008)).
Liang  C. C., , Chen  C. T., , and Lin  W. C., “ Intensity interpolation for reconstructing 3-D images from serial cross-sections. ,” in  Proc. IEEE Eng. Med. Bio. Soc. 10th Int. Conf.. ,  New Orleans, LA , Paper CH2566-8 , pp. 1389–1390  ((1988)).
Liang  C. C., , Chen  C. T., , and Lin  W. C. “ Intensity interpolation for branching in reconstructing three-dimensional objects from serial cross-sections. ,” Proc. SPIE.  0277-786X 1445, , 456-467  ((1991)).
Lin  W.-C., , Liang  C.-C., , and Chen  C.-T., “ Dynamic elastic interpolation for 3-D image reconstruction from cross sections. ,” IEEE Trans. Med. Imaging.  0278-0062 7, (9 ), 225–232  ((1988)).
Chen  S.-Y., and Lin  W.-C., “ Automated surface interpolation technique for 3-D object reconstruction from serial cross sections. ,” Comput. Med. Imaging Graph..  0895-6111 15, (4 ), 265–276  ((1991)).
Liu  Y.-H., , Sun  Y.-N., , Mao  C.-W., , and Lin  C.-J., “ Edge-shrinking interpolation for images. ,” Comput. Vis. Graph. Image Process..  0734-189X 21, (2 ), 91–101  ((1997)).
Grevera  G. J., and Udupa  J. K., “ Shape-based interpolation of multi-dimensional grey-level images. ,” IEEE Trans. Med. Imaging.  0278-0062 15, (12 ) 881–892  ((1996)).
Herman  G. T., , Zheng  J., , and Bucholtz  C. A., “ Shape-based interpolation. ,” IEEE Comput. Graphics Appl..  0272-1716 12, , 69–79  (May (1992)).
Guo  J.-F., , Cai  Y.-L., , and Wang  Y.-P., “ Morphology-based interpolation for 3D image reconstruction. ,” Comput. Med. Imaging Graph..  0895-6111 19, (3 ), 267–279  ((1995)).
Lee  T.-Y., and Wang  W.-H., “ Morphology-based three-dimensional interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 19, (7 ), 711–721  ((2000)).
Goshtasby  A., , Tuner  D. A., , and Ackerman  L. V., “ Matching tomographic slices for interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 11, (4 ), 507–516  ((1992)).
Puff  D. T., , Eberly  D., , and Pizer  S. M., “ Object-based interpolation via cores. ,” Proc. SPIE.  0277-786X 2167, , 143–150  ((1994)).
Chatzis  V., and Pitas  I., “ Interpolation of 3-D binary images based on morphological skeletonization. ,” IEEE Trans. Med. Imaging.  0278-0062 19, (7 ), 699–710  ((2000)).
Berier  T., and Neely  S., “ Feature-based image metamorphosis. ,” in  Proc. Comput. Graphics (SIGGRAPH’92). 26, (2 ), 35–42  ((1992)).
Raya  S. P., and Udupa  J. K., “ Shape-based interpolation of multidimensional objects. ,” IEEE Trans. Med. Imaging.  0278-0062 9, (3 ), 32–42  ((1990)).
Higgins  W. E., , Morice  C., , and Ritman  E. L., “ Shape-based interpolation of tree-like structures in three-dimensional images. ,” IEEE Trans. Med. Imaging.  0278-0062 12, (9 ), 439–450  ((1993)).
Lee  T.-Y., and Lin  C.-H., “ Feature-guided shape-based image interpolation. ,” IEEE Trans. Med. Imaging.  0278-0062 21, (12 ), 1479–1489  ((2002)).
Dougherty  E. R., and Lotufo  R. A.,  Hands-on Morphological Image Processing. ,  SPIE Press ,  Bellingham, WA  ((2003)).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.