This PDF file contains the editorial “Editorial (January 1998)” for JEI Vol. 7 Issue 01

This PDF file contains the editorial “Guest Editorial: Special Section on Computational Intelligence in Image Processing” for JEI Vol. 7 Issue 01

We propose a new method for measuring the shapes of objects. The proposed method is based on integrating stereo vision and shape from shading with evolutionary programming (EP). In some traditional methods, the reflection of objects is diffuse reflection, but some methods cannot be applied to real objects because the reflection of real objects consists of diffuse and specular reflection. Photometric stereo can be applied to real objects, but this method requires many original images to measure the shape accurately. The proposed method applies the Phong model to a reflectivity function and measures the shape of an object that has the diffused and specular reflection effect. Then this method uses EP to identify the reflectivity function accurately.

A free-form three-dimensional (3-D) object recognition system using artificial neural networks (ANNs) is described. The system is able to learn and recognize 3-D objects that have various surface shapes. The types of surface shapes the system is able to handle include not only predefined surfaces such as simple piecewise quadric surfaces but also more complex free-form surfaces. The system utilizes ANNs to derive induced representations and inductive learning of 3-D object classes. Starting with range image processing, the surfaces of objects are segmented into surface parts by analyzing local shape features called surface/sphere intersection signatures (SSISs). Two layers of self-organizing feature maps (SOFMs) are then used to learn those segmented surface parts and their geometrical relationships. By finding corresponding neurons in the SOFMs for all pairs of surface parts appearing in the observed object, the object is described by a binary image that represents firing states of neurons. The learning vector quantization (LVQ) network is used for learning and recognizing 3-D objects from objects' binary image descriptions. The recognition performance of the system is demonstrated using several objects.

In the research areas in computer vision, many applications have been discovered using texture classification techniques, such as the content retrieval in multimedia, the computer-aided diagnosis of medical images, and the segmentation of remote sensing images. The success of the texture classification of a given set of images hinges on the designs of texture features and the classifiers. We present a new texture feature, fuzzy texture spectrum, for texture classification, which is based on the relative gray levels between pixels. A vector of fuzzy values is used to indicate the relationship of the gray levels between the neighboring pixels. The fuzzy texture spectrum can be considered as the distribution of the fuzzified differences between the neighboring pixels. It is an improved version of the reduced texture spectrum, and it is less sensitive to the noise and the changing of the background brightness in texture images. We use 12 Brodatz texture images in the simulations to show the effectiveness of the new texture feature. Our simulation results show that the rate of classification error can be reduced to 0.2083%.

A new method is proposed to eliminate impulse noise with a random intensity distribution in digital images. The method is based on impulse noise detection by means of a self-organizing tree map (SOTM) and a class of noise-exclusive adaptive filters. The SOTM classifies the image pixels into clusters and locates the cluster centers that represent the mean intensities of the impulse noise. Based on the detected impulse noise, the noise-exclusive filters are applied only to the corrupted pixels, using the true neighborhood information to estimate the pixel values. The filtering scheme presented can suppress impulse noise effectively while preserving image edges and fine details. Experimental results demonstrate that the performance of the noise-exclusive adaptive filters are superior to that of the traditional median filter family.

We describe a novel cellular connectionist neural network model for the implementation of clustering-based Bayesian image segmentation with Gibbs random-field spatial constraints. The success of this algorithm is largely due to the neighborhood constraints modeled by the Gibbs random field. However, the iterative enforcement of the neighborhood constraints involved in the Bayesian estimation would generally require tremendous computational power. Such computational requirement hinders the real-time application of the Bayesian image segmentation algorithms. The cellular connectionist model proposed aims at implementing the Bayesian image segmentation with real-time processing potentials. With a cellular neural network architecture mapped onto the image spatial domain, the powerful Gibbs spatial constraints are realized through the interactions among neurons connected through their spatial cellular layout. This network model is structurally similar to the conventional cellular network. However, in this new cellular model, the processing elements designed within the connectionist network are functionally more versatile to meet the challenging needs of Bayesian image segmentation based on the Gibbs random field. We prove that this cellular neural network does converge to the desired steady state with a properly designed update scheme. Examples of CT volumetric medical image segmentation are presented to demonstrate the potential of this cellular neural network for a specific image segmentation application.

A Markov random field (MRF) approach to image segmentation is described. Unlike most previous MRF techniques, which are based on pixel classification, this approach groups pixels that are similar. This removes the need to know the number of image classes. Mean field theory and multigrid processing are used in the subsequent optimization to find a good segmentation and to alleviate local minimum problems. Variations of the MRF approach are investigated by incorporating features/schemes motivated by characteristics of the human vision system (HVS). Experimental results are promising and indicate that multigrid and HVS-based features/schemes can improve segmentation results.

One of the major drawbacks of current acoustically based speech recognizers is that their performance deteriorates drastically with noise. Our focus is to develop a computer system that performs speech recognition based on visual information concerning the speaker. The system automatically extracts visual speech features through image-processing techniques that operate on facial images taken in a normally illuminated environment. To cope with the dynamic nature of change in speech patterns with respect to time as well as the spatial variations in the individual patterns, the proposed recognition scheme uses a recurrent neural network architecture. By specifying a certain behavior when the network is presented with exemplar sequences, the recurrent network is trained with no more than feedforward complexity. The network’s desired behavior is based on characterizing a given word by well-defined segments. Adaptive segmentation is employed to segment the training sequences of a given class. This technique iterates the execution of two steps. First, the sequences are segmented individually. Then, a generalized version of dynamic time warping is used to align the segments of all sequences. At each iteration, the weights of the distance functions used in the two steps are updated in a way that minimizes a segmentation error. The system is implemented and tested on a few words. The results are satisfactory. In particular, the system is able to distinguish between words with common segments. Moreover, it tolerates to a great extent variable-duration words of the same class.

The problem of limited angle tomography in which a complete sinogram is not available is considered. This situation arises in many practical applications where tomographic projection over 180 deg is either physically unrealizable or infeasible. When a complete sinogram is not available, it is well known that the reconstructed images using common reconstruction algorithms, such as convolution back-projection (CBP), will have severe streak artifacts. We present a linear artificial neural network to extrapolate the missing part of the sinogram. Once the complete sinogram is obtained via extrapolation, standard reconstruction techniques such as CBP can be used to generate artifact-free reconstructions. The parameters of the neural network are designed using the sampling theory of signals with noncompact spectral support, the knowledge that complete sinograms have bow-tie-shaped spectral support and regularization. It is found that once designed, these parameters are data independent, especially for images of similar nature. For a sinogram with 2N angular views, each having M raysum per view, if 2L views are available, the computational requirement of the neural network is 4MNL only. The proposed neural network is much more efficient than other iterative algorithms, such as the method of projection onto convex sets; the Papoulis-Gerchberg algorithm; and the Clark-Palmer-Lawrence interpolation method; which requires computations of the order of kMNL, where k is the number of iterations.

Accurate length estimation is very helpful for the successful segmentation and recognition of connected digit strings, in particular, for an off-line recognition system. However, little work has been done in this area due to the difficulties involved. A length estimation approach is presented as a part of our automatic off-line digit recognition system. The kernel of our approach is a neural network estimator with a set of structure-based features as the inputs. The system outputs are a set of fuzzy membership grades reflecting the degrees of an input digit string of having different lengths. Experimental results on National Institute of Standards and Technology (NIST) Special Database 3 and other derived digit strings shows that our approach can achieve an about 99.4% correct estimation if the best two estimations are considered.

We use the "magic TV" network with the maximum a posteriori (MAP) criterion to restore a space-dependent blurred image. This network provides a unique topological invariance mechanism that facilitates the identification of such space-dependent blur. Instead of using parametric modeling of the underlying blurred image, we use this mechanism to accomplish the restoration. The restoration is reached by a self-organizing evolution in the network, where the weight matrices are adapted to approximate the blur functions. The MAP criterion is used to indicate the goodness of the approximation and to direct the evolution of the network.

There has been some recent interest in using machine learning techniques as part of pattern recognition systems. However, little attention is typically given to the validity of the features and types of rules generated by these systems and how well they perform across a variety of features and patterns. We focus on such issues of validity and comparative performance using two different types of decision tree techniques. In addition, we introduce the notion of including legal perturbations of objects in the training set and show that the performance of the resulting classifiers was better than that those trained without such "legal" constructs in the data selection.

A computational model for recognition of three-dimensional (3-D) man-made objects based on visual perception principles is presented. In this approach, object faces are used as perceptual entities in accordance with the perceptual phenomena of face-shape constancy and face-pose consistency. Face-shape constancy is modeled by continuous maps in which topological proximity is the major consideration. To accommodate 3-D space knowledge on viewpoint consistency, a perceptual view space is developed in which face-pose consistency from an arbitrary view point can be deduced qualitatively.

A knowledge-based system that uses one pass only for thinning is described. The study of existing thinning algorithms reveals that appropriate thinning techniques are application dependent. Thinning algorithms suitable for applications such as Chinese characters unfortunately leave extraneous pixels (branches) in other applications such as Arabic characters. Alternatively, existing algorithms that delete extraneous pixels in certain applications may delete important pixels in other applications. We present a general system where the user can select the degree of freedom of having branches in the output pattern according to his or her application. While the user selects one of three possible options, the system default is suitable for most usual applications, such as English characters. Our system is a knowledge-based system whose rules guarantee connectivity and preserve the symbol shape to a great extent. The implemented system can be used as a general thinning method for various applications. By having some knowledge about the input symbols, the user determines the best thinned output pattern or the degree of branches allowed. The implemented system helps to test the effect of each rule separately. Prolog built-in features for backtracking, strong logic handling, and rule manipulation prove to be very helpful in implementing this system.

An optimal filter estimates an unobserved ideal image from an observed image. Optimality is with respect to some error criterion, which is mean absolute error for the binary images. Both the ideal and observed images are random sets and these are governed by parameterized probability laws. The optimal filter is found relative to these laws and depends on the parameters. Qualitatively, a filter is robust if its performance is acceptable for modest changes in the laws; that is, if a filter is optimal for a given vector of parameters, it performs close to optimal for parameter vectors close to the one for which it has been designed. This problem is crucial for practical application, since filters will always be applied to image processes that deviate from design processes. We provide analytic expressions for measuring filter robustness by first expressing the optimal filter analytically in terms of conditional probabilities of template transitions under the degradation operator transforming the ideal to the observed random set. Tractable expressions are derived for sparse-noise models. Robustness surfaces are computed for various text models: independent sparse noise, sparse edge noise, noise under rotation, and restoration of rotation. We show that even when the noise is independent of the signal, the signal plays an important role in robustness.

A new algorithm for image sequence coding with a very low bit rate and high picture quality is proposed. The proposed coding system is composed of interframe and intraframe coding. In the interframe coding, an edge-based motion estimation algorithm is used to find the motion vector of a moving block instead of using the conventional block-matching algorithm. The edge-based method reduces the entropy of the estimated motion vectors within a frame. Since the motion vectors are usually encoded by entropy coding, bit rate is also reduced. In the intraframe coding, a region-based algorithm is used to trace the contour of an object and estimate its parameters from a predetermined database of various geometric shapes. It can reduce the blocking effect. By testing several image sequences such as "Miss America" and "Claire" with 10 frames/s in quarter common intermediate format (QCIF) the proposed coding system has about 1 to 2 dBs higher peak SNRs than the H.263 system.

This PDF file contains the editorial “Special Section on the 1997 Visual Communications and Image Processing Award Papers” for JEI Vol. 7 Issue 01

We propose a novel method of arbitrarily focused image generation using multiple differently focused images. First, we describe our previously proposed select and merge method for all focused image acquisition. We can get good results by using this method but it is not easy to extend this method for generating arbitrarily focused images. Then, based on the assumption that depth of a scene changes stepwise, we derive a formula for reconstruction between the desired arbitrarily focused image and multiple acquired images; we can reconstruct the arbitrarily focused image by iterative use of the formula. We also introduce coarse-to-fine estimation of point spread functions (PSFs) of the acquired images. We reconstruct arbitrarily focused images for a natural scene. In other words, we simulate virtual cameras and generate images focused on arbitrary depths.

The rate constrained block matching algorithm (RCBMA) jointly minimizes displaced frame difference (DFD) variance and entropy, or conditional entropy of motion vectors for determining the motion vectors. It is intended for use in low rate video coding applications, where the contribution of the motion vector rate to the overall coding rate might be significant. The DFD variance versus motion vector rate performance of RCBMA employing size K x K blocks is shown to be superior to that of the conventional minimum distortion block matching algorithm (MDBMA) employing size 2K x 2K blocks. Constraining of the entropy or conditional entropy of motion vectors in RCBMA results in smoother and more organized motion vector fields than those output by MDBMA. The motion vector rate of RCBMA can also be precisely controlled for each frame by adjusting a single parameter.

Despite the widespread experience with block-based video coders, there is little analysis or theory that quantitatively explains the effect of block size on encoding bit rate, and ordinarily the block size for a coder is chosen based on empirical experiments on video sequences of interest. In this work, we derive a procedure to determine the optimal block size that minimizes the encoding rate for a typical block-based video coder. To do this, we analytically model the effect of block size and derive expressions for the encoding rates for both motion vectors and difference frames as functions of block size. Minimizing these expressions leads to a simple formula that indicates how to choose the block size in these types of coders. This formula also shows that the best block size is a function of the accuracy with which the motion vectors are encoded and several parameters related to key characteristics of the video scene, such as image texture, motion activity, interframe noise, and coding distortion. We implement the video coder and use our analysis to optimize and explain its performance on real video frames.

Gabor schemes of multiscale image representation are useful in many computer vision applications. However, the classic Gabor expansion is computationally expensive due to the lack of orthogonality of Gabor functions. Some alternative schemes, based on the application of a bank of Gabor filters, have important advantages such as computational efficiency and robustness, at the cost of redundancy and lack of completeness. In a previous work we proposed a quasicomplete Gabor transform, suitable for fast implementations in either space or frequency domains. Reconstruction was achieved by simply adding together the even Gabor channels. We develop an optimized spatial-domain implementation, using one-dimensional 11-tap filter masks, that is faster and more flexible than Fourier implementations. The reconstruction method is improved by applying fixed and independent weights to the Gabor channels before adding them together. Finally, we analyze and implement, in the spatial domain, two ways to incorporate a high-pass residual, which permits a visually complete representation of the image.

Mathematical morphology is a general theory that studies the decomposition of operators between complete lattices in terms of some families of simple operators: dilations, erosions, antidilations, and antierosions. Nowadays, this theory is largely used in image processing and computer vision to extract information from images. The KHOROS system is an open and general environment for image processing and visualization that has become very popular. One of the main characteristics of KHOROS is its flexibility, since it runs on standard machines, supports several standard data formats, uses a visual programming language, and has tools to help the users to build in and install their own programs. A set of new programs can be organized as a subsystem called a toolbox. We present MMach, a fast and comprehensive mathematical morphology toolbox for the KHOROS system dealing with 1-D and 2-D grayscale and binary images. Each program that is applicable to grayscale and binary images has specialized algorithms for each of these data types, and these algorithms are chosen automatically according to the input data. Several examples illustrate applications of the toolbox in image analysis.

We present a neural network based image restoration approach which makes use of multiple regularization parameters within the same image to achieve adaptive regularization. We derived an unsupervised learning scheme that estimates the appropriate parameter value for each pixel site based on information provided by the dynamics of the neural network. This scheme is based on the principle of adopting small regularization parameters for the highly textured regions to emphasize the details, while using large regularization parameters for the smooth regions to suppress the more visible noise and ringing in those regions. A secondary parameter update process is incorporated after the primary image gray values update process, to determine the appropriate local parameter at each image pixel. The variables required in the adaptation equations of the auxiliary neuron arise as byproducts of computational results of the primary neuron. As a result, the determination of the regularization parameter only involves local computation and no explicit evaluation of the underlying cost function is required. The current algorithm was applied to a number of real world images. The results correspond closely to our original expectation in that the algorithm automatically adopts small regularization parameters for the highly textured regions while maintaining high parameters for smooth regions, thus resulting in an overall pleasing appearance for the restored images.

Multiplicative noise is a type of signal-dependent noise where brighter areas of the images appear noisier. A popular class of image restoration methods is based on local mean, median, and variance. However, simple 333 filters do not take the nonstationary nature of the image and/or noise into account, and the restoration achieved by such filters may not be effective. We present a new adaptive-neighborhood or region-based noise filtering technique for restoring images with multiplicative noise. The method is based on finding variable-shaped, variable-sized adaptive neighborhoods for each pixel in the image, followed by the application of a filter specifically designed for multiplicative noise based on statistical parameters computed over the adaptive neighborhoods. From a visual inspection of restored images, it is clear that the proposed adaptive-neighborhood filter provides greater noise suppression than fixed-neighborhood restoration methods. The proposed method, unlike fixed-neighborhood methods, does not blur or clip object boundaries or corners. The mean squared errors between the results of the proposed method and the original images are considerably lower than those for results of the fixed-neighborhood methods studied, indicating that the image and noise statistics are better estimated by the adaptive-neighborhood method.

We address the problem of adapting the functions controlling the material properties of 2-D snakes, and show how introducing oriented smoothness constraints results in a novel class of active contour models for segmentation, which extends standard isotropic inhomogeneous membrane/thin-plate stabilizers. These constraints, expressed as adaptive L² matrix norms, are defined by two second-order symmetric and positive definite tensors that are invariant with respect to rigid motions in the image plane. These tensors, equivalent to directional adaptive stretching and bending densities, are quadratic with respect to first- and second-order derivatives of the image luminance, respectively. A representation theorem specifying their canonical form is established and a geometrical interpretation of their effects is developed. Within this framework, it is shown that by achieving a directional control of regularization such nonisotropic constraints consistently relate the differential properties (metric and curvature) of the deformable model with those of the underlying luminance surface, yielding a satisfying preservation of image contour characteristics. In particular, this model adapts to nonstationary curvature variations along image contours to be segmented, thus providing a consistent solution to curvature underestimation problems encountered near highcurvature contour points by classical snakes evolving with constant material parameters. Optimization of the model within continuous and discrete frameworks is discussed in detail. Finally, accuracy and robustness of the model are established on synthetic images. Its efficacy is further demonstrated on 2-D MRI sequences for which comparisons with segmentations obtained using classical snakes are provided.

We introduce the linearized perturbation method to find the center of a set of concentric circular fringes. The mathematical foundations of the method presented here are instrumental in the design of a two-dimensional interferometric position sensor, which uses the geometry of the fringe pattern rather than the wavelength. A wavelength-independent physical characteristic of the fringe pattern is derived, which relates the radii of successive fringes. The linearized perturbation method is formulated for multiple fringes as a constrained quadratic optimization problem. Experimental calibrations have shown that using physical constraint information significantly improves the precision on the center position.

This PDF file contains the editorial “Book Rvw: Digital Video: An Introduction to MPEG-2," by Barry G. Haskell, Atul Puri, and Arun N. Netravali for JEI Vol. 7 Issue 01

This PDF file contains the editorial “Digital Video: An Introduction to MPEG-2” for JEI Vol. 7 Issue 01