Regular Articles

Real-time hyperspectral processing for automatic nonferrous material sorting

[+] Author Affiliations
Artzai Picón

Tecnalia Research and Innovation, ICT/ESI, Parque Tecnológico Ed 202, E48170, Zamudio, Spain

Ovidiu Ghita

Dublin City University, Centre for Image Processing and Analysis, School of Electronic Engineering, Glasnevin, Dublin, Ireland D-9

Aranzazu Bereciartua

Tecnalia Research and Innovation, ICT/ESI, Parque Tecnológico Ed 202, E48170, Zamudio, Spain

Jone Echazarra

Tecnalia Research and Innovation, ICT/ESI, Parque Tecnológico Ed 202, E48170, Zamudio, Spain

Paul F. Whelan

Dublin City University, Centre for Image Processing and Analysis, School of Electronic Engineering, Glasnevin, Dublin, Ireland D-9

Pedro M. Iriondo

University of the Basque Country, Systems Engineering and Automatics Department, E48013, Bilbao, Spain

J. Electron. Imaging. 21(1), 013018 (Apr 04, 2012). doi:10.1117/1.JEI.21.1.013018
History: Received June 16, 2011; Revised January 27, 2012; Accepted February 10, 2012
Text Size: A A A

Open Access Open Access

Abstract.  The application of hyperspectral sensors in the development of machine vision solutions has become increasingly popular as the spectral characteristics of the imaged materials are better modeled in the hyperspectral domain than in the standard trichromatic red, green, blue data. While there is no doubt that the availability of detailed spectral information is opportune as it opens the possibility to construct robust image descriptors, it also raises a substantial challenge when this high-dimensional data is used in the development of real-time machine vision systems. To alleviate the computational demand, often decorrelation techniques are commonly applied prior to feature extraction. While this approach has reduced to some extent the size of the spectral descriptor, data decorrelation alone proved insufficient in attaining real-time classification. This fact is particularly apparent when pixel-wise image descriptors are not sufficiently robust to model the spectral characteristics of the imaged materials, a case when the spatial information (or textural properties) also has to be included in the classification process. The integration of spectral and spatial information entails a substantial computational cost, and as a result the prospects of real-time operation for the developed machine vision system are compromised. To answer this requirement, in this paper we have reengineered the approach behind the integration of the spectral and spatial information in the material classification process to allow the real-time sorting of the nonferrous fractions that are contained in the waste of electric and electronic equipment scrap.

Figures in this Article

The advent of modern hyperspectral imaging modalities opened the possibility of implementing a large spectrum of applications that cannot be robustly solved using the standard trichromatic [red, green, blue (RGB)] data. Among many possible applications that were well served by the inclusion of hyperspectral imaging systems (such as remote sensing, biomedical imaging, industrial inspection, etc.),14 the accurate recognition of nonferrous materials represents a prominent example.512 The motivation behind the decision to use hyperspectral information in the nonferrous material sorting process is multifold, where the chief reason being the fact that the color information is not sufficiently descriptive to robustly sample the characteristics associated with each nonferrous fraction. This issue proved to be particularly apparent in our study as the different fractions of nonferrous materials show considerable intraclass dispersion and a relative high interclass similarity in the RGB domain.

As opposed to the standard RGB image-formation process, hyperspectral imaging involves the acquisition and interpretation of multidimensional digital images that are able to sample the properties of the imaged materials in a substantially wider spectral domain than that covered by the visible spectrum.12,13 In this regard, it is worth mentioning that the current range of hyperspectral sensors are able to simultaneously capture hundreds of spectral bands from ultraviolet to far infrared with a spectral resolution of less than 10 nm.14,15 Thus, the main characteristic of the hyperspectral images is that each pixel is defined by a large vector whose elements are the spectral components captured from the light arriving at the spectral sensor.4,13,1619 While there is no doubt that the availability of hyperspectral data is opportune in the classification process, it is important to note that the process of capturing a large number of spectral bands results in a high dimensional data that may raise substantial challenges from practical and computational perspectives. In addition to the extensive dimensionality of the hyperspectral data, the substantial correlation between the closely spaced spectral bands may represent another issue that may affect the accuracy of the material classification process. Thus it is clear that the inclusion of the unprocessed hyperspectral data in the classification process is not only suboptimal, but also this approach will compromise the real-time operation of the developed machine vision system.

To answer the computational constrains and to avoid the Hughes phenomenon,9,15 a large number of studies have been devoted to address the optimal decorrelation of the hyperspectral data, where the ultimate objective has been the reduction of the feature vectors that describe the materials that define the scene objects. To this end, principal component analysis (PCA),1 linear discriminant analysis (LDA),20 wavelet decomposition,2 and uniform band design (UBD),21 were previously used for hyperspectral decorrelation. While these methods proved efficient when applied to diverse practical scenarios, they have the disadvantage of having to require a substantial level of user intervention during the training stage, and in addition these decorrelation approaches need retraining if new materials are included in the classification process. To circumvent these issues, unsupervised decorrelation techniques based on fuzzy sets were proposed,9 which proved particularly efficient when applied to material classification tasks. The experimental results reported in 9 indicate that the application of decorrelation not only improves the computational efficiency, but also increases the accuracy of the classification process. Our prior investigations10 revealed that the pixel-wise features calculated after the application of data decorrelation proved insufficient to accurately model the within-class variability associated with the nonferrous fractions.9 This inadequacy of the pixel-wise features was particularly exacerbated by the specular properties and the various levels of oxidization that are characteristic for the nonferrous materials contained in the waste of electric and electronic equipment (WEEE) scrap. A robust solution resides in the combination of the spectral and spatial features with the aim of generating a descriptor that is able to factor in not only the spectral properties of the imaged materials, but also their textural characteristics as well.4,9,15,2224 One approach toward spectral-spatial integration was proposed by Mercier and Lennon,25 where the authors attempted to model the texture in the hyperspectral domain by computing the marginal distribution of the wavelet coefficients using generalized Gaussian density (GDD). Other approaches implemented texture decomposition using Gabor filters,26 directional filter banks,27 Markov random fields,28 or analyzed the texture using statistical schemes based on the calculation of co-occurrence matrices.29 While these methods have shown promising results when applied to various practical applications, their major limitation is the onerous computational overhead associated with the calculation of the spectral-spatial descriptors (SSDs), a fact that rendered them as unfeasible when deployed in the development of real-time vision systems.

In a recent paper9 we detailed a novel framework where the textural and spectral features are integrated using local distributions, a concept that is well suited for sampling the spectral descriptors that are characteristic for nonferrous materials. The use of local distributions for nonferrous material sorting proved opportune from a classification performance standpoint (accuracy >98%), but although the computational complexity of the proposed algorithm was substantially lower than that associated with standard hyperspectral texture analysis methods,2629 it was still too high to be feasible for real-time operation. Thus, the major goal of this paper is to redesign the process behind the calculation of the local distributions in the decorrelated hyperspectral domain that was presented in 9 with the objective of reducing the computational time to a level that offers real-time operation when the nonferrous material-sorting algorithm is included in the development of a conveyor-based WEEE recycling system. Another important objective in our research was to attain the real-time operation while maintaining the classification accuracy at a similar level when compared to the original implementation.

This paper is organized as follows. In Sec. 2, an overview of the WEEE recycling process is provided. Section 3 details the proposed material classification algorithm and its real-time implementation. Section 4 describes and discusses the experimental results, while Sec. 5 concludes the paper.

The most recent statistics indicate that the WEEE constitutes 4% of the total municipal waste in Europe, and it is increasing by 16 to 28% every five years.5,30 Although certain sectors of the electrical and electronic equipment (EEE) market show signs of stagnation (e.g., TV sets and large kitchen appliances), others, including information and telecommunication equipment, car electronics, and electronic toys, still experience a robust growth. In this context, it is useful to note that the European economic area (EEA) countries generate 6.5 million tons of WEEE per annum, and currently approximately 90% of this potentially hazardous waste is disposed as unsorted in generic municipal landfills. According to the current EU statistics, the WEEE is expected to increase to 12 million tons by 2015, and, as a consequence, legislation that sets specific requirements in regard to WEEE collection and recycling has been recently introduced.31 While the introduction of strict targets toward WEEE recycling had a direct impact on EEE manufacturers, as they had to adjust their environmental policies, it also opened an opportunity for companies that are active in the WEEE recycling sector.

Prior to the application of various technological recycling processes, the WEEE is shredded to allow the separation of its constituent parts. After mechanical, electrostatic, and densiometric sorting,18,32 a distinct component of the WEEE scrap is formed by nonferrous materials (stainless steel, aluminum, copper, zinc, brass, and lead) that cannot be sorted by standard mechanical recycling procedures. This fact has negative economic implications as the unsorted nonferrous materials are sold at a much lower price than the individual nonferrous fractions. Thus the availability of new technologies that would allow the robust sorting of nonferrous materials will increase considerably the profitability of the overall WEEE recycling process. To achieve this goal, the classification of the nonferrous materials has been initially carried out using the RGB data. These approaches proved inefficient when applied to the separation of several fractions of nonferrous materials (such as stainless steel, aluminum, and zinc),8 and as a result new methodologies that perform the classification in the hyperspectral domain were actively explored.9,24 While hyperspectral classification techniques proved accurate when applied to nonferrous sorting, they proved challenging when included in the development of industrial systems due to their high computational demand. To provide an insight into this issue, the specifications for WEEE recycling outlined in the SORMEN European project documentation30 indicate that a sorting speed of one ton/h is required to justify the automatic recycling process from an economical perspective. This translates into a processing speed of 50 m/min (approximately 170 camera lines/s). The algorithms based on pixel-wise classification schemes can meet this computational constraint, but they are not able to properly model the interclass variations associated with different nonferrous materials. As indicated in the introductory section of this paper, the solution to this problem resides in the integration of the spectral and spatial information in the classification process, but there are substantial computational challenges that need to be overcome. In the following sections of the paper we will detail a real-time nonferrous material classification algorithm that is able to attain the processing speed required for an industrial recycling system, while maintaining the classification accuracy at a level above 96%.

System Overview

In order to devise a flexible machine vision solution, we have adopted a modular approach. In this regard, the proposed system consists of several computational modules that include the hyperspectral image acquisition and nonferrous material classification, and mechanical subcomponents that implement the WEEE vibratory delivery system, particle transport module (conveyor), and the pneumatic particle sorting mechanism. The logical arrangement of the constituent modules of the developed machine vision solution is illustrated in Fig. 1, and in the remainder of the paper we will be primarily focused on the description and analysis of the nonferrous material classification algorithm, which is the core component of the system. Details about all mechanical components of the developed machine vision solution can be found in 24.

Graphic Jump LocationF1 :

Schematic of the nonferrous material sorting system.

The computational stages associated with the proposed real-time material classification algorithm can be summarized as follows:

  1. Hyperspectral data decorrelation
  2. Background removal and nonferrous particle labeling
  3. Data quantization and the calculation of the SSD for each nonferrous particle
  4. Material classification.

For clarity, Fig. 2 presents a block diagram that describes the complete overview of the proposed real-time material classification algorithm. In this diagram, arrows illustrate the logical links between the distinct computational components of the proposed nonferrous classification algorithm. It is useful to mention that the main computational bottleneck is associated with the calculation of the SSD for each nonferrous particle (see the shaded component in Fig. 2). Thus the main objective of our work was to devise an optimized approach for the robust integration of spectral-spatial information to allow the implementation of a real-time nonferrous material-sorting system. In this sense, the new approach constructs a single descriptor for each nonferrous particle, as opposed to the approach detailed in 9, which requires the calculation of fuzzy spectral-spatial distributions for each pixel in the hyperspectral image and the application of computationally complex procedures relating to region merging and reclassification. Full details about each computational component shown in Fig. 2 will be provided in the remainder of this paper.

Graphic Jump LocationF2 :

Outline of the real-time nonferrous material classification algorithm.

Hyperspectral Data Decorrelation

In the first step the unprocessed hyperspectral image is subjected to data decorrelation using an unsupervised approach based on spectral fuzzy sets.9 This approach is based on the knowledge that the spectral information varies smoothly over successive spectral bands, and as a result the characteristics associated with nonferrous materials should be sampled by groups of spectral bands rather than selective spectral bands. The data flow associated with the developed fuzzy-based decorrelation scheme is detailed in Fig. 3.

Graphic Jump LocationF3 :

Fuzzy sets-based hyperspectral decorrelation.

The developed hyperspectral decorrelation (see Fig. 4) involves the partitioning of the spectral domain in a predefined number of fuzzy sets, m, and an energy value Eq for each fuzzy set q (q[1,m]) is calculated as follows: Display Formula

Eq=i[1,k]Mfq(λi)I(λi),q[1,m],m<<k.(1)
It can be observed that the energy measure defined in Eq. (1) samples the strength of the intensity signal I(x,y)=[I1(x,y),I2(x,y),,Ik(x,y)] in each fuzzy set. Using this approach, the dimensionality of each hyperspectral pixel I(x,y) is reduced from k to m (k>>m). To provide some insight into the computational efficiency attained by the proposed data decorrelation scheme, the dimensionality of the unprocessed hyperspectral data, k, is 80 (i.e., consists of 80 spectral bands), and our earlier studies revealed that optimal classification results are obtained when the hyperspectral data is decorrelated using eight fuzzy sets (i.e., m=8) (for more details please refer to 9 where a comprehensive set of experimental results are provided).

Graphic Jump LocationF4 :

Calculation of the membership grade for the spectral band λi. In this diagram λD defines the half-width of each fuzzy set.

Background Segmentation and Particle Labeling

The next step of the classification procedure involves the segmentation of the decorrelated spectral data into background and nonferrous particles. As indicated in our earlier paper, where the mechanical design of the WEEE recycling system is outlined,24 the WEEE delivery component (vibratory feeder) was carefully built to ensure a uniform placement of the nonferrous particles onto the belt of the conveyor without any particle overlaps.

Based on the hyperspectral characteristics of the nonferrous materials, it was decided to choose a belt that is coated with matte black material. The main reason behind this decision was to maximize the contrast in the hyperspectral domain between the nonferrous materials and the background information (conveyor belt) and to reduce as much as possible the occurrence of specular reflections due to the adherence of the small WEEE particles to the conveyor belt. By taking advantage of these characteristics, a sole decorrelated energy vector component (i.e., the energy component whose central wavelength corresponds to 510 nm) proved sufficient for robust background segmentation using an experimentally determined threshold. However, when the recycling system has been operated in an industrial environment, we have discovered that vertical scratches occurred on the conveyor belt due to the friction with nonferrous particles (see Fig. 5). Nonetheless, the occurrence of these vertical scratches on the conveyor belt has negative effects on the background segmentation process, as illustrated in Fig. 5(b), and to robustly eliminate these undesired features we devised a filtering approach in the Fourier domain by masking the contribution of the frequencies that are generated by the vertical scratches as shown in Fig. 5(d) and Eq. (2). Display Formula

F(u,v)=I(u,v)H(u,v)H(u,v)={0if(u,v)Ω1otherwise,(2)
where u,v are the spatial frequencies, I(u,v) is the input image after the application of the two-dimensional (2-D) fast Fourier transform (FFT), H(u,v) denotes the Fourier filter, Ω defines the section of the Fourier spectrum associated with vertical scratches, and F(u,v) is the filtered image. The filtered image F(u,v) is converted to the spatial domain using the inverse FFT, and the image resulting from the background segmentation process [see Fig. 5(f)] is subjected to a computationally optimized labeling procedure. Figure 6 illustrates the image resulting from the labeling process.

Graphic Jump LocationF5 :

Background segmentation process. (a) Input image where vertical scratches due to the friction between nonferrous particles and conveyor’s belt are visible. (b) Background segmentation. (c) Fourier spectrum of image (a). (d) Fourier filtering (c). (e) Inverse Fourier transform (d). (f) Background segmentation.

Graphic Jump LocationF6 :

Nonferrous particle labeling.

Calculation of the Particle Spectral-Spatial Descriptor

After the application of the particle segmentation and labeling process, the next step involves the calculation of the spectral descriptor for each nonferrous particle. In our initial studies we have analyzed the feasibility of using pixel-wise spectral information for material classification, but the experimental results indicated a classification success rate of only 71.52% when dealing with six nonferrous fractions (aluminium, copper, brass, lead, stainless steel, and white copper). This classification accuracy clearly indicates that the spectral information provided by a single pixel is not sufficient to robustly model the significant intraclass dispersion caused by the various levels of oxidization that are characteristic for each nonferrous material. Another important factor that increased the intraclass dispersion was generated by the specular properties of the nonferrous materials and by the shadows caused by the three-dimensional (3-D) geometrical profile of each particle. While the issues inserted by the various oxidization levels can be (at least theoretically) modeled in a statistical sense, the incidence of the problems caused by the specular highlights and shadows cannot be predicted as they are caused by external factors. To alleviate the problems introduced by shadows and highlights we normalized the intensity of the decorrelated spectral information as illustrated in Eq. (3) (for more details regarding this hyperspectral intensity normalization scheme the reader can refer to 33). Display Formula

Enorm(λj)=En(λj)mini[1,m][En(λi)],(3)
where En(λj)=E(λj)i=1mE(λi)and m denotes the number of fuzzy sets that is a parameter of the spectral decorrelation scheme detailed in Sec. 3.1.

However, as indicated earlier, the most important problem we confronted is the inability of the pixel-wise spectral features to properly sample the characteristics of the nonferrous materials, and a solution to this problem resides in the integration of the spatial and spectral features in a composite descriptor. Using this concept, the spectral characteristics of the nonferrous materials are sampled by the distribution of the spectral features that are calculated over a region in the image. In the context of the application detailed in this paper, we propose to calculate the distribution of the spectral-spatial information using a fuzzy-based method similar to the approach used for data decorrelation (see Sec. 3.1). This approach is motivated by the fact that several factors such as image noise, uneven illumination, and different levels of oxidization of the nonferrous materials induce undesired and unpredictable changes in the calculation of the SSDs. To alleviate these issues we mapped each component of the normalized spectral energy vectors Ei into a new fuzzy-based representation. The process required to calculate the fuzzy distributions (histograms) entails two steps that are depicted in Figs. 7 and 8. In the first step, the intensity of each energy vector Ei, i[1,m] is subjected to a quantization procedure where each spectral component is projected into a higher dimensional vector space as illustrated in Fig. 7. In this process, p fuzzy sets are employed from which only two will return a value different than zero. In the second step, the SSD is calculated as shown in Fig. 8. As indicated in Fig. 8, for each pixel (x,y) the quantized vectors Qi for each energy Ei are concatenated into a pixel-wise feature vector Q(x,y), while the SSD for a given region R is calculated as follows, Display Formula

SSD(R)=(x,y)RQ(x,y).(4)
The calculation of the SSD, as illustrated in Figs. 7 and 8, has several advantages that are well suited for the application targeted in this paper. First, the vector quantization procedure applied to measure the response of each energy Ei(x,y) allows the construction of soft histograms that avoid the disadvantages associated with crisp binning (i.e., as in the case when the spectral-spatial histograms would be calculated using the Ei values). This is particularly obvious in the case of material classification, as the properties of different nonferrous fractions vary smoothly within the hyperspectral domain. Second, another important advantage resides in the fact that the compound SSD involves the simple summation of the quantized spectral response that is calculated for each pixel in the region of interest R, which opens the opportunity of real-time operation. While the nonferrous particles have various sizes, prior to the classification process, each component (bin) of the SSD is normalized with respect to the number of pixels contained in the region R.

Graphic Jump LocationF7 :

The fuzzy-based vector quantization procedure (p=6 fuzzy sets are applied to sample the decorrelated hyperspectral domain).

Graphic Jump LocationF8 :

Calculation of the spectral-spatial descriptor (SSD).

Multivariate Gaussian Classifier

The histogram shown in Eq. (4) defines the distribution of the spectral energies within the area encompassed by a nonferrous particle, and this information is used as input for classification. In our implementation, the multivariate Gaussian classifier16 is proposed to perform the material classification since the dispersion of the spectral-spatial features within each class of nonferrous materials can be well approximated with normal distributions [the multivariate Gaussian classifier is optimal in the Bayes sense if the relationships between the input vectors that characterize each nonferrous material class, and the output class variables can be accurately modeled by multivariate normal distributions, as illustrated in Eq. (5)]. In this regard, a multivariate Gaussian model is created for each nonferrous material where μ and Σ are the mean vector and the covariance matrix of the modeled class (in the training stage, 7×7SSD descriptors have been used to sample the spectral characteristics for each nonferrous material). Display Formula

N(x|μ,Σ)=1(2·π)D/21|Σ|1/2e[12(xμ)TΣ1(xμ)].(5)
The classification stage is carried out by checking each spectral-spatial vector calculated for each particle SSD (Rs), s being the number of particles resulting after the segmentation and labeling processes, with respect to the normalized distributions that define each material class in the training process (μc,Σc), where c denotes the number of classes. Each particle in the decorrelated hyperspectral image is labeled to the class that achieves the best matching cost as shown in Eq. (6).Display Formula
P[SSD(Rs)ci]=maxciN(x|μci,Σci).(6)

The developed machine vision system consists of four distinct components: the particle feeding device, conveyor, material classification, and particle separation mechanism. The shredded WEEE mixture is automatically loaded onto a nonspecular black conveyor belt (600 mm wide) via a vibratory feeder that has been specifically designed to ensure the nonferrous materials are arranged into a thin layer prior to their arrival at the inspection line. The conveyor speed was set at 20 m/min, and the particle separation mechanism was implemented using a pneumatic part-extractor.

The WEEE mixture is defined by six nonferrous materials: white copper, aluminum, stainless steel, brass, copper, and lead (see Fig. 9). The material samples have been provided by Indumetal Recycling S.A. and IGE Hennemann Recycling GmbH, both members of the SORMEN project consortium.30 The nonferrous materials have been manually sorted by expert operators, and full information was provided about each nonferrous waste fraction. In this study the captured datasets were divided into training and testing sets, where half of the data was used for training, and the remaining half was used for testing. From each of these datasets more than 500,000 SSDs were extracted for validation purposes.

Graphic Jump LocationF9 :

The nonferrous materials investigated in this study.30

Table 1 shows the experimental results obtained by the nonferrous material sorting system detailed in this paper. The processing times reported in Table 1 were obtained when the proposed algorithm, and the algorithms described in 9, were implemented using the Microsoft Visual C++ environment (Windows XP) and executed on a standard PC that is fitted with an Intel Core 2 Duo 2.4-GHz processor and 2-GB RAM. To emphasize the computational advantages associated with the real-time classification technique presented in Sec. 3, we also provide the computational time attained by the classification method for which the SSD is calculated in predefined neighborhoods for each pixel in the image.9 As opposed to the approach presented in 9 where the classification process involves N class assignments, N being the number of pixels present in the image, in the implementation detailed in this paper the classification of each nonferrous particle resulting after segmentation and labeling (see Sec. 3.2) involves a single class membership assignment using Eq. (6). This is illustrated in Eq. (7) where we show the computational cost entailed by the method discussed in 9 (Cneighborhood) and the new approach detailed in Sec. 3 (CR). In Eq. (7) f1 defines the computational cost associated with the extraction of the vector Q(x,y) (see Fig. 8), M is the size of the neighborhood (see Table 1), f2 is the marginal computational cost associated with the class assignment for the unknown SSD, as indicated in Eq. (6), and c denotes the number of classes. Display Formula

Cneighbourhood=f1×M2×N×f2×N×cCR=f1×R×f2×c.(7)
The experimental results depicted in Table 1 confirm the considerable decrease in computational speed attained by the proposed approach, which allows the processing of more than two linear meters of WEEE particles per second at the conveyor speed (20 m/min). In particular, we would like to stress that the decrease in computational cost has been obtained only with a marginal drop in classification accuracy (96.87%, nonferrous classification approach detailed in this paper, and 98.36%, the neighborhood-based nonferrous material classification method described in 9, a fact that recommends the proposed solution for real-time material sorting).

Table Grahic Jump Location
Table 1Classification results.

The major objective of this paper was to detail the development of an industrial compliant nonferrous material sorting system. In this paper we have reviewed the practical issues associated with the WEEE recycling process, and we have analyzed the challenges that have to be addressed during the development of an integrated nonferrous material sorting system when operated in an industrial environment. Among many practical and theoretical challenges, the computational efficiency and the accuracy of the nonferrous classification technique are critical requirements that justify the automatic recycling process from an economical standpoint. To answer these requirements, in this paper we have introduced a redesigned classification scheme in the hyperspectral domain, where the main novelty resides in the optimized computational approach that allows the real-time calculation of the SSDs in the context of nonferrous material sorting. The experimental results reveal that the proposed machine vision system is able to process the nonferrous shredded WEEE scrap at a rate of 2.28 m/s with a classification accuracy of 96.87%. The performance attained by our system exceeds the economic thresholds required for automatic WEEE nonferrous material sorting, and currently the developed machine vision system is fully evaluated in an industrial recycling environment.

We would like to thank Tecnalia, Specim and SORMEN project consortium for providing the data and the nonferrous materials that have been used in the validation of the proposed algorithm.

Feather  B. K. et al., “Compression technique for plume hyperspectral images,” in Algorithms and Technologies for Multispectral, Hyperspectral and Ultraspectral Imagery XI. ,  SPIE ,  Orlando, FL  (2005).
Kempeneers  P. et al., “Generic wavelet-based hyperspectral classification applied to vegetation stress detection,” IEEE Trans. Geosci. Rem. Sens.. 43, (3 ), 610 –614 (2005). 0196-2892 CrossRef
Mesina  M. B., de Jong  T. P., Dalmijn  W. L., “Automatic sorting of scrap metals with a combined electromagnetic and dual energy X-ray transmission sensor,” Int. J. Miner. Process.. 82, (4 ), 222 –232 (2007). 0301-7516 CrossRef
Tso  B., Olsen  R. C., “Scene classification using combined spectral, textural and contextual information,” Proc. SPIE. 5425, , 135 –146 (2004).CrossRef
Bereciartua  A., Echazarra  J., “Sistema basado en identificación multiespectral para la separación de metales no férricos en WEEE en logísitica inversa,” in 1er Congreso de Logística y Gestión de la Cadena de Suministro. (2007).
Gesing  A. J., “ELVs: how they fit in the global material recycling system and with technologies developed for production or recycling of other products and materials,” in  6th International Automobile Recycling Congress ,  Amsterdam, Netherlands  (2006).
Koyanaka  S., Kobayashi  K., “Incorporation of neural network analysis into a technique for automatically sorting lightweight metal scrap generated by ELV shredder facilities,” Res. Cons. Recyc.. 55, (5 ), 515 –523 (2011). 0921-3449 CrossRef
Kutila  M., Viitanen  J., Vattulainen  A., “Scrap metal sorting with colour vision and inductive sensor array,” in  Computational Intelligence for Modelling, Control and Automation ,  Vienna, Austria , pp. 725 –729 (2005).
Picón  A. et al., “Fuzzy spectral and spatial feature integration for classification of nonferrous materials in hyperspectral data,” IEEE Trans. Ind. Infor.. 5, (4 ), 483 –494 (2009).
Picón  A. et al., “Biologically inspired data decorrelation for hyper-spectral imaging,” EURASIP J. Ad. .Sig. Proc.. (2011).doi:10.1186/1687-6180-2011-66
Spencer  D. B., “The high-speed identification and sorting of nonferrous scrap,” JOM J. Min., Met. Mat. Soc.. 57, (4 ), 46 –51 (2005).
Sommer  E. J., Ross  C. E., Spencer  D. B., “Method and apparatus for sorting materials according to relative composition,” U. S. Patent No. 7,099,433 (2006).
Grahn  H., Geladi  P. Eds., Techniques and Applications of Hyperspectral Image Analysis. ,  Wiley ,  Hoboken, NJ , ISBN-10:0-470-01086-X (2007).
Specim Spectral Imaging Ltd. http://www.specim.fi (2011).
Tarabalka  Y., Chanussot  J., Benediktsson  J., “Segmentation and classification of hyperspectral images using watershed transform,” Pattern. Recogn.. 43, (7 ), 2367 –2379 (2010). 0031-3203 CrossRef
Bishop  C. M., Pattern Recognition and Machine Learning. ,  Springer ,  New York  ISBN-10:0-387-31073-8 (2006).
Chang  C. I., Hyperspectral Imaging: Techniques for Spectral Detection and Classification. ,  Springer ,  New York , ISBN:0-306-47483-5 (2003).
Wahab  D. A. et al., “Development of a prototype automated sorting system for plastic recycling,” Am. J. Appl. Sci.. 3, (7 ), 1924 –1928 (2006). 1546-9239 
Wang  J., Chang  C. I., “Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis,” IEEE Trans. Geosci. Rem. Sens.. 44, (6 ), 1586 –1600 (2006). 0196-2892 
Martinez  A. M., Kak  A. C., “PCA versus LDA,” IEEE Trans. Pattern Anal. Mach. Intell.. 23, (2 ), 228 –233 (2001). 0162-8828 CrossRef
Pudil  P., Novovicova  J., Kittler  J., “Floating search methods in feature selection,” Pattern Recogn. Lett.. 15, (11 ), 1119 –1125 (1994). 0167-8655 CrossRef
Ghita  O. et al., “Quality grading of painted slates using texture analysis,” Comp. Ind.. 56, (8–9 ), 802 –815 (2005). 0166-3615 CrossRef
Manian  V., Jimenez  L. O., “Land cover and benthic habitat classification using texture features from hyperspectral and multispectral images,” J. Electron. Imag.. 16, (2 ) (2007). 1017-9909 CrossRef
Picón  A. et al., “Automation of waste recycling using hyperspectral image analysis,” in  IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) ,  Bilbao, Spain , pp. 1 –4, (2010).
Mercier  G., Lennon  M., “On the characterization of the hyperspectral texture,” in  IEEE Interational Geoscience and Remote Sensing Symposium (IGARSS) ,  Toronto, Canada , pp. 2584 –2586, (2002).
Shi  M., Healey  G., “Hyperspectral texture recognition using a multiscale opponent representation,” IEEE Trans. Geosci. Rem. Sens.. 41, (5 ), 1090 –1095 (2003). 0196-2892 
Hong  P. S., Kaplan  L. M., Smith  M. J., “Hyperspectral image segmentation using filter banks for texture augmentation,” in IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data. ,  Greenbelt, MD 254 –258 (2003).
Hazel  G., “Multivariate Gaussian MRF for multispectral scene segmentation and anomaly detection,” IEEE Trans. Geosci. Rem. Sens.. 38, (3 ), 1199 –1211 (2000). 0196-2892 CrossRef
Soh  L. K., Tsatsoulis  C., “Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices,” IEEE Trans. Geosci. Rem. Sens.. 37, (2 ), 780 –795 (1999). 0196-2892 CrossRef
SORMEN: Innovative Separation Method for Nonferrous Metal Waste from Electric and Electronic Equipment (WEEE) based on Multi- and Hyperspectral Identification project, Sixth Framework Programme Horizontal Research Activities Involving SMES Co-Operative Research, http://www.sormen.org/ (2006).
Directive 2002/96/EC of the European Parliament and of the Council of January 27, 2003, on waste electrical and electronic equipment (WEEE): Joint declaration of the European Parliament, the Council and the Commission relating to Article 9.
Hearn  G. L., Ballard  J. R., “The use of electrostatic techniques for the identification and sorting,” Res. Cons. Recyc.. 44, (1 ), 91 –98 (2005). 0921-3449 CrossRef
Stockman  H., Gevers  T., “Detection and classification of hyper-spectral edges,” in  British Machine Vision Conf. (BMVC) ,  Nottingham, UK , pp. 643 –651 (1999).

Artzai Picón received a MEng. degree in industrial engineering (2002) and MRes. degree in automatics electronics and control (2005) and a PhD (2009) from the University of the Basque Country, Spain. In 2000–2002, he collaborated with the Computer Image Processing Group (GTI2) in the Department of Systems Engineering and Automatics. Since then, he has been working as a researcher for Tecnalia Research Corporation where he has been involved in over 30 research projects. His research interests include image classification and segmentation and the application of these technologies to industrial tasks. In 2002 he received the Best MEng. Final Project Award from Accenture Company for the implementation of a facial recognition system; in 2006 he received the ONCE International Research and Development Award in New Technologies for the Blind and Visually Impaired.

Ovidiu Ghita received BE and ME degrees in electrical engineering from Transilvania University Brasov, Romania, and a PhD from Dublin City University, Ireland. From 1994 to 1996 he was an assistant lecturer in the Department of Electrical Engineering at Transilvania University. Since then he has been a member of the Vision Systems Group at Dublin City University (DCU), and currently he holds a position of DCU-Research Fellow. She has authored and co-authored over 90 peer-reviewed research papers in areas of instrumentation, range acquisition, machine vision, texture analysis, and medical imaging.

Aranzazu Bereciartua received her physics degree from the University of the Basque Country, Spain (1999), and her electronics engineer degree from the University of the Basque Country, Spain (2000). Since 2000 she has been employed by Fundación Robotiker (now Tecnalia Research & Innovation), when she joined the Image Processing Group as a software developer. Since 2005 she holds the position of project manager in this company. She has 11 years’ experience in machine vision, image processing algorithms, sensors and advanced automation, artificial intelligence and intelligent interfaces. She has managed and developed applications from the conception to the final installation stage at users’ sites, with particular contributions in system design, SW development, and HW & SW integration. She has taken part in many local, national, and European projects during which time she contributed to one patent and authored several publications in the field of machine vision. Currently, she is working toward the completion of her doctoral thesis in biomedical image processing.

Jone Echazarra received her electrical engineer degree from the Engineering School of Bilbao (1989). In 1990 she joined Philips CFT (Eindhoven, Netherlands) where she worked within the EU project ESPRIT 2017: Automated process and assembly inspection by 3D-Vision. Since 1991 she has been employed by Fundación Robotiker (now Tecnalia Research & Innovation) where in addition to the researcher position within the Computer Vision Group she also serves as project and market manager. She has contributed to several conferences in the field of computer and machine vision, and she served as a coordinator for several EU projects.

Paul F. Whelan received his BEng. (Hons) degree from NIHED, MEng. from the University of Limerick, and his PhD (computer vision) from the University of Wales, Cardiff, UK (Cardiff University). In 1985 to 1990 he was employed by Industrial and Scientific Imaging Ltd. and later Westinghouse (WESL), where he was involved in the research and development of high-speed computer vision systems. He was appointed to the School of Electronic Engineering, Dublin City University (DCU), in 1990 and is currently professor of computer vision (personal chair). He founded the Vision Systems Group in 1990 and the Centre for Image Processing & Analysis in 2006 and currently serves as its director. As well as publishing over 150 peer-reviewed papers, he has co-authored two monographs and co-edited three books. His research interests include image segmentation and its associated quantitative analysis with applications in computer/machine vision and medical imaging. He is a senior member of the IEEE, fellow of IET, chartered engineer, and a member of IAPR. He served as a member of the governing board (1998 to 2007) of the International Association for Pattern Recognition (IAPR), a member of the International Federation of Classification Societies (IFCS) council and president (1998 to 2007) of the Irish Pattern Recognition and Classification Society (IPRCS). Prof. Whelan is a HEA-PRTLI (RINCE, NBIP) funded principal investigator.

Pedro M. Iriondo received a degree in industrial engineering in 1981 and began his professional activity working in different companies. In 1986 he founded the company Adicorp dedicated to the research, integration, and development of industrial vision systems. In 2001 he received a PhD in industrial engineering—electronics and control engineering—from the University of the Basque Country UPV-EHU, Spain. Since 1989 he has been an associate professor with the Department of Automatic Control and Systems Engineering, ETSI of Bilbao, University of the Basque Country UPV-EHU. Currently, he is the assistant director of the School of Engineering of Bilbao, and he is responsible for relations with private companies. His research interest is focused in the fields of image processing, face detection and recognition, image transmission and stochastic systems.

© 2012 SPIE and IS&T

Citation

Artzai Picón ; Ovidiu Ghita ; Aranzazu Bereciartua ; Jone Echazarra ; Paul F. Whelan, et al.
"Real-time hyperspectral processing for automatic nonferrous material sorting", J. Electron. Imaging. 21(1), 013018 (Apr 04, 2012). ; http://dx.doi.org/10.1117/1.JEI.21.1.013018


Figures

Graphic Jump LocationF1 :

Schematic of the nonferrous material sorting system.

Graphic Jump LocationF2 :

Outline of the real-time nonferrous material classification algorithm.

Graphic Jump LocationF3 :

Fuzzy sets-based hyperspectral decorrelation.

Graphic Jump LocationF4 :

Calculation of the membership grade for the spectral band λi. In this diagram λD defines the half-width of each fuzzy set.

Graphic Jump LocationF5 :

Background segmentation process. (a) Input image where vertical scratches due to the friction between nonferrous particles and conveyor’s belt are visible. (b) Background segmentation. (c) Fourier spectrum of image (a). (d) Fourier filtering (c). (e) Inverse Fourier transform (d). (f) Background segmentation.

Graphic Jump LocationF6 :

Nonferrous particle labeling.

Graphic Jump LocationF7 :

The fuzzy-based vector quantization procedure (p=6 fuzzy sets are applied to sample the decorrelated hyperspectral domain).

Graphic Jump LocationF8 :

Calculation of the spectral-spatial descriptor (SSD).

Graphic Jump LocationF9 :

The nonferrous materials investigated in this study.30

Tables

Table Grahic Jump Location
Table 1Classification results.

References

Feather  B. K. et al., “Compression technique for plume hyperspectral images,” in Algorithms and Technologies for Multispectral, Hyperspectral and Ultraspectral Imagery XI. ,  SPIE ,  Orlando, FL  (2005).
Kempeneers  P. et al., “Generic wavelet-based hyperspectral classification applied to vegetation stress detection,” IEEE Trans. Geosci. Rem. Sens.. 43, (3 ), 610 –614 (2005). 0196-2892 CrossRef
Mesina  M. B., de Jong  T. P., Dalmijn  W. L., “Automatic sorting of scrap metals with a combined electromagnetic and dual energy X-ray transmission sensor,” Int. J. Miner. Process.. 82, (4 ), 222 –232 (2007). 0301-7516 CrossRef
Tso  B., Olsen  R. C., “Scene classification using combined spectral, textural and contextual information,” Proc. SPIE. 5425, , 135 –146 (2004).CrossRef
Bereciartua  A., Echazarra  J., “Sistema basado en identificación multiespectral para la separación de metales no férricos en WEEE en logísitica inversa,” in 1er Congreso de Logística y Gestión de la Cadena de Suministro. (2007).
Gesing  A. J., “ELVs: how they fit in the global material recycling system and with technologies developed for production or recycling of other products and materials,” in  6th International Automobile Recycling Congress ,  Amsterdam, Netherlands  (2006).
Koyanaka  S., Kobayashi  K., “Incorporation of neural network analysis into a technique for automatically sorting lightweight metal scrap generated by ELV shredder facilities,” Res. Cons. Recyc.. 55, (5 ), 515 –523 (2011). 0921-3449 CrossRef
Kutila  M., Viitanen  J., Vattulainen  A., “Scrap metal sorting with colour vision and inductive sensor array,” in  Computational Intelligence for Modelling, Control and Automation ,  Vienna, Austria , pp. 725 –729 (2005).
Picón  A. et al., “Fuzzy spectral and spatial feature integration for classification of nonferrous materials in hyperspectral data,” IEEE Trans. Ind. Infor.. 5, (4 ), 483 –494 (2009).
Picón  A. et al., “Biologically inspired data decorrelation for hyper-spectral imaging,” EURASIP J. Ad. .Sig. Proc.. (2011).doi:10.1186/1687-6180-2011-66
Spencer  D. B., “The high-speed identification and sorting of nonferrous scrap,” JOM J. Min., Met. Mat. Soc.. 57, (4 ), 46 –51 (2005).
Sommer  E. J., Ross  C. E., Spencer  D. B., “Method and apparatus for sorting materials according to relative composition,” U. S. Patent No. 7,099,433 (2006).
Grahn  H., Geladi  P. Eds., Techniques and Applications of Hyperspectral Image Analysis. ,  Wiley ,  Hoboken, NJ , ISBN-10:0-470-01086-X (2007).
Specim Spectral Imaging Ltd. http://www.specim.fi (2011).
Tarabalka  Y., Chanussot  J., Benediktsson  J., “Segmentation and classification of hyperspectral images using watershed transform,” Pattern. Recogn.. 43, (7 ), 2367 –2379 (2010). 0031-3203 CrossRef
Bishop  C. M., Pattern Recognition and Machine Learning. ,  Springer ,  New York  ISBN-10:0-387-31073-8 (2006).
Chang  C. I., Hyperspectral Imaging: Techniques for Spectral Detection and Classification. ,  Springer ,  New York , ISBN:0-306-47483-5 (2003).
Wahab  D. A. et al., “Development of a prototype automated sorting system for plastic recycling,” Am. J. Appl. Sci.. 3, (7 ), 1924 –1928 (2006). 1546-9239 
Wang  J., Chang  C. I., “Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis,” IEEE Trans. Geosci. Rem. Sens.. 44, (6 ), 1586 –1600 (2006). 0196-2892 
Martinez  A. M., Kak  A. C., “PCA versus LDA,” IEEE Trans. Pattern Anal. Mach. Intell.. 23, (2 ), 228 –233 (2001). 0162-8828 CrossRef
Pudil  P., Novovicova  J., Kittler  J., “Floating search methods in feature selection,” Pattern Recogn. Lett.. 15, (11 ), 1119 –1125 (1994). 0167-8655 CrossRef
Ghita  O. et al., “Quality grading of painted slates using texture analysis,” Comp. Ind.. 56, (8–9 ), 802 –815 (2005). 0166-3615 CrossRef
Manian  V., Jimenez  L. O., “Land cover and benthic habitat classification using texture features from hyperspectral and multispectral images,” J. Electron. Imag.. 16, (2 ) (2007). 1017-9909 CrossRef
Picón  A. et al., “Automation of waste recycling using hyperspectral image analysis,” in  IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) ,  Bilbao, Spain , pp. 1 –4, (2010).
Mercier  G., Lennon  M., “On the characterization of the hyperspectral texture,” in  IEEE Interational Geoscience and Remote Sensing Symposium (IGARSS) ,  Toronto, Canada , pp. 2584 –2586, (2002).
Shi  M., Healey  G., “Hyperspectral texture recognition using a multiscale opponent representation,” IEEE Trans. Geosci. Rem. Sens.. 41, (5 ), 1090 –1095 (2003). 0196-2892 
Hong  P. S., Kaplan  L. M., Smith  M. J., “Hyperspectral image segmentation using filter banks for texture augmentation,” in IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data. ,  Greenbelt, MD 254 –258 (2003).
Hazel  G., “Multivariate Gaussian MRF for multispectral scene segmentation and anomaly detection,” IEEE Trans. Geosci. Rem. Sens.. 38, (3 ), 1199 –1211 (2000). 0196-2892 CrossRef
Soh  L. K., Tsatsoulis  C., “Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices,” IEEE Trans. Geosci. Rem. Sens.. 37, (2 ), 780 –795 (1999). 0196-2892 CrossRef
SORMEN: Innovative Separation Method for Nonferrous Metal Waste from Electric and Electronic Equipment (WEEE) based on Multi- and Hyperspectral Identification project, Sixth Framework Programme Horizontal Research Activities Involving SMES Co-Operative Research, http://www.sormen.org/ (2006).
Directive 2002/96/EC of the European Parliament and of the Council of January 27, 2003, on waste electrical and electronic equipment (WEEE): Joint declaration of the European Parliament, the Council and the Commission relating to Article 9.
Hearn  G. L., Ballard  J. R., “The use of electrostatic techniques for the identification and sorting,” Res. Cons. Recyc.. 44, (1 ), 91 –98 (2005). 0921-3449 CrossRef
Stockman  H., Gevers  T., “Detection and classification of hyper-spectral edges,” in  British Machine Vision Conf. (BMVC) ,  Nottingham, UK , pp. 643 –651 (1999).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.