Special Section on Quality Control by Artificial Vision

Real-time flatness inspection of rolled products based on optical laser triangulation and three-dimensional surface reconstruction

[+] Author Affiliations
Julio Molleda

University of Oviedo, Department of Computer Science, Campus de Viesques, Gijón 33204 Asturias, Spain

Rubén Usamentiaga

University of Oviedo, Department of Computer Science, Campus de Viesques, Gijón 33204 Asturias, Spain

Daniel F. García

University of Oviedo, Department of Computer Science, Campus de Viesques, Gijón 33204 Asturias, Spain

Francisco G. Bulnes

University of Oviedo, Department of Computer Science, Campus de Viesques, Gijón 33204 Asturias, Spain

J. Electron. Imaging. 19(3), 031206 (July 14, 2010). doi:10.1117/1.3455987
History: Received July 30, 2009; Revised November 30, 2009; Accepted December 18, 2009; Published July 14, 2010; Online July 14, 2010
Text Size: A A A

Open Access Open Access

Flatness is a major geometrical feature of rolled products specified by both production and quality needs. Real-time inspection of flatness is the basis of automatic flatness control. Industrial facilities where rolled products are manufactured have adverse environments that affect artificial vision systems. We present a low-cost flatness inspection system based on optical triangulation by means of a laser stripe emitter and a CMOS matrix camera, designed to be part of an online flatness control system. An accurate and robust method to extract a laser stripe in adverse conditions over rough surfaces is proposed and designed to be applied in real time. Laser extraction relies on a local and a global search. The global search is based on an adjustment of curve segments based on a split-and-merge technique. A real-time recording method of the input data of the flatness inspection system is proposed. It stores information about manufacturing conditions for an offline tuning of the laser stripe extraction method using real data. Flatness measurements carried out over steel strips are evaluated quantitatively and qualitatively. Moreover, the real-time performance of the proposed system is analyzed.

One of the main goals of the metal industry is to fulfill the quality requirements specified by the users of each product. Geometrical features are used to estimate the quality of flat rolled products, such as steel strips or aluminum foils. Quality criteria must be evaluated before the product is delivered to the customer. This is normally done in real time during manufacturing, most of the time in adverse environments. The evaluation also serves as feedback for setting up the manufacturing process.

The final quality of a rolled product is greatly influenced by flatness. From a mathematical point of view, flatness can be defined as the degree to which the surface of a flat product approaches a plane.1 To achieve the required flatness of the rolled products, a flatness control system is used. Flatness control is divided into two tasks. The first adapts the roll gap, the bending, and the tilting of each rolling mill stand.2 The second compensates for the uneven temperature of the material during the rolling process.3 This is done by driving the mechanical and thermal actuators of the rolling mill. Figure 1 shows the feedback model of a flatness control system.

Graphic Jump LocationF1 :

Feedback model of a flatness control system.

Products with poor flatness are more likely to break during later manufacturing phases and move imprecisely along industrial facilities, harming their own surfaces. Thus, in addition to real-time quality control during manufacturing, the use of flatness control systems improves the productivity of both manufacturers and customers facilities.

In the past, the flatness inspection of large manufactured products relied on human operators. Human inspection is subjective and lacks reproducibility and repeatability. Mechanical flatness inspection systems emerged in the late 1970s, and optical systems started to be developed in the mid-1980s. Nowadays, both mechanical and optical systems are used in automatic flatness inspection systems. Flatness inspection systems can be classified as contact or noncontact, depending on whether the sensors of the system are in physical contact with the product to be inspected.

Traditional mechanical flatness inspection systems compute flatness based on radial force measurements. They use strength or pressure transducers integrated into a deflector roll, when the product moves forward along the production line. Early commercial inspection systems were based on this principle.4 However, contact-based inspection systems can not be used with very thick or very hot rolled products because the sensors of the system may be damaged. Furthermore, these systems are not suitable if high quality is required because they can cause scratching of the product surface. More modern mechanical flatness inspection systems are based on noncontact principles, such as those involving periodic excitation of the product. This excitation is generated using airflow to measure movement amplitudes across the product width.5

Optical flatness inspection systems are usually based on noncontact principles, mainly triangulation, using light sources such as white or laser light. The long lifetime of laser light sources minimizes maintenance tasks, whereas white light lamps must be frequently replaced. Moreover, the high spectral radiance of laser radiation reduces the exposure time per measurement by a factor of 4 or more, compared to white-light projectors,6 making laser-based systems the preferred choice in real-time applications.

Early optical flatness inspection systems were based on the analysis of images acquired from illuminated areas of the surface of the product. Incandescent light or laser light were used to illuminate the working area, or the scene. In the late 1980s and early 1990s, several systems based on triangulation techniques were developed. Primitive triangulation systems used discrete light points projected onto the surface of the product to be inspected. They needed moving parts in order to follow the lateral displacements of the product, as well as to relocate the light points equidistantly across the product width.79 Current optical flatness inspection systems are based on one or more laser beams projected onto the surface of the product,1011 making complete three-dimensional surface reconstruction of the product to be inspected possible. Optical flatness inspection systems can also be based on fringe pattern or structured light techniques.1214 Although in the literature, these techniques are treated as separate methods; both are particular cases of the same principle of active triangulation.15

Other noncontact techniques recently used to measure the flatness of rolled products involve stereoscopic principles, using X-ray gauges16 and capacitance principles, using capacitive distance sensor arrays.17

Both mechanical- and optical-based systems are currently in use for online flatness inspection of rolled products. The main drawbacks of mechanical systems are their high maintenance requirements and the long periods of time needed for calibration. The main drawback of optical systems is the high cost of the machine vision subsystem.

In this work, a low-cost, real-time flatness inspection system based on the triangulation principle, which uses a laser beam projected as a stripe onto the surface of the product to be inspected, is proposed. An accurate and robust method to extract the laser stripe from the images acquired by a matrix camera is designed to work over rough surfaces in adverse environments. The robustness of the laser stripe extraction method makes the inspection of rolled products under variable luminance conditions possible without having the machine vision system enclosed or housed. Moreover, a procedure for tuning the laser stripe extraction method using real data from industrial facilities is proposed.

Flatness measurements are computed during manufacturing using several parameters of rolled product geometry. Section 2 describes how flatness of a rolled product is defined and measured, presents common flatness defects and their causes, and introduces the indices used for expressing the magnitude of flatness defects in industry. Section 3 presents the most frequently used nondestructive machine vision methods used to obtain 3-D information about the surface of rolled products. Section 4 describes the quality inspection system for real-time flatness measurement of rolled products proposed in this paper. The testing procedure and experimental results are discussed in Sec. 5, and Sec. 6 analyzes the compliance of the real-time requirements of the proposed system. Finally, Sec. 7 contains the concluding remarks.

Flatness is one of the main geometrical features specified by manufacturers and end users of rolled products for an efficient process control and quality assurance. In this section, steel strips are used to explain this feature. The flatness of a steel strip is commonly expressed as deviation indices between the surface of the strip and a horizontal flat surface. When a strip is perfectly flat, all its points make contact with the flat surface.

Steel strips are manufactured from large, hot slabs that are passed through several successive rolling stands in a rolling mill, reducing their thickness and increasing their length. Afterward, the steel strips are cooled. Flatness defects are mainly caused during rolling and cooling. Other processes that influence steel strip flatness are coiling, uncoiling, cutting, and lifting. Flatness defects during rolling are caused by irregularly shaped sections of the incoming material, as well as nonuniform pressure applied to the entire transversal section of the strip. Flatness defects during cooling are produced by uneven temperature across the width of the strip.

Flatness defects in steel strips are quantified by considering the strip as multiple, adjacent longitudinal fibers, as presented in Fig. 2. When all the fibers of the strip achieve the same elongation during the rolling process, the final rolled strip is perfectly flat. If each fiber could be elongated independently, the geometry of a steel strip with flatness defects would be as shown in Fig. 2. However, in a real scenario, the differences between the elongations of the fibers of a steel strip generate waves, as shown in Fig. 2.

Graphic Jump LocationF2 :

Longitudinal fibers of a steel strip: (a) Strip without irregularities during rolling, (b) theoretical defects, and (c) real flatness defects.

Common flatness defects are bad leveling, wavy edges, and center buckle, as presented in Fig. 3. Bad leveling is generated when the length of the fibers increases progressively from one longitudinal edge of the strip to the other. When the length of the fibers at the edges of the strip is longer than the length of the fibers at the center, waves appear at both edges, generating flatness defects called wavy edges. If the length of the fibers at the center of the strip is longer than the length of the rest of the fibers then a center buckle defect appears. Other flatness defects include quarter buckle and herringbone.

Graphic Jump LocationF3 :

Common steel strip flatness defects: (a) Bad leveling, (b) wavy edges, and (c) center buckle.

Flatness defects of steel strips are quantitatively expressed by means of several flatness indices, such as I-units (international units) and steepness.18 The I-unit index is calculated usingDisplay Formula

1I-unit=ΔLjLref105=LjLrefLref105,
where Lref is the wavelength of a reference fiber of the strip, fref, and ΔLjLref is the elongation of the fiber fj with regard to the reference fiber, as shown in Fig. 4. The steepness index is calculated usingDisplay Formula
2Steepness=Hλ102,
where H is the peak-to-peak wave amplitude and λ is the fiber wavelength. Although these two indices are in use, the most commonly used index is the I-unit.

Graphic Jump LocationF4 :

Computation of flatness indices and flatness profiles.

Because the I-unit index expresses the relation between the elongation of a fiber of the strip with regard to a reference fiber, a flatness profile can be obtained by computing the I-unit index for each fiber, that is, the distribution of the relative elongation of the fibers.

Because rolled products may be hundreds or thousands of meters long, flatness inspection systems compute flatness profiles at fixed longitudinal sections of the strip, called integration intervals. These sections may be partially overlapped along the length of the strip, generating a higher density of profiles for the same strip. Figure 4 shows an example of integration intervals overlapping; the number of flatness profiles computed using the intervals i1 and i2, which generate flatness profiles fpi1 and fpi2, is doubled by using the intervals i1 and i2, generating the sequence of flatness profiles fpi1, fpi1, fpi2, fpi2. Figure 5 illustrates several flatness profiles computed from a steel strip during manufacturing. The analysis of the flatness profile provides the type and magnitude of the flatness defects.19

Graphic Jump LocationF5 :

Flatness profiles computed from a steel strip during manufacturing: (a) Bad leveling, (b) wavy edges, and (c) center buckle and wavy edges.

The sequence of flatness profiles computed along a strip can be represented in a 2-D image, commonly called a flatness map. Figure 6 shows an example of a flatness map computed during the manufacturing of the same steel strip from which the flatness profiles of Fig. 5 were obtained.

Graphic Jump LocationF6 :

Flatness map computed from a steel strip during manufacturing.

Flatness profiles are the main output of flatness inspection systems of rolled products. Hence, measurements of the length of the fibers at several fixed longitudinal sections of the rolled product are necessary. The acquisition of these measurements depends on the technique used to obtain the 3-D information of the surface of the strip.

Section 3 describes the artificial vision techniques used to obtain the 3-D reconstruction of the surface of an object, that is, obtaining the 3-D surface map of rolled products during manufacturing.

Flatness inspection systems require 3-D information of the surface of the rolled products to compute the length of their fibers. Many different optical techniques have been proposed for measuring 3-D shape.20 Three-dimensional surface measurements, obtained as depth information of the scene, are stored as a collection of distance measurements from a known reference coordinate system to target surface points.15 This collection results in a digital image, known as a range image. On the basis of 3-D optical techniques, several machine vision methods have been developed for nondestructive determination of geometrical features of objects. Usually, 3-D machine vision methods are classified into two groups, passive and active.

In computer vision, passive 3-D machine methods are known as shape-from-X techniques, where X stands for different 3D cues such as stereo, shading, motion, texture, and silhouettes. The most widely known passive method is shape from stereo or stereovision,21 but it is somewhat limited to obtain 3-D surface maps, due to the correspondence problem.22 The correspondence problem can be alleviated by active 3-D machine methods.

Active 3-D machine vision methods project light patterns onto the surface of the object to be inspected, whereas passive methods only require the environmental light of the scene. Active methods include techniques such as triangulation, moiré interferometry, and holographic interferometry. Although all these techniques simplify the mathematical solution of the correspondence problem, triangulation is the most commonly used. Holographic interferometry provides the highest accuracy of the active 3-D machine vision methods (1010m); however, this degree of accuracy is unnecessary for flatness inspection of rolled products, making the cost of holographic equipment unjustifiable.

Active triangulation, mainly laser-based, is the technique most commonly used in flatness inspection systems of rolled products. Laser-based flatness inspection systems can use spots, single stripes, or color-encoded stripes projected onto the surface of the product to be inspected. Spot and stripe-based systems are slower but more robust than those based on encoded stripe schemes, which are more prone to error.

Spot-based flatness inspection systems reconstruct the surface of products using the same number of fibers as make up the single-point subsystems. These systems achieve greater accuracy than single stripe systems, also known as slit scanners, as the whole sensor can be optimized for a given value of measurement.23 However, the projection of a laser stripe onto the surface of the product to be inspected makes the analysis of the continuous deformation of the profile possible because the number of fibers is limited only by the width (in pixels) of the camera sensor.

The accuracy of online 3-D reconstruction of the surface of rolled products depends on a set of cross-related issues, such as calibration, camera resolution, optics distortion, noise, and environmental conditions. Section 4 describes the real-time laser-based flatness inspection system proposed in this paper and how these issues are taken into account to achieve the accuracy required.

The quality inspection system proposed in this paper is a flatness inspection system based on the noncontact optical triangulation principle. The system is designed to inspect flatness of steel strips in real time and can also be applied to inspect the flatness of many other metallic and nonmetallic rolled products. The real-time constraint of this system is the time in which flatness must be measured and sent to the flatness-control actuators for the actions of the feedback to take effect. The system consists of a laser emitter, which projects a laser stripe transversally onto the surface of the strip to be inspected. The optical sensor is a CMOS matrix camera centered over the roll path of the production line, as is shown in Fig. 7.

Graphic Jump LocationF7 :

Flatness inspection system layout.

The region of interest (ROI) of the scene is determined by both the width of the roll path and the vertical fluctuation of the laser stripe due to the presence of flatness defects in the rolled product. The computation of the laser stripe projected onto the product surface, that is, the surface profile, makes it possible for the system to inspect flatness.

The triangulation angle, which is a major factor in optical triangulation performance, is determined by the dimensions of the ROI of the scene, the dimensions of the camera sensor, and the vertical fluctuation of the rolled product surface. General purpose slit scanners use a 30-deg triangulation angle. However, certain systems use a different angle in order to meet customer requirements.24 In general, as the triangulation angle increases, the sensor range decreases and resolution increases. Because of the form factor of the ROI of the scene in the proposed system, the triangulation angle is only limited by sensor occlusions. The optimal triangulation angle for the proposed flatness inspection system is 45deg, which allows one to achieve the highest resolution, eliminating the likelihood of missing data due to sensor occlusions. This value is calculated based on several experiments simulating vertical fluctuations of a rolled product surface with flatness defects.

Because the system is designed to be installed in industrial environments with adverse conditions, the first phase of flatness measurement involves image filtering. Once the noise of the scene has been reduced, the laser stripe projected onto the strip surface must be extracted in order to compute the strip profile. Computing the strip profile requires two steps. First, the projection of the laser stripe must be expressed in the image coordinate system. Next, a transformation between coordinate systems is needed in order to express the strip profile in world coordinates, based on the camera model and its calibration.

Next, the flatness inspection system uses the strip profiles of the integration interval to create a 3-D reconstruction of the strip surface of the interval and to compute the flatness profile based on this reconstruction. Once the flatness profile is available, the online flatness map of the steel strip is updated.

One main feature of the system is its ability to record all the input data required for the flatness inspection of a steel strip. This allows the replication of the tests carried out on the system with identical manufacturing conditions. Therefore, this data make the tuning of the laser stripe extraction method, using real instead of synthetic data, possible.

Image Filtering

Industrial facilities where rolled products are manufactured may have adverse conditions, such as uncontrolled environmental light and hostile atmosphere with smoke, steam, humidity, and dust. Thus, the extraction of the laser stripe onto the strip surface requires a previous filtering stage over the scene to acquire the sharpest image.

This task can be performed using an optical filter, which does not require computing time, satisfying the real-time constraints of the flatness inspection system. Besides increasing the signal-to-noise ratio of the image, the optical filtering stage reduces the amount of information that does not belong to the laser projection, cutting down the posterior image-processing computational requirements.

The optical filter used for this stage is an interference filter, which is a multilayer thin-film device designed to act, in this case, as a bandpass filter. The spectral properties of this filter are based on destructive light interference, or wavelength interference, rather than absorption. Thus, a specific frequency band can be transmitted while maintaining a nearly zero coefficient of transmission of the spectral bands of no interest.

The laser emitter used in the proposed system projects a beam, with Gaussian intensity distribution, of 670±10nm wavelength. The passband of the optical filter should match this range if the light coming from the scene would incise with a normal angle. However, the horizontal field of view of the camera and the length of the projected beam onto the strip surface make the angle of incidence of the laser stripe reflected from the edges of the scene increase as much as 12deg. Thus, the central wavelength of the interference filter at normal incidence, λcwl, shifts to λθ at angle of incidence θ according toDisplay Formula

3λθ=λcwl1sin2θNϵ2,
where Nϵ is the effective refraction index of the interference filter.

To address this issue a broadband interference filter with a full width at half maximum range of 80nm is selected and tuned to the desired central wavelength. This avoids filtering the projection at the edges of the scene at the expense of admitting some noise in the image.

Laser Stripe Extraction

Laser stripe extraction is an easy task under controlled conditions of surrounding light and laser power.24 Several methods have been designed,2426 which are mainly focused on the accuracy of the laser detection tending to neglect the issue of robustness. Because they are greatly affected by noise, these methods are not suitable when the laser is projected and acquired in industrial environments. In these environments, the laser stripe extraction methods must be accurate as well as robust in order to deal with adverse situations, such as variable luminance, light reflection, uneven surfaces of the products to be inspected, and laser degradation.

Laser stripe extraction is usually carried out by means of a detection method, considering each column or row of the image as independent signals.27 The laser profile in each signal corresponds to a Gaussian profile, and thus, the laser center should correspond to the point of the signal with maximum luminance. The most common laser stripe extraction methods,2829 such as Gaussian approximation, center of mass, linear approximation, Blaix and Rioux detector, and parabolic estimator, provide the position of the center of the laser with subpixel precision, but they are highly affected by noise. Under controlled conditions, these methods provide a continuous stripe. However, if noise appears in the image, the maximum luminance may correspond to spurious noise, providing a sequence of points that describes a discontinuous stripe, with gaps if the laser is not detected in some areas. Therefore, the extraction of a laser stripe in industrial environments cannot be accomplish using only a single peak detection method.

The laser stripe extraction method proposed in this paper is based on a laser peak detection method, followed by a peak-linking method aimed to avoid issues caused by adverse conditions. Peak linking can be assimilated to edge linking in computer vision because there are no specific linking methods for laser peaks.

In the proposed flatness inspection system, the laser stripe is longer than the width of the inspected product. Thus, a section of the stripe is projected onto the product surface and the rest is projected in the roll path of the production line. Figure 8 shows the projection of the laser stripe in the roll path, that is, in the background of the scene. Figures 8 show the laser stripe projected when a rolled product is in the ROI of the scene (see Fig. 7). The laser stripe projected in the scene is called the image foreground, whereas the rest of the image is called the image background. In this paper, different methods are proposed to detect and link the laser onto the surface of the product to be inspected and the laser projected in the roll path of the production line.

Graphic Jump LocationF8 :

Images acquired by the camera of the proposed flatness inspection system: (a) Laser stripe projected in the background of the scene, (b) laser stripe projected onto the uneven surface of a rolled product showing laser degradation, and (c) laser stripe projected onto the surface of a rolled product showing spurious noise in the image.

In controlled environments, laser peak detection can be reduced as a simple thresholding to accurately obtain the laser position in each column of the image. However, in industrial environments the surrounding light can be variable, especially with the interferencial filter used in the proposed flatness inspection system, which allows more noise at the center of the scene to pass to the camera sensor in certain circunstances (see Fig. 8). Therefore, the luminance of both the foreground and background can vary considerably between images acquired in industrial environments. Figure 9 shows the luminance of a column of three different images in which the laser stripe is projected with different environmental conditions. There is no single threshold value that can be used to differentiate the laser peak from the rest of the scene in the three images at the same time. Dynamic thresholding methods may be applied to obtain the optimal threshold,30 but they can only be used in cases of a strong or weak laser because these methods will fail when the surrounding light is variable.

Graphic Jump LocationF9 :

Laser peak detection: (a) Luminance of different laser profiles and (b) proposed laser peak detection method.

Center of mass is the method that provides the best results in laser stripe detection,2829 but when applied to an image, it assumes that the laser stripe must be found in each column. In industrial environments, this assumption is not reasonable because surrounding light, uneven surfaces, or laser degradation [see Fig. 8] can mislead the detection of the laser peak in a column. In these situations, the proposed method leaves the laser stripe undetected in the affected columns of the image, and afterward, based on contextual information of the laser stripe in the rest of the image, it fills in these gaps by applying the laser peak-linking method.

The proposed laser peak detection method considers each column of the image as an independent signal, as do the methods proposed in the literature cited above, and searches for the maximum point of luminance of each column. Then, the center of mass is calculated in a small area around it, minimizing the effects of reflection or surrounding light. If the difference between the light intensity of the maximum and the light intensity of the rest of the column (the background of the column) is not significant, then the proposed method considers that laser detection cannot be accomplished in this column of the image, leaving a gap in the detected stripe. A fast and robust approach is applied to estimate the light intensity in the background of the column, as the average of a set of pixels around the laser, except for the area used to search the maximum luminance. An example of the proposed laser detection method is shown in Fig. 9, applied to the luminance of a column of an image where the laser stripe is projected.

Extraction of the laser stripe projected in the roll path

The projection of the laser stripe in the background of the scene does not depend on the surface conditions of the products to be inspected. Thus, it can be characterized to be used as contextual information of the scene. This characterization consists of a curve-fitting process, by the least-squares method applied to the sequence of points obtained by the laser peak detection in each column of the image. The type of curve used in this fitting is a quadratic polynomial y(x)=Ax2+Bx+C. The characterization of the laser projected in the background can be carried out off line and needs not be recomputed unless the background of the scene is modified.

The proposed laser peak detection method is not applied to all the pixels of each column of the image. Contextual information of the scene is used to apply the laser detection only in the selected areas in which the laser is most likely to be found, reducing the computing time consumed in the extraction process. The proposed laser stripe extraction method extracts both the laser stripe projected in the roll path and the laser stripe projected onto the surface of the rolled product, combining the laser peak detection proposed previously with linking processes.

The laser in the roll path is extracted by means of a linking process based on local search in order to fill the possible gaps produced by the laser peak detection. The characterization of the laser stripe projected on the background of the scene is used for driving the local linking process. This process starts searching simultaneously from both sides of the image and moves to the center using trapezoidal searching areas, as represented in Fig. 10. The searching areas are initially located at coordinate y(0) of both the right and left columns of the image. The laser extraction in the roll path finishes when there is no laser peak detected in either searching area, determining the limits of the laser stripe projected in the background of the scene.

Graphic Jump LocationF10 :

Laser stripe extraction: (a) Laser stripe projected onto a rolled product with the areas used in local and global search, (b) laser peak detection onto the surface of a rolled product, (c) laser peak linking using Akima splines, and (d) laser stripe extracted from the surface of a rolled product.

The laser projected onto the surface of the rolled product is extracted by means of a linking process based on global processing. For this purpose, the search area is defined by both contextual information and the results of the local searching process. The laser stripe projected onto the product is found above the laser stripe projected on the roll path within the limits identified by the local search. Furthermore, as the laser stripe is emitted from a single-point laser device, the right and left limits of the laser projected on the roll path can be linked to the laser focus, expressed in image coordinates, reducing the width of the global searching area, as shown in Fig. 10. Once the global searching area of the image is determined, the proposed laser peak detection method is applied, providing the sequence of peaks shown in Fig. 10.

Extraction of the laser stripe projected onto the surface of the rolled product

Noise in the global searching area may produce spurious points and gaps in the sequence of points provided by the laser peak detection process. Therefore, a linking process based on a global search through a split-and-merge segmentation method is proposed to accurately extract the laser stripe onto the surface of the rolled product. The proposed linking process is divided into three stages.

The first stage is based on the iterative end-point fit procedure,31 which considers a set, s0, containing all the detected peaks and fits a line to the set. The peak with the maximum distance to the line, Pd, is detected and compared to a threshold, ϵ. If the distance is greater than the threshold, then the set of peaks s0 is split at point Pd into s1 and s2 sets, then fitting the new sets of peaks. This procedure finishes when all the peaks are fitted under the threshold value. The choice of this value depends on the strength of the laser stripe projected onto the surface of the rolled products and is calculated as a trade-off between the fitting precision and number of segments obtained in the fitting procedure. To apply this procedure, the type of curve used to fit each set of peaks must be defined. Several experiments have been carried out in order to determine the best type of curve to fit the laser stripe onto the surface of rolled products. Although linear and quadratic curves can be used, Akima splines32 have proved to be the most stable to the outliers. In addition, Akima splines fit the laser stripe most accurately with a similar computational cost as linear or quadratic curves. As shown in Fig. 10, this stage provides a sequence of splines, representing the continuous laser stripe in the whole global searching area, without the gaps provoked by the laser peak detection. Nevertheless, noisy areas are fitted using small splines.

The second stage of the proposed linking process based on global search is designed to filter the noise provided by the first stage, removing the splines that fit spurious points and noisy areas. The splines to be removed can be determined by several criteria. The proposed method removes those shorter than a threshold value because the noise is fit with small splines.

Finally, the third stage is aimed to fill any gaps generated by the previous filtering stage. The removed splines are replaced by new splines. The new splines are computed using the points of the removed splines in which the fitting error with adjacent splines agree with the fitting precision, ϵ. In the worst case, the new spline fits the edges of the adjacent splines of the removed splines. The result provided by this stage is the sequence of points that represents the projection of the continuous laser stripe onto the surface of the rolled product, as shown in Fig. 10.

Transformation of Coordinate Systems and Camera Calibration

The laser stripe extraction method provides the position of the projected beam onto the strip surface in the 2-D–coordinate system of the image. However, taking metric measurements of the laser stripe requires this information in the 3-D–coordinate system of the scene or world. Thus, a translation between image and world coordinate systems is required. The calibration of the camera, which makes the translation of the coordinate systems possible, is divided into two tasks: modeling the formation of the image in the camera and computing the parameters of the model.

The mathematical camera model is based on the approximation of the internal geometry as well as the position and orientation of the camera in the scene. Several camera models have been defined.3335 The simplest are based on linear transformations, whereas the most complicated are based on nonlinear transformations, which include camera lens distortions in the model. Camera modeling determines a set of parameters that explains the physical and optical behavior of the camera. Regardless of the camera model, two kinds of parameters are involved in the model: intrinsic parameters, which describe the projection of the light onto the sensor, and extrinsic parameters, which determine the position and orientation of the camera in the scene.

The second task of camera calibration is to compute the optimal values of the parameters determined by the camera model. Therefore, the camera calibration depends on the camera model used. The parameters of linear models, such as the calibration method proposed by Hall et al. ,36 are estimated using the least-squares method. In nonlinear models, such as the calibration methods proposed by Tsai,33 Zhang,34 or Heikkilä and Silven,35 the estimation of the parameters is obtained by means of a two-stage technique. The first stage estimates an initial guess of the parameters by solving an overdetermined system of linear equations. The second stage is a nonlinear optimization process by means of iterative methods, such as the standard Levenberg–Marquardt algorithm. Calibration methods using nonlinear camera models provide higher accuracy than those using liner models.3738

The relationship between world and image coordinate systems can also be established by means of a fast methodology, such as the approach proposed by Marzani et al. ,39 rather than using a classical calibration method.

The camera of the flatness inspection system proposed in this paper is calibrated following the nonlinear method proposed by Heikkilä and Silven.35 Other methods,37 such as that developed by Tsai, provide slightly better results. However, empirical evaluations of the Heikkilä method show that the effort required to achieve the high level of accuracy of the Tsai method is excessive.38

In the camera calibration method proposed by Heikkilä and Silven,35 the mapping from world coordinates (XW,YW,ZW) to pixel coordinates (xp,yp) is carried out by means of four transformations. The first one translates world coordinates into camera coordinates (XC,YC,ZC), and it is computed as Display Formula

4[XCYCZC]=R[XWYWZW]+t,
where R is the rotation matrix and t is a translation vector. Both R and t express the extrinsic parameters.

The next three transformations take into account the intrinsic parameters, which are the principal point of the camera expressed in pixels, also called the image center, (cx,cy); the effective focal length, f; the image scale factor, sx; the inverse of the center-to-center distances between adjacent pixels in x and y directions, Dx and Dy; the radial distortions, δxr and δyr; and the tangential distortions, δxt and δyt.

The second transformation relies on the pinhole projective model for computing the undistorted projection, (xu,yu), of the point (XC,YC,ZC) to the image plane asDisplay Formula

5[xuyu]=1ZC[XCYC].

The next transformation applies the distortion model for computing the distorted projection (xd,yd) in the image plane asDisplay Formula

6[xdyd]=[δxrδyr]+[δxtδyt].

Any iterative method that models lens distortion provides a similar accuracy level regardless of the kind of lens modeled.37 Modeling radial lens distortion is sufficient when high accuracy is required. Therefore, tangential distortions of the camera lens are not taken into account in the camera model. Thereby, the second addend of Eq. 6 is null in the camera calibration of the proposed flatness inspection system.

The last transformation expresses the distorted projection on the image plane in image coordinates asDisplay Formula

7[xpyp1]=[sxDx0cx0Dycy001][xdyd1].

The transformation from world coordinates to image coordinates is available when the camera model is solved. However, the transformation from image coordinates to world coordinates requires more information about the scene. Figure 7 shows that the ROI of the scene for the camera of the flatness inspection system is located in the laser plane in which the laser stripe is moved due to height variations of the strip while it is moving forward along the production line. Therefore, the transformation from the 2-D–image coordinate system to the 3-D–scene coordinate system is reduced to a 2-D–to–2-D transformation between planes, called homography. These planes are the image plane, I, and the laser plane, L. The homography is a 3×3 transformation matrix, Hil. Applying this homography, it is possible to compute for each point of the image, (xp,yp,1), the point of the region of interest of the scene, (XW,YW,1), asDisplay Formula

8[XWYW1]=Hil[xpyp1],
making the translation of the laser stripe in the image to real-world coordinates possible.

The camera calibration procedure of the flatness inspection system uses a set of checkered images with several reference points with known positions for computing the intrinsic parameters of the camera model. This pattern consists of a 12×17 matrix of alternating white and black squares, where reference points are the vertices of each square. Figure 11 shows the images acquired by the camera for this purpose.

Graphic Jump LocationF11 :

Set of images used for computing intrinsic parameters of the camera of the proposed flatness inspection system.

Images from a second pattern covering the whole ROI of the scene are acquired for computing the extrinsic parameters of the camera model. This pattern consists of a 13×4 matrix of black rhombuses, where reference points are the centroid of each rhombus. The height of the pattern covers the height of the ROI of the scene but does not cover the whole width. Thus, the pattern is moved horizontally to acquire a set of images in which the pattern covers the whole ROI of the scene. Figure 12 shows an example of the images taken for extrinsic calibration in the roll path of a steel strip production line, where the pattern is located at the center of the ROI of the scene.

Graphic Jump LocationF12 :

Example of an image used for computing extrinsic parameters of the camera of the proposed flatness inspection system.

The acquisition of both sets of images should be carried out under controlled lighting of the scene in order to easily extract with subpixel accuracy the reference points of each pattern. However, if these sets of images were acquired under uncontrolled lighting, detection methods of interest points, such as the Harris corner detector,40 could be used to increase the detection accuracy.

Table 1 shows the accuracy of the camera-calibration procedure of a prototype of the flatness inspection system proposed in this paper. The accuracy evaluation of the camera calibration is based on analyzing the discrepancy between the real position of the reference points of the pattern in the scene with respect to the 3-D position estimated from its 2-D projection. For each 2-D reference point extracted from the image, the calibrated model is used to project back, or reproject, the point over the laser plane of the scene, measuring the distance between the 3-D reference point and its reprojection.

Table Grahic Jump Location
Back-projection error.
Flatness Profile Computation

Three-dimensional surface reconstruction of a rolled product using slit scanners involves two separate stages: intraframe and interframe processing. Once a frame with the projection of the laser stripe onto the surface of the product is acquired, it must be processed in order to determine the surface profile. To this purpose, the intraframe processing is split up into three major tasks. The first task consists of extracting the laser stripe from the image and computing the strip profile expressed in the image coordinate system. The second task translates each point of the detected stripe to obtain the profile of the surface expressed into world coordinates. The third task involves software tracking to compensate for the effects of small lateral displacements while the strip is being processed. Figure 13 shows the steps carried out by the intraframe processing stage in order to compute the position of the fibers of the strip for each frame.

Graphic Jump LocationF13 :

Flatness profile computation: (a) Intraframe processing stage and (b) interframe processing stage.

The intraframe processing stage in the proposed flatness inspection system is based on both the laser stripe extraction method proposed in this paper and the camera-calibration procedure described above. The translation between coordinate systems is carried out by means of a lookup table computed during camera calibration. This stage is carried out in real time.

The interframe processing stage must use an indirect method to obtain the 3-D strip surface map because there is no information about the length of the fibers in the surface profile. The strip surface map is then used to compute the length of the fibers in each integration interval, making it possible to comptute flatness profiles.

Once the strip has moved forward by the length corresponding to an integration interval, the interframe processing stage must reconstruct the 3-D surface of the strip in this interval in order to compute a new flatness profile. Figure 13 shows the steps carried out by the interframe processing stage for each integration interval of the strip. This stage is also carried out in real time.

The information obtained from the laser beam projection onto the strip surface in the intraframe processing stage, together with the speed of the strip movement, makes the computation of the 3-D surface reconstruction of the strip possible.

The acquisition rate of the camera of the flatness inspection system is automatically adjusted, depending on the strip movement speed. The speed of the strip is measured by means of a laser surface velocimeter, an optical device for noncontact measurement of 1-D surface velocities.

Measuring the length of the fibers at fixed longitudinal sections of the 3-D surface map is the main task required for computing the flatness of the strip. The length of each fiber is computed in the interframe processing stage, once the intraframe stage has provided the position of the fibers in the scene for each image processed in the integration interval. The length is computed using an integration technique, based on both the height variations of the fiber between consecutive images and the speed of the strip while moving along the production line, defined byDisplay Formula

9Lj=i=1n(hijhi1j)2+vi2(titi1)2,
where Lj is the computed approximation of the length of the fiber fj; hij is the height of the fiber fj measured in the frame i; ti is the time stamp when the frame i was taken; ui is the average speed of the strip between the frames i and i1; and n is the total amount of height frames in the integration interval. Figure 14 shows a diagram representing the approximation carried out in this integration technique.

Graphic Jump LocationF14 :

Approximation of the length of a rolled product fiber.

Once the length of each fiber of the strip in the whole integration interval is computed, the system is able to compute a flatness profile of the strip expressed in I-units using Eq. 1. Computing a flatness profile for each integration interval of the strip makes the computation of the flatness map of the whole strip possible (see Fig. 6).

Support for Tuning the Laser Stripe Extraction Method

The replication in a laboratory of all the surface conditions of rolled products and their manufacturing environment is prohibitively costly. Thus, only a few manufacturing scenarios can be emulated in a laboratory to allow a rough tuning of the image-processing algorithms of a machine vision-based system. Fine tuning cannot be fully accomplished with frames acquired from laboratory prototypes.

To overcome this drawback, the tuning of the image-processing algorithms must be accomplished using images acquired from the real manufacturing processes in industrial facilities. However, as each rolled product is processed only once; it is not possible to evaluate the influence of tuning the image-processing algorithms.

The tuning of the laser stripe extraction method of the proposed flatness inspection system using the real conditions in a manufacturing facility involves recording all the inputs needed to compute the flatness map. The flatness inspection system uses the frames sequence provided by the camera and the strip speed measurements provided by the laser surface velocimeter of the production line to compute the flatness map of a steel strip, as shown in Fig. 15. Thus, a recording service must store in real time both the stream of images and the stream of speed values, as shown in Fig. 15. This information makes the off-line replication of real manufacturing scenarios possible, allowing fine tuning of the image-processing algorithms. This replication relies on a playback service, as illustrated in Fig. 15.

Graphic Jump LocationF15 :

Tuning of the laser stripe extraction method: (a) Flatness inspection system under working conditions, (b) video recording service for registration of real manufacturing conditions, and (c) offline tuning of the laser stripe extraction method using real manufacturing conditions.

The solution proposed for tuning the laser stripe extraction method consists of a video recording and a video playback service. The first stores the input streams of the system in a multimedia container in real time. The second retrieves the information stored in the container and plays the role of both the camera and the laser surface velocimeter of the system.

The storage size required for the data is the main limitation of recording all the input data received by the flatness inspection system during the manufacturing of a steel strip. Therefore, a lossless encoding and decoding algorithm (codec) is required. In order to determine both the optimal multimedia container and the optimal codec for the video recording service, several performance evaluation experiments have been carried out following the systematic procedure proposed by Jain.41 The performance evaluation procedure requires the selection of appropriate performance metrics and the definition of the factors that could affect these metrics. Then, a performance-evaluation experiment must be carried out for each relevant combination of the values of the factors, as summarized in Fig. 16.

Graphic Jump LocationF16 :

Factors and metrics considered in multimedia container and compression algorithm performance evaluation experiments.

The whole evaluation procedure and the results of the experiments carried out are described in detail in Ref. 42. This procedure concludes that the appropriate multimedia container for storing the input streams of the proposed flatness-inspection system is the audio video interleave container, and the optimal encoding and decoding algorithm is the zlib lossless codec.43

The testing procedure of the proposed flatness inspection system is divided into two phases: quantitative testing carried out in the laboratory and qualitative testing proposed to be carried out in the industrial facility.

Once the system is calibrated, quantitative testing is carried out using a mechanical device working as a strip emulator, which replicates the vertical movements of the surface of a rolled product presenting flatness defects when it moves forward along a production line. The magnitudes of the emulated flatness defects are known with some uncertainty, the greater the magnitude of the emulated flatness defect is, the greater the uncertainty. The system must detect the laser stripe projected onto the strip emulator in the same way as it was projected onto the surface of a rolled product, and compute the 3-D surface reconstruction of the emulated strip. Once the 3-D surface reconstruction of an integration interval of the strip is available, the system must compute a new flatness profile.

The quantitative testing of the proposed flatness inspection system consists of 40 experiments. Each experiment measures the flatness of an emulated strip 30 times. The surface of the strip is considered to be composed of 200 fibers. The experiments are classified into four groups, at a rate of 10 experiments per group, emulating low, medium, high, and very high flatness defects, respectively. Table 2 summarizes the results of these experiments, showing the mean error and error dispersion with 90% confidence for each flatness defect range evaluated. Taking all the experiments into account, the measurement error is within the uncertainty interval of the magnitude of the simulated flatness defects and the absolute error is always <10 I-units.

Table Grahic Jump Location
Quantitative testing of flatness measurement.

The second phase of the testing procedure involves an evaluation of the proposed flatness inspection system after it has been installed at the production line. In such a placement, it is not possible to carry out a quantitative evaluation because the magnitude of the flatness defects of the steel strips are not precisely known a priori. Therefore, a qualitative evaluation based on both the expert knowledge of the human operators of the industrial facility and several manual measurements is carried out after the system is installed.

The main requirements of the proposed flatness inspection system are to provide online flatness profiles of rolled products during the manufacturing process in order to serve as a basis for a real-time flatness control system, as well as generating an online representation of the flatness map of the rolled product. To achieve these requirements, the image processing and the flatness profile computation algorithms of the flatness inspection system have been designed to be applied in real time.

The deadline of the image-processing algorithms of the intraframe stage is imposed by the acquisition rate of the camera. The laser stripe in a frame must be extracted and expressed in world coordinates before a new frame is acquired. Moreover, the deadline of the flatness profile computation in the interframe stage is also imposed by the acquisition rate of the camera. Once the rolled product has moved the length corresponding to an integration interval, the system must compute the flatness profile of the integration interval before a new frame is acquired. The acquisition rate of the camera also determines the deadline for the encoding algorithm of the video recording service when it is activated.

In order to assess compliance with the real-time requirements, three separate experiments were carried out to measure the time consumed by each task mentioned above. Each experiment is repeated 30 times and consists of the execution of a specific task 100 times, and then the computation of the average and maximum computing times. The algorithms have been programmed in the C++ language and measured running in a PC equipped with and an Intel Pentium D 930 microprocessor (3.0GHz) with 4MB of cache memory and 2GB of RAM memory. The results obtained in these experiments are shown in Table 3. The computing time of the rest of the tasks of the flatness inspection system (shown in Fig. 13) is negligible.

Table Grahic Jump Location
Computing time of the main tasks of the system.

The acquisition rate of the camera determines the sampling frequency of the flatness inspection system. Because the strip speed is not constant during the whole manufacturing process, the acquisition rate of the camera is automatically adjusted in the function of the strip speed in order to sample a similar number of strip profiles per integration interval. The optimal number of strip profiles in an integration interval must be computed using simulation techniques based on both the type of rolled products and type of flatness defects the system is designed to inspect.

Typical sampling frequencies for rolled products in conventional industrial facilities varies from 30to60Hz. Thus, the deadline for each task mentioned above is near 16ms in this scenario. Therefore, as seen in Table 3, the computing time consumed by both the proposed laser stripe extraction and the proposed flatness profile computation methods are lower than the maximum time available for the frame processing. The low computational cost of the proposed laser stripe extraction method allows the acquisition rate of the camera of the proposed flatness inspection system to be as high as 250fps.

The computing time consumed by the encoding algorithm is also lower than the time available for the task, making it possible to record all the input data received by the flatness inspection system during the manufacturing of a rolled product. The selected codec makes it possible to record the input streams of the flatness inspection system in the selected container at a maximum acquisition rate of 135fps. This rate decreases to 70fps if the system is both recording the input data and measuring flatness, simultaneously.

In this paper, a real-time machine vision system for quality inspection of rolled products is proposed. The flatness inspection system has been designed to be part of a real-time quality control system.

The proposed system reconstructs the 3-D surface of rolled products based on a laser triangulation principle and uses this information to compute flatness in real time. The machine vision system uses a laser stripe emitter and a CMOS matrix camera. On the basis of the information acquired by the camera, and taking the strip speed of the rolled product into account, flatness profiles are computed in real time. This computation involves two stages. The first is an intraframe processing stage in which the laser stripe is extracted and expressed in the world coordinate system. The second is an interframe processing stage in which the 3-D surface of the rolled product is reconstructed, and then, flatness indices are computed. The tuning process of the laser stripe extraction method has been addressed using the real conditions of the industrial facility by means of a real-time video recording and playback service.

The flatness inspection system proposed in this paper uses a simple triangulation geometry, which requires minimum space in the production line as well as minimum maintenance cost. It does not require housing or an enclosure for luminance control or specific refrigeration devices, as other similar systems do, and the power consumption is as low as 3.5W plus the power consumption of the computer system in which the software is run.

An implementation of the proposed flatness inspection system has been developed and installed in a steel strip industrial facility of ArcelorMittal Steel Company, specifically in a conditioning line of the rolling plant located in Asturias, Spain. The steel strips processed in this facility have a maximum width of 1800mm, maximum length of 1600m, and maximum movement speed during manufacturing of 200mmin. Taking this information into account, and based on several simulation experiments, the optimal acquisition rate of the camera of the flatness inspection system is 40fps. This acquisition rate establishes a deadline for the critical tasks of the system of 25ms, making real-time flatness inspection possible. The qualitative tests carried out in the industrial facility proved the effectiveness of the proposed flatness inspection system, which, since its installation and calibration, the flatness of 28,000 steel strips has been successfully inspected.

This work was partially supported by European Union funds through the European Coal and Steel Community program under Contract No. ECSC/04/RFS/CR/04016. The authors thank the technicians and engineers of ArcelorMittal Asturias for their helpful assistance during tests, as well as the anonymous reviewers for their valuable comments and suggestions.

Carr  K., and Ferreira  P., “ Verification of form tolerances, Part I: basic issues, flatness, and straightness. ,” Precis. Eng..  0141-6359 17, (2 ), 131–143  ((1995)).
Qiao  J., , Guo  G., , Chai  T., , and Shao  C., “ A new method of flatness control in cold rolling process. ,” in  Proc. of American Control Conf.. , Philadelphia, PA, Vol. 6, , pp. 3828–3832  ((1998)).
Usamentiaga  R., , García  D. F., , González  D., , and Molleda  J., “ Compensation for uneven temperature in flatness control systems for steel strips. ,” in  Proc. of IEEE Industry Applications Soc. Annual Meeting. , pp. 521-527  ((2006)).
ABB Automation Technologies AB, , Switzerland, Stressometer flatness measurement systems ((2005)).
Spreitzhofer  G., , Duemmler  A., , Riess  M., , and Tomasic  M., “ Si-flat contactless flatness measurement for cold rolling mills and processing lines. ,” Rev. Metall.. 102, (9 ), 589–595 , vii–xi ((2005)).
Noll  R., and Krauhausen  M., “ Online laser measurement technology for rolled products. ,” Ironmaking Steelmaking.  0301-9233 35, (3 ), 221–227  ((2008)).
Pirlet  R., , Mulder  J., , Adriaensen  D., , Boelens  J., , and Lochen  C., “ Non-contact system for measuring hot strip flatness. ,” Iron Steel Eng.. , 60, (7 ), 45–50  ((1983)).
García  D. F., , García  M., , Obeso  F., , and Fernández  V., “ Real-time flatness inspection system for steel strip production lines. ,” Real-Time Imag..  1077-2014 5, (1 ), 35–47  ((1999)).
García  D. F., , García  M., , Obeso  F., , and Fernández  V., “ Flatness measurement system based on a nonlinear optical triangulation technique. ,” IEEE Trans. Instrum. Meas..  0018-9456 51, (2 ), 188–195  ((2002)).
López  C., , García  D. F., , Usamentiaga  R., , González  D., , and González  J. A., “ Real time system for flatness inspection of steel strips. ,” Proc. SPIE.  0277-786X 5679, , 228–238  ((2005)).
LDV Systeme, , Germany, ShapeFlex, customized non-contact flatness measurement.
Sun  J., , Zhang  J., , Su  L., , and Cao  J., “ Flatness measurement system for steel strip based on projected fringe method. ,” J. Sci. Instrum..  0950-7671 28, (4 ), 743–747  ((2007)).
Borchers  J., and Gromov  A., “ Topometric measurement of the flatness of rolled products—the system topplan. ,” Metallurgie (Halle).  0368-9573 52, (3–4 ), 247–252  ((2008)).
Paakkari  J., “ On-line flatness measurement of large steel plates using moiré topography. ,” Ph.D. thesis, University of Oulu ((1998)).
Besl  P. J.,  Advances in Machine Vision. , pp. 1–63 ,  Springer-Verlag ,  New York  ((1989)).
Jackson  R. R., , Kelly  P. E., , and Adams  D. W., “ A multi-function measure system for hot-strip mills. ,”  Millenium Steel. , Vol. 4, , pp. 1–4  ((2003)).
Lopera  J. M., , Villegas  P. J., , Linera  F. F., , Hernández-Magadan  F., , Martín-Ramos  J., , Díaz  J., , Vecino  G., , and Rendueles  J. L., “ Flatten it out: designing a low-cost system for flatness monitoring in metal processes. ,” IEEE Ind. Appl. Mag..  1077-2618 14, (2 ), 62–66  ((2008)).
AISE, ,  Hot Strip Mill Profile and Flatness Study, Phase I. ,  Association of Iron and Steel Engineers ,  Pittsburgh, PA  ((1986)).
García  D. F., , López  C., , Canga  I., , González  D., , Usamentiaga  R., , and González  J. A., “ Visualization of the flatness of steel strips during and after their manufacturing. ,” in  Proc. of IASTED Int. Conf. on Visualization, Imaging and Image Processing. , pp. 843–848 ,  ACTA Press ,  Canada  ((2003)).
Chen  F., , Brown  G. M., , and Song  M., “ Overview of three-dimensional shape measurement using optical methods. ,” Opt. Eng..  0091-3286 39, (1 ), 10–22  ((2000)).
Barnard  S. T., and Fischler  M. A., “ Computational stereo. ,” Comput. Surveys. 14, (4 ), 553–572  ((1982)).
Batlle  J., , Mouaddib  E., , and Salvi  J., “ Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. ,” Pattern Recogn..  0031-3203 31, (7 ), 963–982  ((1998)).
Blais  F., “ Review of 20years of range sensor development. ,” J. Electron. Imaging.  1017-9909 13, (1 ), 231–243  ((2004)).
Levoy  M., , Rusinkiewicz  S., , Ginzton  M., , Ginsberg  J., , Pulli  K., , Koller  D., , Anderson  S., , Shade  J., , Curless  B., , Pereira  L., , Davis  J., , and Fulk  D., “ The digital michelangelo project: 3D scanning of large statues. ,” in  Proc. of ACM SIGGRAPH Conf. on Computer Graphics. , pp. 131–144 ,  ACM Press/Addison-Wesley ,  New York  ((2000)).
Robinson  A., , Alboul  L., , and Rodrigues  M., “ Methods for indexing stripes in uncoded structured light scanning systems. ,” J. WSCG. 12, (3 ), 371–378  ((2004)).
Orghidan  R., , Salvi  J., , and Mouaddib  E., “ Modelling and accuracy estimation of a new omni-directional depth computation sensor. ,” Pattern Recogn. Lett..  0167-8655 27, (7 ), 843–853  ((2006)).
Forest  J., , Salvi  J., , Cabruja  E., , and Pous  C., “ Laser stripe peak detector for 3d scanners. a fir filter approach. ,” in  Proc. of 17th Int. Conf. on Pattern Recognition. , Vol. 3, , pp. 646–649 ,  IEEE ,  New York  ((2004)).
Fisher  R. B., and Naidu  D. K., “ A comparison of algorithms for subpixel peak detection. ,”  Image Technology: Advances in Image Processing, Multimedia and Machine Vision. , pp. 385–404 ,  Springer-Verlag ,  Heidelberg  ((1996)).
Haug  K., and Pritschow  G., “ Robust laser-stripe sensor for automated weld-seam-tracking in the shipbuilding industry. ,” in  Proc. of Industrial Electronic Conf..  Vol. 2, , pp. 1236–1241 ,  IEEE ,  New York  ((1998)).
Ridler  T. W., and Calvard  S., “ Picture thresholding using an iterative selection method. ,” IEEE Trans. Syst. Man Cybern..  0018-9472 8, (8 ), 630–632  ((1978)).
Duda  R. O., and Hart  P. E.,  Pattern Classification and Scene Analysis. ,  Wiley ,  Hoboken, NJ  ((1973)).
Akima  H., “ A method of bivariate interpolation and smooth surface fitting for irregularly distributed data points. ,” ACM Trans. Math. Softw..  0098-3500 4, (2 ), 148–159  ((1978)).
Tsai  R. Y., “ A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. ,” IEEE J. Rob. Autom..  0882-4967 3, (4 ), 323–344  ((1987)).
Zhang  Z., “ A flexible new technique for camera calibration. ,” IEEE Trans. Pattern Anal. Mach. Intell..  0162-8828 22, (11 ), 1330–1334  ((2000)).
Heikkilä  J., and Silven  O., “ A four-step camera calibration procedure with implicit image correction. ,” in  Proc. of IEEE Computer Soc. Conf. on Computer Vision and Pattern Recognition. , pp. 1106–1112  ((1997)).
Hall  E. L., , Tio  J. B. K., , McPherson  C. A., , and Sadjadi  F. A., “ Measuring curved surfaces for robot vision. ,” IEEE Comput.. 15, (12 ), 42–54  ((1982)).
Salvi  J., , Armangue  X., , and Batlle  J., “ A comparative review of camera calibrating methods with accuracy evaluation. ,” Pattern Recogn..  0031-3203 35, , 1617–1635  ((2002)).
Sun  W., and Cooperstock  J. R., “ An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. ,” Mach. Vision Appl..  0932-8092 17, (1 ), 51–67  ((2006)).
Marzani  F. S., , Voisin  Y., , Voon  L. F. C. L. Y., , and Diou  A., “ Calibration of a three-dimensional reconstruction system using a structured light source. ,” Opt. Eng..  0091-3286 41, (2 ), 484–492  ((2002)).
Harris  C., and Stephens  M., “ A combined corner and edge detector. ,” in  Proc. of 4th Alvey Vis. Conf.. , pp. 147–151 ,  Elsevier North-Holland ,  Amsterdam  ((1988)).
Jain  R.,  The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. ,  Wiley ,  Hoboken, NJ  ((1991)).
García  D. F., , Molleda  J., , González  F., , and Usamentiaga  R., “ Performance evaluation of compression algorithms for developing and testing industrial imaging systems. ,” in  Proc. of World Academy of Science, Engineering and Technology. , 23, , pp. 71–76 ,  World Scientific and Engineering Academy and Society ,  Wisconsin  ((2007)).
Zlib, , “ zlib lossless data compression library. ,” ⟨http://zlib.net⟩ (Date accessed: June 2007).

Grahic Jump LocationImage not available.

Julio Molleda is associate professor in the Department of Computer Science and Engineering at the University of Oviedo. He received his MS in computer science in 2005 and his PhD in 2008 from the Engineering School of the University of Oviedo. He has been working in real-time imaging research projects in the computer engineering area at the University of Oviedo. His research interests include real-time imaging systems and range measurement techniques.

Grahic Jump LocationImage not available.

Rubén Usamentiaga is associate professor in the Department of Computer Science and Engineering at the University of Oviedo. He received his MS and PhD in computer science from the University of Oviedo in 1999 and 2005, respectively. He has been working on several projects related to computer vision and industrial systems. His research interests include real-time imaging systems and thermographic observers for industrial processes.

Grahic Jump LocationImage not available.

Daniel F. García is a full professor in the Department of Computer Science and Engineering at the University of Oviedo, where he received his PhD in electrical engineering in 1988. Since 1994, he has been responsible for the university’s computer engineering area. His current research interest is in the area of the development of high-performance real-time and embedded systems applied to quality assurance and production inspection in industry, where he has more than 100 published papers. For the last ten years, Dr. García has been conducting research projects in the area of information technologies applied to industry at national and European levels. He is a member of ACM and the IEEE Computer Society.

Grahic Jump LocationImage not available.

Francisco G. Bulnes is associate professor in the Department of Computer Science and Engineering at the University of Oviedo. He received his MS in computer science from the Engineering School of the University of Oviedo in 2007 and is currently working toward his PhD in computer science. His current research interest is in the area of real-time imaging systems.

© 2010 SPIE and IS&T

Citation

Julio Molleda ; Rubén Usamentiaga ; Daniel F. García and Francisco G. Bulnes
"Real-time flatness inspection of rolled products based on optical laser triangulation and three-dimensional surface reconstruction", J. Electron. Imaging. 19(3), 031206 (July 14, 2010). ; http://dx.doi.org/10.1117/1.3455987


Figures

Graphic Jump LocationF1 :

Feedback model of a flatness control system.

Graphic Jump LocationF6 :

Flatness map computed from a steel strip during manufacturing.

Graphic Jump LocationF5 :

Flatness profiles computed from a steel strip during manufacturing: (a) Bad leveling, (b) wavy edges, and (c) center buckle and wavy edges.

Graphic Jump LocationF4 :

Computation of flatness indices and flatness profiles.

Graphic Jump LocationF3 :

Common steel strip flatness defects: (a) Bad leveling, (b) wavy edges, and (c) center buckle.

Graphic Jump LocationF2 :

Longitudinal fibers of a steel strip: (a) Strip without irregularities during rolling, (b) theoretical defects, and (c) real flatness defects.

Graphic Jump LocationF7 :

Flatness inspection system layout.

Graphic Jump LocationF12 :

Example of an image used for computing extrinsic parameters of the camera of the proposed flatness inspection system.

Graphic Jump LocationF13 :

Flatness profile computation: (a) Intraframe processing stage and (b) interframe processing stage.

Graphic Jump LocationF14 :

Approximation of the length of a rolled product fiber.

Graphic Jump LocationF15 :

Tuning of the laser stripe extraction method: (a) Flatness inspection system under working conditions, (b) video recording service for registration of real manufacturing conditions, and (c) offline tuning of the laser stripe extraction method using real manufacturing conditions.

Graphic Jump LocationF16 :

Factors and metrics considered in multimedia container and compression algorithm performance evaluation experiments.

Graphic Jump LocationF8 :

Images acquired by the camera of the proposed flatness inspection system: (a) Laser stripe projected in the background of the scene, (b) laser stripe projected onto the uneven surface of a rolled product showing laser degradation, and (c) laser stripe projected onto the surface of a rolled product showing spurious noise in the image.

Graphic Jump LocationF9 :

Laser peak detection: (a) Luminance of different laser profiles and (b) proposed laser peak detection method.

Graphic Jump LocationF10 :

Laser stripe extraction: (a) Laser stripe projected onto a rolled product with the areas used in local and global search, (b) laser peak detection onto the surface of a rolled product, (c) laser peak linking using Akima splines, and (d) laser stripe extracted from the surface of a rolled product.

Graphic Jump LocationF11 :

Set of images used for computing intrinsic parameters of the camera of the proposed flatness inspection system.

Tables

Table Grahic Jump Location
Back-projection error.
Table Grahic Jump Location
Quantitative testing of flatness measurement.
Table Grahic Jump Location
Computing time of the main tasks of the system.

References

Carr  K., and Ferreira  P., “ Verification of form tolerances, Part I: basic issues, flatness, and straightness. ,” Precis. Eng..  0141-6359 17, (2 ), 131–143  ((1995)).
Qiao  J., , Guo  G., , Chai  T., , and Shao  C., “ A new method of flatness control in cold rolling process. ,” in  Proc. of American Control Conf.. , Philadelphia, PA, Vol. 6, , pp. 3828–3832  ((1998)).
Usamentiaga  R., , García  D. F., , González  D., , and Molleda  J., “ Compensation for uneven temperature in flatness control systems for steel strips. ,” in  Proc. of IEEE Industry Applications Soc. Annual Meeting. , pp. 521-527  ((2006)).
ABB Automation Technologies AB, , Switzerland, Stressometer flatness measurement systems ((2005)).
Spreitzhofer  G., , Duemmler  A., , Riess  M., , and Tomasic  M., “ Si-flat contactless flatness measurement for cold rolling mills and processing lines. ,” Rev. Metall.. 102, (9 ), 589–595 , vii–xi ((2005)).
Noll  R., and Krauhausen  M., “ Online laser measurement technology for rolled products. ,” Ironmaking Steelmaking.  0301-9233 35, (3 ), 221–227  ((2008)).
Pirlet  R., , Mulder  J., , Adriaensen  D., , Boelens  J., , and Lochen  C., “ Non-contact system for measuring hot strip flatness. ,” Iron Steel Eng.. , 60, (7 ), 45–50  ((1983)).
García  D. F., , García  M., , Obeso  F., , and Fernández  V., “ Real-time flatness inspection system for steel strip production lines. ,” Real-Time Imag..  1077-2014 5, (1 ), 35–47  ((1999)).
García  D. F., , García  M., , Obeso  F., , and Fernández  V., “ Flatness measurement system based on a nonlinear optical triangulation technique. ,” IEEE Trans. Instrum. Meas..  0018-9456 51, (2 ), 188–195  ((2002)).
López  C., , García  D. F., , Usamentiaga  R., , González  D., , and González  J. A., “ Real time system for flatness inspection of steel strips. ,” Proc. SPIE.  0277-786X 5679, , 228–238  ((2005)).
LDV Systeme, , Germany, ShapeFlex, customized non-contact flatness measurement.
Sun  J., , Zhang  J., , Su  L., , and Cao  J., “ Flatness measurement system for steel strip based on projected fringe method. ,” J. Sci. Instrum..  0950-7671 28, (4 ), 743–747  ((2007)).
Borchers  J., and Gromov  A., “ Topometric measurement of the flatness of rolled products—the system topplan. ,” Metallurgie (Halle).  0368-9573 52, (3–4 ), 247–252  ((2008)).
Paakkari  J., “ On-line flatness measurement of large steel plates using moiré topography. ,” Ph.D. thesis, University of Oulu ((1998)).
Besl  P. J.,  Advances in Machine Vision. , pp. 1–63 ,  Springer-Verlag ,  New York  ((1989)).
Jackson  R. R., , Kelly  P. E., , and Adams  D. W., “ A multi-function measure system for hot-strip mills. ,”  Millenium Steel. , Vol. 4, , pp. 1–4  ((2003)).
Lopera  J. M., , Villegas  P. J., , Linera  F. F., , Hernández-Magadan  F., , Martín-Ramos  J., , Díaz  J., , Vecino  G., , and Rendueles  J. L., “ Flatten it out: designing a low-cost system for flatness monitoring in metal processes. ,” IEEE Ind. Appl. Mag..  1077-2618 14, (2 ), 62–66  ((2008)).
AISE, ,  Hot Strip Mill Profile and Flatness Study, Phase I. ,  Association of Iron and Steel Engineers ,  Pittsburgh, PA  ((1986)).
García  D. F., , López  C., , Canga  I., , González  D., , Usamentiaga  R., , and González  J. A., “ Visualization of the flatness of steel strips during and after their manufacturing. ,” in  Proc. of IASTED Int. Conf. on Visualization, Imaging and Image Processing. , pp. 843–848 ,  ACTA Press ,  Canada  ((2003)).
Chen  F., , Brown  G. M., , and Song  M., “ Overview of three-dimensional shape measurement using optical methods. ,” Opt. Eng..  0091-3286 39, (1 ), 10–22  ((2000)).
Barnard  S. T., and Fischler  M. A., “ Computational stereo. ,” Comput. Surveys. 14, (4 ), 553–572  ((1982)).
Batlle  J., , Mouaddib  E., , and Salvi  J., “ Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. ,” Pattern Recogn..  0031-3203 31, (7 ), 963–982  ((1998)).
Blais  F., “ Review of 20years of range sensor development. ,” J. Electron. Imaging.  1017-9909 13, (1 ), 231–243  ((2004)).
Levoy  M., , Rusinkiewicz  S., , Ginzton  M., , Ginsberg  J., , Pulli  K., , Koller  D., , Anderson  S., , Shade  J., , Curless  B., , Pereira  L., , Davis  J., , and Fulk  D., “ The digital michelangelo project: 3D scanning of large statues. ,” in  Proc. of ACM SIGGRAPH Conf. on Computer Graphics. , pp. 131–144 ,  ACM Press/Addison-Wesley ,  New York  ((2000)).
Robinson  A., , Alboul  L., , and Rodrigues  M., “ Methods for indexing stripes in uncoded structured light scanning systems. ,” J. WSCG. 12, (3 ), 371–378  ((2004)).
Orghidan  R., , Salvi  J., , and Mouaddib  E., “ Modelling and accuracy estimation of a new omni-directional depth computation sensor. ,” Pattern Recogn. Lett..  0167-8655 27, (7 ), 843–853  ((2006)).
Forest  J., , Salvi  J., , Cabruja  E., , and Pous  C., “ Laser stripe peak detector for 3d scanners. a fir filter approach. ,” in  Proc. of 17th Int. Conf. on Pattern Recognition. , Vol. 3, , pp. 646–649 ,  IEEE ,  New York  ((2004)).
Fisher  R. B., and Naidu  D. K., “ A comparison of algorithms for subpixel peak detection. ,”  Image Technology: Advances in Image Processing, Multimedia and Machine Vision. , pp. 385–404 ,  Springer-Verlag ,  Heidelberg  ((1996)).
Haug  K., and Pritschow  G., “ Robust laser-stripe sensor for automated weld-seam-tracking in the shipbuilding industry. ,” in  Proc. of Industrial Electronic Conf..  Vol. 2, , pp. 1236–1241 ,  IEEE ,  New York  ((1998)).
Ridler  T. W., and Calvard  S., “ Picture thresholding using an iterative selection method. ,” IEEE Trans. Syst. Man Cybern..  0018-9472 8, (8 ), 630–632  ((1978)).
Duda  R. O., and Hart  P. E.,  Pattern Classification and Scene Analysis. ,  Wiley ,  Hoboken, NJ  ((1973)).
Akima  H., “ A method of bivariate interpolation and smooth surface fitting for irregularly distributed data points. ,” ACM Trans. Math. Softw..  0098-3500 4, (2 ), 148–159  ((1978)).
Tsai  R. Y., “ A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. ,” IEEE J. Rob. Autom..  0882-4967 3, (4 ), 323–344  ((1987)).
Zhang  Z., “ A flexible new technique for camera calibration. ,” IEEE Trans. Pattern Anal. Mach. Intell..  0162-8828 22, (11 ), 1330–1334  ((2000)).
Heikkilä  J., and Silven  O., “ A four-step camera calibration procedure with implicit image correction. ,” in  Proc. of IEEE Computer Soc. Conf. on Computer Vision and Pattern Recognition. , pp. 1106–1112  ((1997)).
Hall  E. L., , Tio  J. B. K., , McPherson  C. A., , and Sadjadi  F. A., “ Measuring curved surfaces for robot vision. ,” IEEE Comput.. 15, (12 ), 42–54  ((1982)).
Salvi  J., , Armangue  X., , and Batlle  J., “ A comparative review of camera calibrating methods with accuracy evaluation. ,” Pattern Recogn..  0031-3203 35, , 1617–1635  ((2002)).
Sun  W., and Cooperstock  J. R., “ An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. ,” Mach. Vision Appl..  0932-8092 17, (1 ), 51–67  ((2006)).
Marzani  F. S., , Voisin  Y., , Voon  L. F. C. L. Y., , and Diou  A., “ Calibration of a three-dimensional reconstruction system using a structured light source. ,” Opt. Eng..  0091-3286 41, (2 ), 484–492  ((2002)).
Harris  C., and Stephens  M., “ A combined corner and edge detector. ,” in  Proc. of 4th Alvey Vis. Conf.. , pp. 147–151 ,  Elsevier North-Holland ,  Amsterdam  ((1988)).
Jain  R.,  The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. ,  Wiley ,  Hoboken, NJ  ((1991)).
García  D. F., , Molleda  J., , González  F., , and Usamentiaga  R., “ Performance evaluation of compression algorithms for developing and testing industrial imaging systems. ,” in  Proc. of World Academy of Science, Engineering and Technology. , 23, , pp. 71–76 ,  World Scientific and Engineering Academy and Society ,  Wisconsin  ((2007)).
Zlib, , “ zlib lossless data compression library. ,” ⟨http://zlib.net⟩ (Date accessed: June 2007).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.