JEI Letters

CMOS image sensor for spatiotemporal image acquisition

[+] Author Affiliations
B. Bellach

Laboratoire LE2I – I.U.T. Le Creusot, 12, rue de la fonderie, 71200 Le Creusot, France

G. Cathébras

LIRMM – Université Montpellier II, 161, rue Ada, 34392 Montpellier Cedex 5, France

L. F. C. Lew Yan Voon, O. Aubreton, B. Lamalle

Laboratoire LE2I – I.U.T. Le Creusot, 12, rue de la fonderie, 71200 Le Creusot, France

J. Electron. Imaging. 15(2), 020502 (April 20, 2006). doi:10.1117/1.2191767
History: Received June 20, 2005; Revised December 07, 2005; Accepted March 02, 2006; Published April 20, 2006
Text Size: A A A

Open Access Open Access

We present a 64-×64-pixel CMOS image sensor chip that can acquire a 16-gray-level image containing both spatial and temporal information of the moving luminous object under observation. The image can next be processed using any image processing software in order to determine the speed, trajectory, and direction of motion of the moving object. The architecture of the sensor and an example of its application to the determination of the speed of a moving laser spot are described.

Figures in this Article

The determination of the speed of a moving object requires the knowledge of the trajectory and the time taken by the object to cover a certain distance along the trajectory. To achieve this by machine vision using classical cameras, one needs to take several images of the object at different time instances so as to obtain the trajectory, displacement, and time information. Even for relatively slow moving objects this requires that the images be taken and transferred to the computer at a relatively high rate. Let us consider, for instance, an object moving horizontally at 2ms and suppose that the width of the view field of the vision system is 10cm. The object would take 0.05s to cover the 10-cm distance. Since at least two images of the object are necessary for computing the average speed, this means that the images must be acquired at a minimum rate of one image every 0.05s (note that the video rate, 25imagess, is one image every 0.04s). For faster objects or if more than two images are required then the image rate would increase.

It is well known that in classical systems the image transfer rate is a bottleneck and one solution is to store the images on the chip at a very high rate and later transfer them to the computer at a much lower rate. The CMOS image sensor chip design that we present herein proposes an alternative solution by furnishing an image that contains all the required spatial and temporal information.1

Chip Architecture

The image sensor chip is implemented in standard 0.6μm CMOS process from Austria Micro Systems. It is composed of the main functional blocks represented in Fig. 1. These blocks are:

  • An array of 64×64pixels that we have designed to digitalize the temporal information and store it in memory devices integrated at the pixel level.
  • A classical 4-bit Gray code counter used instead of a binary counter to avoid state transition hazards.
  • A control block used to generate all the signals necessary for the correct functioning of the circuit. It is implemented as a finite state machine.
  • A row decoder for selecting the pixels one row at a time during initialization and readout of the contents of the memory devices integrated at the pixel level.
  • 64 voltage difference sensing amplifiers similar to the one described in Ref. 2 (one per column) and a demultiplexer for reading out the contents of the 4-bit memory devices.

  • An array of 64×64pixels that we have designed to digitalize the temporal information and store it in memory devices integrated at the pixel level.
  • A classical 4-bit Gray code counter used instead of a binary counter to avoid state transition hazards.
  • A control block used to generate all the signals necessary for the correct functioning of the circuit. It is implemented as a finite state machine.
  • A row decoder for selecting the pixels one row at a time during initialization and readout of the contents of the memory devices integrated at the pixel level.
  • 64 voltage difference sensing amplifiers similar to the one described in Ref. 2 (one per column) and a demultiplexer for reading out the contents of the 4-bit memory devices.

Graphic Jump LocationF1 :

Main functional blocks of the chip.

Pixel Design

The architecture of the pixel is a digital sensor pixel34 with an implementation of the analog-to-digital converter using a classical Gray code counter common to the array of pixels combined with a pulse generator and memory devices. Figures 2 represent the schematic circuit diagram of the pixel and its layout. The light sensing element is a photodiode. Unlike classical sensors where the voltage drop across the photodiode is measured to deduce intensity information, we define a threshold for the voltage drop and measure the time needed to achieve this voltage drop. To do so we use a monitoring inverter followed by a pulse generator at the output of the photodiode so that when the voltage across it falls below the threshold of the inverter, a pulse is generated. The pulse is next applied to the 4-bit memory device in order to memorize the current value of the Gray code counter.

Graphic Jump LocationF2 :

(a) Schematic diagram of the pixel; (b) layout of the pixel (fill factor of about 18%).

The time at which the pulse is generated is equal to the sum of the time at which the pixel is illuminated by the object and the photodiode discharge time. If the object is highly luminous or if the threshold of the photodiode voltage monitoring inverter is close to the maximal potential then the photodiode discharge time can be neglected and the value stored in the 4-bit memory corresponds to the time at which the pixel is illuminated by the object. In our case the threshold is fixed by design (physical dimensions of the monitoring inverter) to about half the maximal potential so that the sensor can work as a classical imager also for testing purposes. For sensors dedicated to speed measurement only the monitoring inverter must be designed so that the threshold value is close to the maximal potential.

Working Principle

The acquisition of a spatiotemporal image is illustrated in Fig. 3 on an array of 6×6pixels. The object is supposed to be of size one pixel and moving along the diagonal from the top left corner to the bottom right one of the array of pixels.

Graphic Jump LocationF3 :

Spatiotemporal image acquisition.

At the start of the acquisition of an image all the 4-bit memory devices as well as the 4-bit counter common to the array of pixels are first initialized to the value 0. Next, the counter is clocked at a frequency that is chosen according to the speed of the moving object under observation. The higher the speed is, the higher the frequency must be. Each time a pixel is illuminated by the bright object for the first time, the current value of the counter corresponding to the mumber of clock cycles since the start of acquisition is stored in the pixel's memory device. In the example of Fig. 3, for instance, the pixels at coordinates (2, 2), (3, 3), and (4, 4) have been illuminated after 2, 4, and 5 clock cycles, respectively.

The acquisition is over after the counter has made a complete turn, that is, after 16 clock cycles. The acquisition time is thus equal to 16T where T is the clock period. Finally, the values stored in the pixel's memories are read-out and displayed as a 16-gray-level image.

The circuit has been tested to be fully functional. Figure 4 represents the experimental setup used to acquire a spatiotemporal image for speed measurement. It is composed of an electric motor with a slightly inclined reflective surface mounted at the end of its shaft and a screen. A small laser beam is fired at the reflective surface that reflects it back onto the screen. As the motor turns round, the spot due to the laser beam describes a circle of 2cm in diameter on the screen.

Graphic Jump LocationF4 :

Laser spot in circular motion.

An example of the image of the moving laser spot obtained using our sensor mounted in a case with a lens of focal length 25mm is given in Fig. 5. One can notice that the circular trajectory of the spot is composed of several regions of different gray levels. Each region represents in fact the position of the spot at the time instant given by the gray level. Indeed, the gray level, unlike classical sensors, is not intensity information but time information since it corresponds to the number of clock cycles since the start of acquisition before the pixels are illuminated by the incident light from the object. So, by considering any two regions we can compute the average speed of the spot using the following distance-over-time equation:Display Formula

1speed=dT×(G2G1),
where d is the distance between the center of gravity of any two regions representing the initial and final positions of the object, G1 and G2 are the gray levels of these two regions, and T is the period of the clock. Thus, to compute the speed of the laser spot we need to first segment the image in order to clearly delimit the different regions, then determine for each region its center of gravity, and finally choose any two regions and compute the speed according to Eq. 1. The distance d along the trajectory between the two regions can be easily computed knowing the scale factor between the real scene and the image (a function of the focal length of the lens and the distance between the sensor and the scene) and the size of the pixels (49.2×49.2μm).

Graphic Jump LocationF5 :

(a)Image of the laser spot; (b) center of gravity.

The image of Fig. 5 after segmentation and determination of the centers of gravity is shown in Fig. 5. The segmentation algorithm that we have used is very simple in this case since we have a rather good image of the spot. It consisted of considering all the pixels of the circular trajectory as pertaining to the same region if they have the same gray level. For poor-quality images that may result from low contrast between the observed moving object and its background, a more complex algorithm may be necessary. The segmentation problem is thus application dependent. Once we have obtained the different regions, the center of gravity (xk, yk) of a region k is computed according to:Display Formula

2xk=i=1Nj=1MxiGk(i,j)i=1Nj=1MGk(i,j),yk=i=1Nj=1MyjGk(i,j)i=1Nj=1MGk(i,j),
where Gk(i,j) is the gray level of the region k if the pixel at coordinates (i,j) pertains to the region k and 0 otherwise, and N and M are the size of the image.

Using the experimental setup of Fig. 4, several experiments have been carried out and the speed of the laser spot determined by both image processing using Eq. 1 and calculations from the data furnished by a tachymeter. Some of the results are given in Table 1.

Table Grahic Jump Location
Speed measurement.

One can note that the speed values determined from the image acquired by the sensor are quite close to the calculated ones. The difference is less than 7%. Moreover, we would like to point out here that for the image of Fig. 5 taken at a clock frequency of 500Hz for a laser spot moving at about 1.5ms, the acquisition time is 0.032s. This image contains the coordinates and time information of the object for about ten different positions in the course of its displacement (ten different regions). To obtain the same information using classical cameras would require the acquisition and readout of ten images in about 0.032s (more than ten times the video rate). The maximum frequency of the clock is by design 20MHz but in practice we have only been able to test the chip for a maximum frequency of 1MHz (acquisition time of 1.6μs corresponding to 16 clock cycles).

Some possible applications of our image sensor are, for example, in areas such as automotive, medical or biomedical, and electrical devices manufacturing. The sensor can be used to study the trajectory and speed of the deformation of moving objects upon collision (car crash test), the displacement of certain moving parts of an object or human body (study of the motion of the legs of an athlete), the formation of electrical arcs in electrical devices, etc. Since our sensor requires that the moving object be very bright, it can be envisaged to use strong illumination over a dark background or bright markers placed on the objects.

For all these applications, depending on the speed of the phenomenon, charged-coupled devices (CCD), CMOS imagers, or high-speed cameras are used. Compared to these devices, our sensor has certain advantages. The speed of the moving object that can be observed using our sensor is higher than with CCD or classical CMOS imagers. This is due to the fact that unlike classical CCD or CMOS devices that require the acquisition and readout of several images, our sensor furnishes only one image containing all the required spatiotemporal information, resulting thus in a shorter readout time. Moreover, the trajectory is clearly visible on the image and need not be determined from successive images.

Compared to high-speed cameras, the performance of our sensor is much less but it has the advantage of being less expensive too. So, for certain applications for which the use of a high-speed camera is not compulsory, our image sensor is a good cost-effective solution.

We have described an image sensor fabricated in standard 0.6-μm CMOS process. It can furnish both spatial and temporal information on the moving object under observation in a single image for speed measurement applications. The sensor has been tested to be fully functional and the circuit design validated by the successful application of the sensor to the measurement of the speed of a moving laser spot with an error less than 7% compared to the value calculated from data furnished by a measuring instrument.

Bellach  B., “ Capteurs d'images: application à l'observation et à la mesure de vitesse de phénomènes lumineux rapides. ,” PhD Thesis, Université de Bourgogne (Dec. 2003).
Yukawa  A., “ A CMOS 8-bit high speed A/D converter IC. ,” IEEE J. Solid-State Circuits.  0018-9200 SC-20, (3 ), 775–779  (June (1985)).
El Gamal  A., , Yang  D., , and Fowler  B., “ Pixel level processing—why, what, and how?. ,” Proc. SPIE.  0277-786X 3650, , 2–13  (Jan. (1999)).
Kleinfelder  S., , Lim  S., , Liu  X., , and El Gamal  A., “ A 10000frames∕s CMOS digital pixel sensor. ,” IEEE J. Solid-State Circuits.  0018-9200 36, (12 ), 2049–2059  (Dec. (2001)).
© 2006 SPIE and IS&T

Citation

B. Bellach ; G. Cathébras ; L. F. C. Lew Yan Voon ; O. Aubreton and B. Lamalle
"CMOS image sensor for spatiotemporal image acquisition", J. Electron. Imaging. 15(2), 020502 (April 20, 2006). ; http://dx.doi.org/10.1117/1.2191767


Figures

Graphic Jump LocationF1 :

Main functional blocks of the chip.

Graphic Jump LocationF2 :

(a) Schematic diagram of the pixel; (b) layout of the pixel (fill factor of about 18%).

Graphic Jump LocationF3 :

Spatiotemporal image acquisition.

Graphic Jump LocationF4 :

Laser spot in circular motion.

Graphic Jump LocationF5 :

(a)Image of the laser spot; (b) center of gravity.

Tables

Table Grahic Jump Location
Speed measurement.

References

Bellach  B., “ Capteurs d'images: application à l'observation et à la mesure de vitesse de phénomènes lumineux rapides. ,” PhD Thesis, Université de Bourgogne (Dec. 2003).
Yukawa  A., “ A CMOS 8-bit high speed A/D converter IC. ,” IEEE J. Solid-State Circuits.  0018-9200 SC-20, (3 ), 775–779  (June (1985)).
El Gamal  A., , Yang  D., , and Fowler  B., “ Pixel level processing—why, what, and how?. ,” Proc. SPIE.  0277-786X 3650, , 2–13  (Jan. (1999)).
Kleinfelder  S., , Lim  S., , Liu  X., , and El Gamal  A., “ A 10000frames∕s CMOS digital pixel sensor. ,” IEEE J. Solid-State Circuits.  0018-9200 36, (12 ), 2049–2059  (Dec. (2001)).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.