JEI Letters

Photometric limits for digital camera systems

[+] Author Affiliations
Michael Schöberl, André Kaup

University of Erlangen-Nuremberg, Multimedia communications and Signal Processing, Cauerstrasse 7, 91058 Erlangen, Germany

Andreas Brückner

Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Strasse 7, 07745 Jena, Germany

Siegfried Foessel

Fraunhofer Institute for Integrated Circuits, Am Wolfsmantel 33, 91058 Erlangen, Germany

J. Electron. Imaging. 21(2), 020501 (Jun 15, 2012). doi:10.1117/1.JEI.21.2.020501
History: Received November 21, 2011; Revised February 10, 2012; Accepted May 14, 2012
Text Size: A A A

Open Access Open Access

Abstract.  Image sensors for digital cameras are built with ever decreasing pixel sizes. The size of the pixels seems to be limited by technology only. However, there is also a hard theoretical limit for classical video camera systems: During a certain exposure time only a certain number of photons will reach the sensor. The resulting shot noise thus limits the signal-to-noise ratio. In this letter we show that current sensors are already surprisingly close to this limit.

Figures in this Article

The steady progress in semiconductor technology allows the manufacturing of smaller and smaller structures and image sensors with ever shrinking pixel sizes. One can get the impression that the pixel size is just limited by the technology and even smaller pixels are desirable. Today, consumer products with pixel sizes dp=1.4μm are already on the market and devices with dp=1.1μm are in production.1 In comparison, photo receptors in the human eye are reported to be larger than 3 μm.2

The general modeling of light is well understood,3 and simulation with commercial tools like ISET4 is possible. In contrast, this letter addresses parameters like aperture and pixel size and their photometric consequences for modeling the amount of light that is available in a digital video camera system. One of the design parameters is the resulting image quality. With small pixels only a few photons will hit a single pixel during an exposure period and the signal-to-noise power ratio (SNR) will be poor due to shot noise.5 Apart from all technological limitations, this physical boundary limits the performance of today’s video cameras.

The scene radiates a certain amount of light. This is described by an average radiance in object space Lobj. The sensor sees an effective amount of light equivalent to the cone with a solid angle Ω as shown in Fig. 1(a). This cone is defined by the sphere of radius equal to the focal length f and a circular aperture disk with diameter D. The solid angle thus calculates to6Display Formula

Ω=π·(D/2)2f2=π4·(f/D)2[sr].(1)
The sensor receives an irradiance IDisplay Formula
I=ηlens·Ω·Lobj[W·m2],(2)
for a lens with optical transmittance ηlens.

Graphic Jump LocationF1 :

Parameters of (a) focal length f, aperture diameter D and resulting solid angle Ω and (b) quadratic pixels with fill factor γff.

On the sensor some area is used for interconnects and transistors so that only some of the area is sensitive to light. Figure 1(b) shows pixels of size dp. The ratio of active to total areas is expressed as an effective sensor fill factor γff. Even with clever manufacturing like micro-lenses or back side illumination, γff<1 holds. A single pixel thus captures a certain amount of radiant power (radiant flux) Φpix of the sensor irradiance Display Formula

Φpix=dp2·γff·I[W].(3)
A single photon of wavelength λ has the energy hcλ with the speed of light c and Planck’s constant h. The radiant flux Φ thus consists of Nphot photons Display Formula
Nphot=1hcλ·τexp·Φpix,(4)
during a certain time interval (exposure time) of τexp. In the photoreceptor only some of these photons are converted into electrons Nelec=ηqe·Nphot while others are not, due to reflection, recombination and other material interactions. The conversion rate is expressed as quantum efficiency ηqe.7 The electrons are then collected in the pixel. Although we will see Nelec, on average, the charge is still quantized and the actual number of electrons is subject to shot noise due to the occurrence of random events. For N electrons the associated shot noise is of strength N.5 As NelecΦpix, signal power is represented with Nelec and SNR thus calculates to Display Formula
SNR=Nelec/Nelec=Nelec.(5)
In CCD or CMOS technology there are further sources of sensor noise,8 which are neglected in the ideal case.

SNR is a parameter that is directly visible in the final images. For answering the original question, we can combine the above equations. This leads to Display Formula

dp,min=SNR2·hcληqe·γff·τexp·4·(f/D)2ηlens·π·Lobj.(6)

At first we assume ideal technology. A typical indoor scene is illuminated with a luminance of Lv=100cdm2.9 For the peak sensitivity of the human eye at a wavelength of λ=555nm the SI unit candela is defined10 as radiant intensity of 1/683Wsr1. The radiance in object space is then Display Formula

Lobj=100·16830.146Wsr1m2.(7)
We further assume a perfectly transparent lens with ηlens=1, a wide aperture f/D=2.8, fill factor γff=1 and quantum efficiency ηqe=1. For achieving typical video frame rates a maximum exposure time of τexp=0.03s is used. For a human observer, images without visible noise are preferred. From psychophysical studies a thousand-photon limit is reported as the threshold for visibility of shot noise.5 We therefore set SNR=100032. With green light with λ=555nm the minimum pixel size calculates to dp,min=0.9μm.

The influence of different apertures is shown in Fig. 2. With larger aperture diameters, even smaller pixels can be used. A variation of luminance is also possible: In practice, the human color perception (photoptic vision) starts at Lv=3cdm2.9 The luminance in daylight exterior scenarios is typically Lv=104cdm2.9 The resulting minimum pixel sizes thus range from 5 to 0.09 μm as shown in Fig. 3.

Graphic Jump LocationF2 :

Minimum pixel sizes for photon limited system with varying apertures, ideal system with SNR=1000 and scene with luminance of Lv=100cdm2, dashed line for f/D=2.8.

Graphic Jump LocationF3 :

Minimum pixel sizes for photon limited system with varying luminance, ideal system with SNR=1000 and aperture f/D=2.8, dashed line for Lv=100cdm2.

Up to now, we used monochromatic light only. We now extend this and also include the spectral distribution of light. Again, we start with a scene with a luminance of Lv=100cdm2. Now, the light is made up of radiation from a light bulb. This is modeled as a black body at a certain color temperature T and a spectral radiance of Display Formula

Lobj,λ(λ,T)=L0·2hc2λ5·1ehcλkT1[W·sr1·m3].(8)
With the photoptic luminous efficiency function11Vm, we set Display Formula
Km·0Vm·(λ)·Lobj,λ(λ,T)dλ=!Lv,(9)
with Km=683lmW1. The resulting normalized spectral radiance Lobj,λ(λ,T) is now perceived by the human eye as a luminance of Lv=100cdm2. Figure 4 shows the resulting set of normalized spectral radiances for typical color temperatures.

Graphic Jump LocationF4 :

Spectral radiance of black bodies with temperatures T, intensity scaled to be perceived as luminance of 100cdm2.

Today, most cameras are used to capture scenes for later viewing by a human. The camera should therefore create a representation of the scene that is similar to that of the human visual system. We simulate an ideal camera with the spectral sensitivity curves based on the Stockman and Sharpe cone measurements of the human eye.12 The corresponding spectral sensitivity functions for long (L), medium (M) and short (S) wavelengths are shown in Fig. 5. However, we assume an ideal camera with ideal color filters and material without any attenuation (ηqe=1) at peak efficiency.

Graphic Jump LocationF5 :

Sensitivity functions of 10-deg cone fundamentals for L, M and S cone and luminous efficiency function Vm.

In Table 1, the resulting minimum pixel sizes are shown for the radiometric simulation. The luminosity case with monochromatic light at λ=555nm corresponds to the ideal simulation from above. There is less than 10% error for the simulation with L and M cones compared to the luminosity. This is plausible from the high similarity of the respective sensitivity curves. However, the capturing of blue light (short wavelengths with cone S) requires larger pixels. At short wavelengths, the individual photons have a higher energy and thus, there are fewer for a given radiant flux. This explains the problem of inferior performance of blue color channels in typical digital cameras. The extreme case of observing monochromatic green light with a short wavelength sensitivity leads to even fewer photons and would require pixels with 26 μm. In general, the monochromatic calculation is only slightly optimistic but gives a good approximation to a radiometric computation.

Table Grahic Jump Location
Table 1Minimum pixel sizes (in μm) based on radiometric calculations for light sources with black body radiation of temperature T and monochromatic light source.

The above numbers represent the theoretical limit for ideal sensors. In practice, a real world camera does not achieve these numbers. For example, a highly optimized three layer stacked image sensor is reported by Hannebauer et al.13 For pixels of size dp=4.8μm a high fill factor of γff=0.95 and quantum efficiency of ηqe=0.8 is possible with many (costly) optimizations. In current 1.4 μm consumer grade sensors the backside illumination (BSI) technology enables close to 100% fill factor.14 For color imaging, the spectral sensitivity is not without attenuation and peak quantum efficiencies of about ηqe0.5 are reported by OmniVision14 and Aptina.15 In scientific CMOS sensors, the combined sensor readout noise is reported as low as 1.3electrons/pixel16 and can thus be neglected among 1000 electrons. The combined assumption of ηlens=0.95, γff=0.95 and ηqe=0.5 leads to a minimum pixel size of dp,min=1.34μm. With mass-market sensors and additional noise,8 larger pixels are required.

These small pixels also reach another technological limit of decreasing full well capacity. For example Aptina reports15C=5000 electrons, which leaves only a dynamic range of 51 from noise visibility5 to overexposure. As a result, most of the image will still look noisy. However, this is a technological challenge that could be addressed with multiple readouts during the exposure.17

Another limitation comes with optical diffraction. Even in ideal optics the achievable resolution of a camera system is limited. The Sparrow criterion suggests3 that there is no gain in resolution below a critical pixel size of dp,crit=λ2·f/D. For our example of f/D=2.8 and λ=555nm, we obtain dp,crit=0.78μm. Achieving this limit, however, is challenging, especially in the off-axis field, and leads to expensive optics. A further decrease in aperture requires a dramatic increase of the technological efforts and smaller tolerances for optics manufacturers.

In our photometric analysis, we discuss the number of photons per pixel. With small pixels the image quality is limited by shot noise, and for indoor scenarios the current video cameras are surprisingly close to this fundamental limit. We estimate that even with ideal technology, a pixel size below dp=0.9μm will not capture enough light to generate visually pleasing videos any more. Current technology is far from perfect and with optimistic assumptions, the limit at dp=1.34μm is close to current sensors. However, for other imaging scenarios like outdoor daylight still photography, there is plenty of room at the bottom.

Fontaine  R., “A review of the 1.4 um pixel generation,” in  Int. Image Sensor Workshop (IISW) ,  Hokkaido, Japan  (2011).
Jonas  J. B., Schneider  U., Naumann  G. O. H., “Count and density of human retinal photoreceptors,” Graefes Arch. Clin. Exp. Ophthalmol.. 230, , 505 –510 (1992). 0721-832X CrossRef
Goodman  J., Introduction to Fourier Optics. ,  Roberts & Company Publishers ,  Englewood, Colorado, USA  (2005).
Farrell  J. et al., “A simulation tool for evaluating digital camera image quality,” in SPIE Electron. Imag.—Image Quality and System Performance. 5294, , 124 –131 (2004).CrossRef
Xiao  F., Farrell  J., Wandell  B., “Psychophysical thresholds and digital camera sensitivity: the thousand photon limit,” in SPIE Electron. Imag.—Digital Photography. 5678, , 75 –84 (2005).CrossRef
Kingslake  R., Optical System Design. ,  Academic Press ,  London 1,  ( Oct. 1983).
Fowler  B. et al., “A method for estimating quantum efficiency for CMOS image sensors,” in SPIE Electron. Imag.—Solid State Sensor Arrays: Development and Applications II. 3301, , 178 –185 (1998).CrossRef
Gow  R. et al., “A comprehensive tool for modeling CMOS image-sensor-noise performance,” IEEE Trans. Electron. Devices. 54, , 1321 –1329 ( June 2007). 0018-9383 CrossRef
Smith  W., Modern Optical Engineering: The Design of Optical Systems. ,  Tata McGraw-Hill Education ,  Englewood, Colorado, USA  (1990).
Giacomo  P., “News from the BIPM: resolution 3—definition of the candela,” Metrologia. 16, (1 ), 55 –61 (1980). 0026-1394 CrossRef
Sharpe  L. et al., “A luminous efficiency function, V*(λ), for daylight adaptation,” J. Vision. 5, (11 ), 948 –968 (2005). 1534-7362 CrossRef
Stockman  A., Sharpe  L., “The spectral sensitivities of the middle-and long-wavelength-sensitive cones derived from measurements in observers of known genotype,” Vis. Res.. 40, (13 ), 1711 –1737 (2000). 0042-6989 CrossRef
Hannebauer  R. et al., “Optimizing quantum efficiency in a stacked CMOS sensor,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial, Scientific, and Consumer Applications XII. 7875, (1 ), 787505  (2011).CrossRef
Rhodes  H. et al., “The mass production of second generation 65 nm BSI CMOS image sensors,” in  Int. Image Sensor Workshop (IISW) ,  International Image Sensor Society (IISS)  (2011).
Agranov  G. et al., “Pixel continues to shrink … pixel development for novel CMOS image sensors: a review of the 1.4 um pixel generation,” in  Int. Image Sensor Workshop (IISW) ,  International Image Sensor Society (IISS)  (2011).
Fowler  B. et al., “A 5.5 mpixel 100  frames/sec wide dynamic range low noise CMOS image sensor for scientific applications,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial/Scientific Applications XI. 7536, , 753607  ( Jan. 2010).CrossRef
Schöberl  M. et al., “Digital neutral density filter for moving picture cameras,” in SPIE Electron. Imag.—Computational Imag. VIII. 7533, , 75330L  ( January 2010).CrossRef
© 2012 SPIE and IS&T

Citation

Michael Schöberl ; Andreas Brückner ; Siegfried Foessel and André Kaup
"Photometric limits for digital camera systems", J. Electron. Imaging. 21(2), 020501 (Jun 15, 2012). ; http://dx.doi.org/10.1117/1.JEI.21.2.020501


Figures

Graphic Jump LocationF1 :

Parameters of (a) focal length f, aperture diameter D and resulting solid angle Ω and (b) quadratic pixels with fill factor γff.

Graphic Jump LocationF2 :

Minimum pixel sizes for photon limited system with varying apertures, ideal system with SNR=1000 and scene with luminance of Lv=100cdm2, dashed line for f/D=2.8.

Graphic Jump LocationF3 :

Minimum pixel sizes for photon limited system with varying luminance, ideal system with SNR=1000 and aperture f/D=2.8, dashed line for Lv=100cdm2.

Graphic Jump LocationF4 :

Spectral radiance of black bodies with temperatures T, intensity scaled to be perceived as luminance of 100cdm2.

Graphic Jump LocationF5 :

Sensitivity functions of 10-deg cone fundamentals for L, M and S cone and luminous efficiency function Vm.

Tables

Table Grahic Jump Location
Table 1Minimum pixel sizes (in μm) based on radiometric calculations for light sources with black body radiation of temperature T and monochromatic light source.

References

Fontaine  R., “A review of the 1.4 um pixel generation,” in  Int. Image Sensor Workshop (IISW) ,  Hokkaido, Japan  (2011).
Jonas  J. B., Schneider  U., Naumann  G. O. H., “Count and density of human retinal photoreceptors,” Graefes Arch. Clin. Exp. Ophthalmol.. 230, , 505 –510 (1992). 0721-832X CrossRef
Goodman  J., Introduction to Fourier Optics. ,  Roberts & Company Publishers ,  Englewood, Colorado, USA  (2005).
Farrell  J. et al., “A simulation tool for evaluating digital camera image quality,” in SPIE Electron. Imag.—Image Quality and System Performance. 5294, , 124 –131 (2004).CrossRef
Xiao  F., Farrell  J., Wandell  B., “Psychophysical thresholds and digital camera sensitivity: the thousand photon limit,” in SPIE Electron. Imag.—Digital Photography. 5678, , 75 –84 (2005).CrossRef
Kingslake  R., Optical System Design. ,  Academic Press ,  London 1,  ( Oct. 1983).
Fowler  B. et al., “A method for estimating quantum efficiency for CMOS image sensors,” in SPIE Electron. Imag.—Solid State Sensor Arrays: Development and Applications II. 3301, , 178 –185 (1998).CrossRef
Gow  R. et al., “A comprehensive tool for modeling CMOS image-sensor-noise performance,” IEEE Trans. Electron. Devices. 54, , 1321 –1329 ( June 2007). 0018-9383 CrossRef
Smith  W., Modern Optical Engineering: The Design of Optical Systems. ,  Tata McGraw-Hill Education ,  Englewood, Colorado, USA  (1990).
Giacomo  P., “News from the BIPM: resolution 3—definition of the candela,” Metrologia. 16, (1 ), 55 –61 (1980). 0026-1394 CrossRef
Sharpe  L. et al., “A luminous efficiency function, V*(λ), for daylight adaptation,” J. Vision. 5, (11 ), 948 –968 (2005). 1534-7362 CrossRef
Stockman  A., Sharpe  L., “The spectral sensitivities of the middle-and long-wavelength-sensitive cones derived from measurements in observers of known genotype,” Vis. Res.. 40, (13 ), 1711 –1737 (2000). 0042-6989 CrossRef
Hannebauer  R. et al., “Optimizing quantum efficiency in a stacked CMOS sensor,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial, Scientific, and Consumer Applications XII. 7875, (1 ), 787505  (2011).CrossRef
Rhodes  H. et al., “The mass production of second generation 65 nm BSI CMOS image sensors,” in  Int. Image Sensor Workshop (IISW) ,  International Image Sensor Society (IISS)  (2011).
Agranov  G. et al., “Pixel continues to shrink … pixel development for novel CMOS image sensors: a review of the 1.4 um pixel generation,” in  Int. Image Sensor Workshop (IISW) ,  International Image Sensor Society (IISS)  (2011).
Fowler  B. et al., “A 5.5 mpixel 100  frames/sec wide dynamic range low noise CMOS image sensor for scientific applications,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial/Scientific Applications XI. 7536, , 753607  ( Jan. 2010).CrossRef
Schöberl  M. et al., “Digital neutral density filter for moving picture cameras,” in SPIE Electron. Imag.—Computational Imag. VIII. 7533, , 75330L  ( January 2010).CrossRef

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.