// the one-minute version
A pixel is the end of a physical pipeline. A 3D point is projected through a camera model; lenses introduce focus, aperture, and distortion; illumination interacts with material; exposure integrates photons; filters and sensor electronics turn light into sampled numbers. The pinhole equation x=fX/Z explains perspective, while a camera matrix combines intrinsics and pose. Calibration estimates those parameters. Radiometry separates light transport from geometry; color spaces separate physical spectra from device responses and perception. Aliasing, saturation, blur, rolling shutter, demosaicing, and noise are not cosmetic defects—they change the evidence every later algorithm receives.
Mira’s detector works in the laboratory and misses the same delivery cart outdoors. At noon its white panel saturates; in shade it turns blue; near the image edge its straight rail bends; while the robot turns, its wheels lean diagonally. Training more boxes treats symptoms. Chapter 2 asks what happened before the detector: how did a world point become a pixel?
01 Coordinates make geometry computable
A point in the world has coordinates Pw=[X,Y,Z,1]ᵀ in a chosen world frame. Camera pose—rotation R and translation t—expresses it in camera coordinates: Pc=RPw+t. The distinction matters because changing the camera and changing the world produce related image motion.
Homogeneous coordinates add a scale-equivalent component: [x,y,w] and [kx,ky,kw] denote the same image point after division by w. This lets perspective projection, affine transforms, and translations become matrix multiplication. Points at infinity represent directions and vanishing points.
02 The pinhole camera derives perspective
Similar triangles give the central projection:
Double X and Z together and the pixel stays fixed: one view cannot recover absolute scale. Increasing focal length magnifies the image; increasing depth shrinks objects. Parallel world lines not parallel to the image plane converge at a vanishing point.
Projection preserves rays, not Euclidean distances. Depth is lost unless another cue restores it.
03 Intrinsics and extrinsics form the camera matrix
Extrinsics [R|t] place the camera in the world. Intrinsics K convert normalized camera rays to pixels: focal lengths fx,fy, principal point (cx,cy), and sometimes skew. Together, homogeneous image p satisfies p∼K[R|t]P.
This equation separates what belongs to the device from where the device is. Resizing or cropping changes intrinsics; moving the camera changes extrinsics. Mixing them is a frequent implementation bug.
04 Real lenses add focus, aperture, and distortion
A lens gathers more light than a pinhole. The thin-lens equation 1/f=1/z+1/z′ relates object distance, image distance, and focal length. Points at the focus plane become sharp; others form circles of confusion. Aperture controls light and depth of field: a wide aperture gathers photons but narrows the sharp depth range.
Radial distortion bends straight lines outward or inward; tangential distortion comes from decentered elements. A common model maps normalized radius r with coefficients k1,k2,… . Wide-angle and fisheye cameras need models appropriate to their projection, not aggressive pinhole polynomials.
05 Calibration turns a device into a measuring instrument
Calibration estimates intrinsics and distortion from known geometry, often many views of a planar target. Each observed corner contributes constraints; nonlinear refinement minimizes reprojection error over parameters. Pose estimation then finds R,t for a calibrated camera and known 3D points.
Low reprojection error is necessary, not sufficient. Bad target coverage, nearly identical views, corner bias, autofocus changes, or a wrong distortion model can produce unstable parameters. Validate on held-out views and inspect error across the image, not only one RMS number.
06 Radiometry explains brightness
Geometry says where a point projects; radiometry says how much light reaches the sensor. Illumination hits a surface, the bidirectional reflectance distribution function describes how it scatters by incoming and outgoing direction, visibility handles shadows, and the lens integrates radiance over aperture and exposure time.
Lambertian surfaces appear proportional to cosine-weighted illumination and approximately independent of viewing direction. Specular materials violate that simplification. Brightness constancy used by tracking and stereo is therefore an assumption about lighting, reflectance, exposure, and camera response—not a law.
07 Color is spectrum filtered by sensor and perception
Light has a spectrum; a camera records a few channel integrals through spectral sensitivities. Many spectra yield the same RGB triplet—metamerism. White balance estimates illumination color; a color correction matrix maps device responses toward a standard space; gamma or transfer curves make stored values nonlinear.
Linear-light arithmetic is required for physically meaningful blending and exposure combination. sRGB values are convenient for display but should be linearized before averaging. Chromaticity separates color ratios from intensity, while perceptual spaces try to make numerical distance align better with visible difference.
08 Sensors sample, integrate, and corrupt
Pixels integrate photons over area and time. Shot noise grows with signal variance; read noise is added by electronics; dark current accumulates without light; quantization maps continuous charge to finite codes. Dynamic range is bounded below by noise and above by saturation.
Color-filter arrays measure only one color per photosite, so demosaicing estimates missing channels. An optical low-pass filter and adequate sampling reduce aliasing. If spatial detail exceeds the Nyquist limit, false patterns appear and cannot be removed reliably after sampling.
09 Time is part of image formation
Long exposure causes motion blur: measurements integrate a moving scene. Rolling-shutter sensors expose rows at different times, turning camera rotation into skew and wobble. Automatic exposure and gain change brightness between frames. Video algorithms must model or stabilize these processes.
10 Questions a master’s student should be able to answer
Why can one image not recover absolute depth?
Projection depends on X/Z and Y/Z. Scaling all scene coordinates from the camera by the same factor leaves pixels unchanged, so metric scale needs another view, known size, focus cue, active sensor, or prior.
What changes K when an image is resized?
Focal lengths and principal-point coordinates scale with image dimensions. Cropping additionally shifts the principal point. Distortion is usually defined in normalized coordinates but must remain consistent with preprocessing.
Why does a smaller aperture increase depth of field?
It narrows the cone of rays reaching each pixel, reducing circles of confusion for points away from the focus plane. It also passes less light, requiring longer exposure or gain.
Is RGB an intrinsic property of a surface?
No. Recorded RGB depends on surface reflectance, illumination spectrum, geometry, camera spectral sensitivities, exposure, processing, and color space.
How should calibration quality be reported?
Use held-out reprojection error, spatial residual plots, parameter stability across subsets, coverage and pose diversity, straight-line checks after undistortion, and downstream metric accuracy.
11 Chapter summary and master’s study sheet
- Projection: p∼K[R|t]P converts world points to pixels and loses scale along a ray.
- Optics: focal length, aperture, focus, and distortion shape the measurement.
- Calibration: estimates device geometry and must be validated beyond training reprojection error.
- Radiometry: illumination, material, visibility, exposure, and response determine brightness.
- Color: RGB is a device-filtered, processed summary of spectra—not surface truth.
- Sensing: integration, noise, sampling, demosaicing, saturation, and timing define downstream evidence.
derive
lab
12 Source trail and scope
Independent companion material following the author’s official second-edition page and Springer’s Image Formation chapter record. The narrative, explanations, diagram, and exercises are original.