// the one-minute version
Computational photography co-designs capture and computation. Multiple exposures extend dynamic range; tone mapping compresses it for display. Bursts provide repeated noisy measurements for denoising and super-resolution, but must be aligned and deghosted. Deblurring estimates a latent sharp image, sometimes an unknown blur kernel, under strong priors. Matting separates foreground color, background color, and fractional opacity; compositing must use premultiplied alpha and linear light. Flash/no-flash, focus stacks, coded exposure, and light fields deliberately vary capture to make an inverse problem better constrained. Quality is perceptual and task-dependent, so hallucinated detail and temporal instability must be audited.
Mira’s night image is either sharp and noisy or clean and blurred. A single exposure cannot collect more photons without integrating motion. The solution is not one perfect filter; it is to capture several complementary measurements and solve for the latent scene.
01 A camera pipeline is an estimator
Raw sensor data passes through black-level correction, demosaicing, denoising, white balance, color correction, sharpening, tone mapping, and compression. Each stage changes statistics used by the next. Modern pipelines optimize stages jointly and may preserve raw bursts rather than one processed frame.
02 HDR combines exposures without trusting saturated pixels
Short exposures preserve highlights; long exposures reveal shadows. Linearize camera response, divide by exposure time, weight reliable unsaturated measurements, align, and merge radiance. Moving objects require selecting one exposure or motion-aware deghosting.
03 Tone mapping is display adaptation
Global curves compress all luminance consistently but can flatten contrast. Local operators preserve local detail but risk halos and reversed contrast. Work with luminance carefully, preserve color ratios, and evaluate under actual display conditions. HDR capture and HDR display are different problems.
More frames help only if their differences are modeled rather than averaged blindly.
04 Burst denoising depends on alignment
Independent noise averages down with repeated measurements, while scene detail does not. Align frames to a reference, reject moving or occluded regions, and combine according to signal-dependent noise. Misregistration creates waxy texture and double edges; temporal selection avoids corruption at the cost of less denoising.
05 Deblurring is an ill-conditioned inverse
Model y=k*x+n. Non-blind deconvolution assumes k known; blind deblurring estimates k and x, with many equivalent explanations. Natural-image, sparsity, or learned priors stabilize missing frequencies. Ringing reveals mismatch or aggressive inversion. Motion sensors and coded exposure can make k easier to recover.
06 Super-resolution needs complementary subpixels
Small camera shifts sample the scene on different grids. After motion estimation and blur modeling, joint reconstruction can recover frequencies better than interpolation. Occlusion and nonrigid motion violate one global warp. Learned single-image SR predicts plausible texture from prior data and may hallucinate details not measured.
07 Matting models fractional visibility
Given one color C, foreground F, background B, and alpha α are underdetermined. Trimaps, color models, smoothness, and deep priors constrain them. Hair, transparency, motion blur, and reflection make binary segmentation inadequate.
08 Designed capture changes the inverse problem
Flash/no-flash pairs separate high-detail flash structure from ambient appearance. Focus stacks extend depth of field. Structured light projects known patterns for depth. Light fields sample rays across viewpoint and allow refocusing. Coded apertures and exposures trade conventional image quality for recoverable information.
09 Mobile photography is a real-time systems problem
Frame selection, motion estimation, semantic masks, neural enhancement, memory, thermal budget, latency, and preview consistency interact. Evaluate faces, text, fine texture, moving subjects, low light, and frame-to-frame stability. Preserve raw evidence when downstream measurement matters.
10 Questions
Why bracket exposure?
No single exposure simultaneously avoids highlight saturation and shadow noise when scene dynamic range exceeds the sensor.
Why is blind deblurring ambiguous?
Many sharp-image/kernel pairs can convolve to similar observations, so priors and capture cues are required.
Does super-resolution recover truth?
Multi-frame methods recover measured complementary samples; single-image learned methods may generate plausible but unobserved detail.
Why premultiply alpha?
It stores αF, making filtering and compositing behave correctly near transparent boundaries and avoiding color fringes.
How should computational photos be evaluated?
Use fidelity, perceptual preference, artifact slices, temporal stability, uncertainty, and task-specific consequences.
11 Summary and lab
- Capture diversity can make inference identifiable.
- Alignment and rejection precede burst fusion.
- Restoration always combines a forward model with a prior.
- Matting represents mixed pixels, not just labels.
- Plausibility must not be confused with measured truth.
12 Source trail
Original notes following the official book page and Springer’s chapter record.