// the one-minute version
Image processing changes the representation before interpretation. Point operators remap each pixel; histograms summarize distributions; linear filters compute weighted neighborhoods through convolution; derivatives expose change; Fourier analysis explains frequency, blur, sampling, and aliasing. Pyramids represent an image at several scales. Median, bilateral, and morphological operators add nonlinear, edge-aware behavior. Resampling requires reconstruction followed by filtering and sampling; inverse warping avoids holes. Every operation has a transfer function and a failure mode: smoothing suppresses noise but destroys detail, sharpening amplifies both edges and noise, and interpolation invents values rather than recovering lost evidence.
Mira reduces sensor noise with a blur and the robot stops seeing the thin edge of a curb. She sharpens instead and compression artifacts become “obstacles.” Image processing is not a bag of beautification filters. Each operator encodes what variation should count as signal.
01 Point operations reshape intensity and color
A point operator computes O(x,y)=f(I(x,y)) independently. Exposure scaling, gamma correction, contrast curves, clipping, color matrices, and white balance fit this form. They are fast but cannot distinguish a dark object from a shadow because no neighborhood is consulted.
Histograms count intensities. Histogram equalization uses the cumulative distribution to spread common values; local methods adapt to illumination but can amplify noise. Matching a reference histogram aligns distributions, not scene semantics.
02 Convolution is a local weighted question
The kernel h asks the same local question at every position, giving translation equivariance. A normalized box or Gaussian averages; derivative kernels measure change. Correlation omits kernel reversal; many libraries call correlation “convolution,” so verify conventions.
Boundary rules—zero, replicate, reflect, or circular—change results near the image edge.
03 Gaussian smoothing and derivatives form a scale space
The Gaussian gives smooth, separable low-pass filtering: a 2D convolution becomes two 1D passes. Its standard deviation σ chooses scale. Derivatives of Gaussians combine noise suppression with gradients; gradient magnitude estimates edge strength and orientation estimates normal direction.
Second derivatives and Laplacians respond to ridges and blobs but amplify high-frequency noise. There is no scale-free edge: structures appear or vanish as σ changes.
04 Fourier analysis reveals what filters preserve
A sinusoid is an eigenfunction of a linear shift-invariant filter. The Fourier transform represents an image as frequencies; convolution in space becomes multiplication in frequency. Low-pass filters attenuate rapid change; high-pass filters emphasize it. An ideal frequency cutoff rings in space because sharp support in one domain spreads in the other.
Sampling copies the spectrum. If copies overlap, high frequencies masquerade as lower ones—aliasing. Blur before downsampling; upsampling cannot reconstruct frequencies already aliased or discarded.
05 Pyramids make scale explicit
A Gaussian pyramid repeatedly smooths and subsamples. A Laplacian pyramid stores band-pass differences between levels and supports reconstruction. Coarse levels permit large motions or structures to be handled cheaply; fine levels refine detail. Wavelets similarly separate spatial location and scale.
Multi-scale processing is not merely faster. It changes optimization by creating a path from broad structure to detail, though coarse levels can erase small objects permanently.
06 Nonlinear filters protect structure differently
A median filter rejects impulsive outliers and preserves step edges better than averaging. Bilateral filtering weights by both spatial distance and intensity similarity, smoothing within regions while resisting cross-edge mixing. Guided filters use a reference image to shape smoothing.
These methods can staircase gradients, preserve noise mistaken for edges, or create halos. Edge-aware does not mean artifact-free.
07 Morphology reasons with shapes and sets
Erosion keeps pixels whose structuring element fits; dilation keeps pixels touched by it. Opening removes small bright structures; closing fills small dark gaps. Distance transforms record distance to a set; connected components turn binary masks into objects.
The structuring element encodes scale and orientation. Morphology is powerful after segmentation, but thresholds and connectivity choices can dominate the result.
08 Resampling is reconstruction plus sampling
Nearest neighbor is fast and blocky; bilinear interpolation fits a tent kernel; bicubic methods use wider smooth kernels. Interpolation estimates a continuous function from samples, then evaluates it elsewhere. It cannot restore detail absent from the samples.
For a warp, inverse mapping asks where each destination pixel came from and samples the source. Forward mapping sends source pixels outward and creates holes unless splatted. Minification requires footprint-aware prefiltering, not merely bilinear lookup.
09 Restoration needs a degradation model
Denoising estimates a clean image under a noise prior. Deblurring inverts a point-spread function, but frequencies attenuated near zero make the inverse unstable. Wiener filtering balances inverse restoration against noise power; regularized methods penalize implausible solutions.
10 Questions a master’s student should answer
Why is Gaussian filtering separable?
The 2D isotropic Gaussian factors into horizontal and vertical 1D Gaussians, reducing an n×n kernel from quadratic to linear work per pixel.
Why does sharpening amplify noise?
Sharpening boosts high frequencies, and sensor noise often has substantial high-frequency energy. Edge signal and noise occupy overlapping bands.
When is a median preferable to a Gaussian?
For sparse impulsive corruption, because the order statistic rejects extreme samples instead of averaging them into neighbors.
Why use inverse warping?
It visits every destination pixel and samples a source location, avoiding unassigned holes produced by forward point mapping.
How should a preprocessing change be evaluated?
Measure its own distortion and downstream task performance, slice by scale and frequency, hold compute constant, inspect failure images, and ablate each operation.
11 Summary and study sheet
- Point operations change values, not spatial relationships.
- Convolution is linear, local, and translation equivariant.
- Frequency analysis explains blur, sharpening, sampling, and aliasing.
- Pyramids trade resolution for scale and optimization reach.
- Nonlinear and morphological filters encode edge or shape priors.
- Resampling must account for the destination footprint.
12 Source trail
Original companion notes following the official book page and Springer chapter record.