Chapter 7 — Feature Detection and Matching, explained.

Mira needs to recognize the same wall point after the robot moves. Raw pixel coordinates changed, brightness changed, and part of the wall is occluded. She needs landmarks that survive the nuisance changes but preserve enough identity to avoid matching every brick to every other brick.

01 Detection asks where measurement is stable

An edge has strong gradient normal to the boundary but is ambiguous along it—the aperture problem. A corner changes in two directions. The second-moment matrix M=Σw∇I∇Iᵀ summarizes local gradient energy: two large eigenvalues indicate a corner, one an edge, none a flat patch.

02 Harris and Shi–Tomasi score cornerness

Harris uses det(M)−k trace(M)²; Shi–Tomasi uses the smaller eigenvalue. Non-maximum suppression selects isolated responses and subpixel fitting improves localization. Threshold and neighborhood determine density; uniform spatial selection prevents all features clustering on texture.

03 Scale space finds characteristic size

The same structure occupies different pixels at different distance. Search a Gaussian scale space and find extrema of normalized Laplacian or Difference of Gaussians across x,y,σ. The selected σ gives a canonical scale. Orientation from local gradients supplies rotational normalization.

04 Descriptors trade invariance for information

SIFT pools local gradient orientations in spatial cells, clips large bins, and normalizes contrast. Rotation and scale normalization add invariance. Binary descriptors compare selected pixel pairs and match quickly with Hamming distance. Excessive invariance can merge genuinely different structures.

05 Matching is retrieval under ambiguity

Brute-force nearest neighbor is exact but expensive; approximate indexes trade speed for recall. Euclidean distance suits normalized float descriptors; Hamming suits binary strings. The ratio d₁/d₂ rejects a nearest match that is not substantially better than its alternative. Mutual nearest-neighbor checks add consistency.

06 Geometry turns matches into evidence

A homography models a planar scene or pure camera rotation; an essential/fundamental matrix models two-view epipolar geometry. RANSAC samples minimal matches, fits a model, and scores reprojection or symmetric transfer error. Spatially coherent inliers are more persuasive than descriptor distance alone.

07 Edges, lines, and regions carry complementary structure

Canny smooths, differentiates, suppresses non-maxima, and uses hysteresis thresholds to link edges. Hough voting detects parametric lines or circles from edge evidence. MSER and affine-covariant regions handle broader support. Lines dominate man-made scenes where point texture is weak.

08 Learned local features optimize the pipeline

Networks can learn repeatable keypoint scores, orientation, descriptors, and match confidence using correspondences or geometric self-supervision. Attention matchers use context across both feature sets. Training data determines invariance; classical geometry remains valuable for verification and labels.

09 Evaluation separates detector and matcher failures

Measure repeatability under known transforms, localization error, matching precision/recall, inlier ratio after geometry, coverage, and runtime. Warp support regions consistently when viewpoint changes. Benchmark texture, blur, illumination, compression, repetitive patterns, and nonrigid objects separately.

10 Questions

11 Summary and lab

• Corners constrain two-dimensional position.
• Scale and orientation normalization support invariance.
• Descriptors summarize local identity.
• Ambiguity requires ratio/mutual checks.
• Geometric consensus validates correspondence.

12 Source trail

Original notes following the official book page and Springer’s chapter record.