// the one-minute version
Structure from motion jointly recovers camera poses and 3D points from correspondences. Two calibrated views obey x′ᵀEx=0 with E=[t]×R; uncalibrated views use F. Decomposing E yields pose candidates, disambiguated by points lying in front of both cameras. Triangulation intersects viewing rays but becomes uncertain for small baseline or distant points. PnP estimates camera pose from 3D–2D matches. Bundle adjustment minimizes all reprojection errors using sparse nonlinear least squares. Visual odometry estimates local motion; SLAM maintains a map, tracks uncertainty, recognizes revisited places, closes loops, and globally corrects drift. Monocular systems have scale gauge; dynamic scenes and repeated places threaten observability and data association.
Mira’s robot returns to its starting door, but its reconstructed trajectory misses by six meters. Every local motion estimate looked accurate. Tiny biases accumulated because odometry had no global memory of having seen the door before.
01 Two cameras turn a pixel into intersecting rays
A pixel defines a viewing ray, not a 3D point. A second camera supplies another ray; their geometry constrains depth. Camera centers and a point define an epipolar plane. Its intersection with each image is an epipolar line, reducing correspondence search from 2D to 1D.
Nearly parallel rays intersect poorly; geometry can estimate a point while correctly reporting large depth uncertainty.
02 Essential and fundamental matrices encode two-view geometry
E applies to normalized calibrated coordinates; F incorporates intrinsics for pixels. Robust minimal solvers estimate them from matches. Normalize data, reject degeneracy, and score geometric distance. A planar scene or pure rotation makes general translation recovery poorly constrained.
03 Pose decomposition has ambiguity
E gives rotation and translation direction, not translation magnitude. Algebra yields four candidate configurations; cheirality selects the one placing triangulated points in front of both cameras. Monocular reconstruction remains globally scale ambiguous.
04 Triangulation must respect uncertainty
Linear triangulation gives an initial point; nonlinear refinement minimizes reprojection error. Depth variance increases with distance, pixel noise, and inverse baseline. Reject points with low parallax, negative depth, or large residual; do not fill a map with unstable landmarks.
05 PnP localizes a camera in an existing map
Given 3D points and their pixels, perspective-n-point estimates R,t. RANSAC handles wrong correspondences, then nonlinear pose refinement uses all inliers. Coplanar or clustered points weaken constraints. Relocalization retrieves candidate places before geometric verification.
06 Bundle adjustment makes all estimates agree
Camera and point variables create a sparse Jacobian. Schur complement eliminates points efficiently. Fix one pose and scale to remove gauge freedoms. Robust loss limits mismatches, but poor data association can still create a coherent wrong map.
07 Incremental SfM grows a reconstruction
Choose a well-conditioned initial pair, triangulate, register a new camera with PnP, add points, and bundle-adjust locally and globally. Global SfM estimates rotations/translations more jointly. Track selection, view graph connectivity, and loop evidence determine stability.
08 SLAM adds online state and uncertainty
Filtering methods update a probabilistic state sequentially; keyframe graph methods optimize selected poses and landmarks. Direct SLAM aligns intensities; feature SLAM tracks descriptors. Inertial measurements provide short-term motion, gravity, and scale information but require timing and bias calibration.
09 Loop closure corrects drift but can destroy a map
Place recognition proposes revisits; geometric verification confirms them; a pose graph adds a loop constraint and distributes correction. Perceptual aliasing—two similar corridors—creates false loops, so conservative verification and robust graph optimization are essential.
10 Questions
Why is monocular scale unknown?
Scaling translations and all 3D points together leaves every projection unchanged.
Essential versus fundamental matrix?
E relates calibrated normalized rays; F relates uncalibrated pixel coordinates and includes intrinsics.
Why bundle adjust?
Pairwise estimates are inconsistent; joint optimization uses every observation to refine cameras and points coherently.
Visual odometry versus SLAM?
Odometry emphasizes local incremental motion; SLAM maintains a reusable map and uses loop closure for global consistency.
How should SLAM be evaluated?
Absolute and relative trajectory error after a declared alignment, scale drift, map quality, loop precision, runtime, and failure rate.
11 Summary and lab
- Epipolar geometry restricts two-view matches.
- Pose and triangulation have scale and baseline ambiguities.
- PnP localizes against known 3D.
- Bundle adjustment is sparse joint refinement.
- SLAM adds maps, loops, uncertainty, and recovery.
12 Source trail
Original notes following the official book page and Springer’s chapter record.