Computer Vision — cheatsheet, everything on one card.

A dense, scannable revision card for AIMLCZG525 Computer Vision — BITS Pilani M.Tech AIML mid-sem material from CV1 through CV7. It compresses the lecture flow: why vision is hard, image formation, sampling, histograms, filtering, edges, Canny, Hough, Harris, HoG and SIFT. Use it as a last-pass sheet: every section gives the idea, the trigger, the formula and the exam gotchas.

Course map — what Computer Vision is trying to do

CV1

• Computer vision goal: recover useful facts about the world from images: 3D shape, object identity, people, motion, text, medical evidence, scene layout and actions. The lecture phrase is "perceive the story behind the picture".
• Why it is hard: a 2D image is an ambiguous projection of a 3D world. Many scenes can create the same image because viewpoint, illumination, scale, occlusion, background clutter, intra-class variation, motion blur and local ambiguity all change pixels without changing the underlying object.
• Classic pipeline: acquire image → represent pixels → improve/enhance → detect local structures → group structures → recognize, track, reconstruct or measure. Deep learning changes many implementation details, but these low-level building blocks still explain what the network is learning.
• Applications: OCR, license plates, face detection and biometrics, computational photography, super-resolution, autonomous driving, robotics, medical imaging, AR, object detection, segmentation, tracking and visual SLAM.

Image formation, pinhole camera and lenses

CV2

• Image as a function: a grayscale image is \(f(x,y)\), where coordinates \((x,y)\) locate a point and \(f\) returns brightness. A digital image is a matrix after sampling and quantization.
• Physical model: intensity is controlled by illumination and reflectance: \(f(x,y)=i(x,y)r(x,y)\). Here \(i(x,y)\) is incident light and \(r(x,y)\in[0,1]\) is how much the surface reflects. \(r=0\) means total absorption; \(r=1\) means total reflectance.
• Pinhole camera: a tiny aperture lets each scene point send a narrow ray to the image plane. The ideal model gives perspective projection. With focal length \(f\), a 3D point \((X,Y,Z)\) maps approximately to \(x=fX/Z,\;y=fY/Z\). Bigger depth \(Z\) makes objects smaller; parallel 3D lines may meet at vanishing points.
• Image inversion: a physical pinhole camera forms an inverted image on the back image plane. In computer vision we usually use a virtual image plane in front of the pinhole to avoid carrying a sign flip.
• Why lenses: pinholes are sharp but dim. A smaller hole increases depth of field but gathers little light and needs long exposure. Lenses gather more light, reduce exposure time and can be stacked/focused to produce a clearer image while approximating the same projection model.
• Lens caveats: real lenses introduce blur, radial distortion, tangential distortion, defocus and chromatic effects. Calibration estimates intrinsic parameters and distortion coefficients so pixel measurements can be corrected before geometry tasks.

Term	Exam meaning	Usual gotcha
Illumination	Light falling on the scene, \(i(x,y)\)	Changing light can change pixels without changing objects.
Reflectance	Surface property, \(r(x,y)\)	Vision often wants reflectance/object facts, not raw brightness.
Perspective	Projection where size depends on depth	Straight 3D lines stay straight; angles and lengths generally do not.

Sampling, quantization and digital image properties

CV2

• Sampling: digitizing the coordinate values. It turns continuous \((x,y)\) positions into a grid of pixels. Spatial resolution answers: how close can two details be and still appear separate?
• Quantization: digitizing amplitude values. It turns continuous brightness into finite gray levels. Intensity resolution answers: how small a brightness difference can the image store?
• Matrix view: an \(M\times N\) grayscale image has \(MN\) samples. If each pixel uses \(k\) bits, then \(L=2^k\) gray levels are possible; for 8-bit, \(L=256\) and intensities are \(0\) to \(255\).
• Color depth: 24-bit RGB uses 8 bits each for red, green and blue, so total colors are \((2^8)^3=16{,}777{,}216\).
• Aliasing: if sampling is too coarse for the detail frequency, high-frequency texture masquerades as low-frequency patterns. Low-pass filtering before downsampling reduces aliasing.
• Quantization artifacts: too few levels cause false contours and banding. The lecture examples show the same spatial resolution with 256, 128, 64, ..., 2 levels: spatial detail remains, but tones collapse.

Histograms and histogram equalization

CV2–CV3

• Histogram: counts how many pixels have each gray level. If \(n_k\) pixels have gray level \(r_k\), the un-normalized histogram is \(n_k\). The normalized histogram is \(p(r_k)=n_k/(MN)\), and \(\sum_k p(r_k)=1\).
• Appearance clue: a dark image has mass at low intensities; bright image at high intensities; low-contrast image in a narrow band; high-contrast image spread across the range. Histogram shape is related to image appearance but does not encode spatial arrangement.
• Histogram equalization goal: transform the image so it uses the available gray range more evenly. It often reveals detail in low-contrast images, but can also amplify noise or produce unnatural contrast.
• Core formula: use the cumulative distribution function: \(s_k=(L-1)\sum_{j=0}^{k}p(r_j)\). In words: add all probabilities up to current gray level, multiply by the maximum gray value, then round for a discrete image.
• Continuous view: \(s=T(r)=(L-1)\int_0^r p_r(w)dw\). This maps intensities through their CDF; if assumptions hold, the output distribution tends toward uniform.
• Exam workflow: table columns are gray level, count \(n_k\), PDF \(n_k/MN\), CDF, \((L-1)\text{CDF}\), rounded new gray level. Then replace every old pixel value using the mapping.

Point and intensity transforms

CV2–CV3

• Point operation: output at a pixel depends only on the input intensity at that same pixel: \(s=T(r)\). It does not look at neighbors. Histogram equalization, negative, log, gamma, thresholding and contrast stretching are point transforms.
• Negative: for an \(L\)-level image, \(s=L-1-r\). It reverses black and white and is useful when light objects on dark background become easier to inspect after inversion, such as mammogram examples.
• Log transform: \(s=c\log(1+r)\). It expands low intensities and compresses high intensities. Common trigger: displaying Fourier spectra, because a few very large values hide the rest unless compressed.
• Power-law / gamma: \(s=cr^\gamma\), with \(c>0,\gamma>0\). If inputs are normalized to \([0,1]\), \(\gamma<1\) brightens dark regions; \(\gamma>1\) darkens. Gamma correction compensates nonlinear display response, especially old CRT behavior around \(1.8\)–\(2.5\).
• Contrast stretching: a piecewise linear map expands a narrow input interval into a wider output interval. Use it when the histogram occupies a small part of the range but you do not want full equalization.
• Thresholding: \(s=0\) if \(r
• Intensity slicing: highlight a chosen range \([A,B]\). Either suppress everything else to a low value or leave outside pixels unchanged while marking the target band.

Transform	Formula	Use when
Negative	\(s=L-1-r\)	Invert bright/dark relationships.
Log	\(s=c\log(1+r)\)	Compress dynamic range; reveal small values.
Gamma	\(s=cr^\gamma\)	Brightness correction and display compensation.
Stretch	piecewise linear	Expand low-contrast data.

Spatial filtering, correlation and convolution

CV3

• Spatial filter: a neighborhood plus an operation. Instead of changing each pixel independently, it combines nearby pixels. This is how we smooth noise, sharpen detail and compute derivatives.
• Kernel origin: the image origin is top-left, but the kernel origin is normally its center. Centering symmetric kernels makes formulas and implementation easier.
• Correlation: slide the kernel over the image and multiply matching positions without flipping the kernel. For many symmetric kernels, correlation and convolution give the same result.
• Convolution: flip the kernel in both directions, then slide and sum: \((f*w)(x,y)=\sum_s\sum_t w(s,t)f(x-s,y-t)\). It is linear, shift-invariant and associative, which lets us combine smoothing and derivative operations efficiently.
• Boundary handling: at image edges the window falls partly outside. Choices include zero padding, replication, reflection, wrap-around or computing only the valid interior. Exam numericals often ignore border pixels.
• Kernel normalization: smoothing kernels usually sum to 1 so average brightness is preserved. Derivative kernels usually sum to 0 so constant regions produce zero response.

Smoothing: box, Gaussian and median

CV3–CV5

• Why smooth: derivatives amplify noise. Smoothing removes high-frequency variation before edge detection, thresholding, Hough voting, Harris response or descriptor extraction. The trade-off: less noise but blurred edges and lost fine detail.
• Box filter: all entries have equal weight, e.g. \(\frac{1}{9}\mathbf{1}_{3\times3}\). It is cheap and intuitive, but can create blocky blur and has weaker frequency behavior than Gaussian smoothing.
• Gaussian filter: weights fall with distance from the center: \(G(x,y;\sigma)=\frac{1}{2\pi\sigma^2}e^{-(x^2+y^2)/(2\sigma^2)}\). Larger \(\sigma\) means more blur. Larger kernel size is needed to capture the Gaussian tail.
• Gaussian advantages: isotropic, smooth, separable into 1D horizontal and vertical filters, and mathematically tied to scale-space theory. Canny, SIFT and many blob/feature algorithms build on Gaussian smoothing.
• Median filter: replace a pixel by the median of its neighborhood. It is non-linear, excellent for salt-and-pepper noise and better at preserving edges than an average filter, but not represented by convolution.
• Choosing size: 3×3 removes light noise; 11×11 or 21×21 produces strong low-pass filtering. If important details are smaller than the kernel, they may disappear.

Color spaces: RGB, HSV/HSI, CMY and YCbCr

CV3–CV5

• Color depends on three things: spectral power of the light source, reflectance of the object and spectral sensitivity of the sensor. We do not see objects directly; we see light reflected or transmitted by them.
• Trichromacy: human cones are roughly short, medium and long wavelength sensors. The lecture maps these to S/M/L or B/G/R sensitivities, with visible light around 400–700 nm.
• RGB: additive, device-oriented and used by monitors/cameras. Red + green = yellow, red + blue = magenta, green + blue = cyan. 24-bit RGB stores over 16 million colors.
• CMY/CMYK: subtractive, printer-oriented. Complement relations are \(C=1-R\), \(M=1-G\), \(Y=1-B\). CMYK adds black ink, often \(K=\min(C,M,Y)\), then subtracts \(K\) from C, M and Y.
• HSV/HSI: perception-oriented. Hue is dominant color impression, saturation is purity/amount of white mixed in, intensity/value is brightness. Useful because chromatic information can be partly decoupled from illumination.
• YCbCr/YUV/YIQ: separate luminance \(Y\) from chromaticity. Video and compression keep more bandwidth for luminance because humans notice brightness detail more strongly than color detail.

Gradients, finite differences, Sobel and Prewitt

CV4–CV5

• Edge intuition: an edge is a rapid intensity change. Treating the image as a function, edges look like steep cliffs. The gradient points in the direction of maximum increase; the edge direction is perpendicular to the gradient.
• Digital derivative choices: reconstruct a continuous function then differentiate, or use finite differences directly. Course numericals use finite-difference masks on a 3×3 neighborhood.
• Gradient vector: \(\nabla f=[G_x,G_y]^T=[\partial f/\partial x,\partial f/\partial y]^T\). Magnitude is \(\sqrt{G_x^2+G_y^2}\), often approximated by \(|G_x|+|G_y|\) for speed. Direction is \(\theta=\tan^{-1}(G_y/G_x)\), preferably \(\operatorname{atan2}(G_y,G_x)\).
• Prewitt: derivative plus simple averaging. \(G_x\) mask \(\begin{bmatrix}-1&0&1\\-1&0&1\\-1&0&1\end{bmatrix}\), \(G_y\) mask \(\begin{bmatrix}-1&-1&-1\\0&0&0\\1&1&1\end{bmatrix}\).
• Sobel: like Prewitt but center row/column has weight 2, giving a little extra smoothing. \(G_x=\begin{bmatrix}-1&0&1\\-2&0&2\\-1&0&1\end{bmatrix}\), \(G_y=\begin{bmatrix}-1&-2&-1\\0&0&0\\1&2&1\end{bmatrix}\).
• Noise issue: derivatives respond strongly to random pixel jumps. That is why the lecture shows edge detection before and after smoothing; the smoothed gradient magnitude is much more stable.

Edge models and detector requirements

CV4–CV5

• Step edge: abrupt transition from one intensity to another. First derivative has a peak at the transition; second derivative crosses zero near the edge.
• Ramp edge: gradual transition spread across several pixels, often due to blur, defocus or sampling. The derivative is broader, so localization is harder.
• Roof edge: thin ridge or line-like structure where intensity rises then falls. First derivative gives positive and negative responses; second derivative has a strong central response.
• Edge detector output: should produce edge position, magnitude/strength and orientation/direction. The task is to convert a 2D image into a set of curves made from edge pixels.
• Three objectives: good detection (respond to real edges, not noise), good localization (detected point near true edge), and single response (one detected point per true edge). These are also Canny's design criteria.

Marr-Hildreth and Laplacian of Gaussian

CV4–CV5 context

• Second derivative idea: first derivatives peak at edges; second derivatives cross zero where the first derivative changes sign. Marr-Hildreth detects zero crossings after smoothing.
• Laplacian: \(\nabla^2 f=\partial^2f/\partial x^2+\partial^2f/\partial y^2\). It is isotropic, meaning it responds similarly in every direction, but it is very noise-sensitive.
• LoG: smooth first with a Gaussian, then apply the Laplacian: \(\nabla^2(G_\sigma*f)=(\nabla^2G_\sigma)*f\). This combines noise reduction and second-derivative edge detection.
• Zero crossing: after LoG filtering, mark pixels where the response changes sign and the magnitude change is large enough. Without a threshold, tiny noise oscillations create too many false edges.
• DoG approximation: Difference of Gaussians approximates scale-normalized LoG and becomes important in SIFT. \(G(k\sigma)-G(\sigma)\) is cheaper than exact LoG.

Canny edge detector pipeline

CV4–CV5

• Pipeline: smooth → gradient → non-maximum suppression → double threshold → hysteresis edge linking. The slides call it the first full computer vision pipeline and emphasize that it is still widely used.
• 1. Smooth: filter with a Gaussian low-pass filter. \(\sigma\) sets the scale: small \(\sigma\) detects fine edges and noise; large \(\sigma\) detects coarse strong boundaries and suppresses noise.
• 2. Gradient: compute \(G_x,G_y\), magnitude \(g=\sqrt{G_x^2+G_y^2}\) and orientation \(\theta=\operatorname{atan2}(G_y,G_x)\).
• 3. Non-maximum suppression: keep a pixel only if its gradient magnitude is a local maximum along the gradient direction. This thins broad gradient ridges into one-pixel edge candidates.
• 4. Double threshold: choose high threshold \(T_H\) and low threshold \(T_L\). Common high-to-low ratio is 2:1 to 3:1. Pixels above \(T_H\) are strong edges; between thresholds are weak candidates; below \(T_L\) are rejected.
• 5. Hysteresis: strong edges are valid. Weak edges are valid only if connected to a strong edge, usually through an 8-neighborhood. This keeps real edge curves and drops isolated noise.
• Canny criteria: high detection rate, good localization and single response. If asked, connect each pipeline step to a criterion: Gaussian improves detection under noise, NMS gives single response/localization, hysteresis links true curves without keeping isolated noise.

Hough transform for lines and circles

CV5

• Problem after edge detection: edge maps are fragmented, noisy and cluttered. Line fitting must decide how many lines exist, which points belong to which line and how to bridge missing evidence.
• Voting idea: each feature votes for all model parameters compatible with it. Noise votes too, but true structure receives consistent votes in one accumulator bin.
• Slope-intercept Hough: line \(y=mx+b\). An image point \((x_0,y_0)\) maps to a line in parameter space: \(b=-x_0m+y_0\). Points lying on the same image line generate parameter-space curves that intersect at the line's \((m,b)\).
• Normal form: use \(\rho=x\cos\theta+y\sin\theta\), where \(\rho\) is perpendicular distance from origin and \(\theta\) is the angle of that perpendicular. This avoids infinite slope problems for vertical lines.
• Algorithm: compute edge points → for each point and each sampled \(\theta\), calculate \(\rho\) → increment accumulator \(A(\rho,\theta)\) → find peaks → convert peak parameters back to image lines.
• Circles: circle model \((x-a)^2+(y-b)^2=R^2\). If radius is known, each edge point votes in \((a,b)\); if radius unknown, vote in 3D parameter space \((a,b,R)\). Using gradient direction can reduce votes.
• Trade-offs: coarse bins merge nearby lines; fine bins split votes. Threshold too low gives false lines, too high misses faint lines. Hough works well with missing edge segments because votes accumulate globally.

Harris corner detector

CV6

• Corner intuition: a good corner is locally unique. Shift a small window in any direction and the intensity pattern changes a lot. Flat regions change little in all directions; edges change strongly only across the edge, not along it.
• Window shift energy: Harris starts from \(E(u,v)=\sum_{x,y}w(x,y)[I(x+u,y+v)-I(x,y)]^2\). Using a Taylor expansion, this becomes a quadratic form in the shift vector.
• Structure tensor: \(M=\sum w\begin{bmatrix}I_x^2&I_xI_y\\I_xI_y&I_y^2\end{bmatrix}\). It captures how gradients are distributed inside the local window.
• Eigenvalue interpretation: eigenvalues \(\lambda_1,\lambda_2\) describe intensity change along two principal directions. Both small → flat. One large, one small → edge. Both large → corner.
• Harris response: \(R=\det(M)-k(\operatorname{trace}M)^2\), with \(k\approx0.04\)–\(0.06\). \(R\) large positive means corner; negative with large magnitude means edge; near zero means flat.
• Algorithm: compute image gradients → form products \(I_x^2,I_y^2,I_xI_y\) → smooth these products over a window → compute \(R\) → threshold \(R\) → keep local maxima.
• Applications: object recognition, image matching, panorama stitching, tracking, calibration checkerboards, stereo reconstruction, SLAM, robotics, AR and OCR document structure.

Histogram of Oriented Gradients (HoG)

CV6

• What HoG is: a feature descriptor for object detection. It summarizes local edge direction distributions rather than raw pixels. It is famous for pedestrian detection with a linear SVM.
• Core idea: object shape can often be recognized by the pattern of gradient orientations. Each pixel votes into an orientation histogram using its gradient magnitude as vote strength.
• Standard window: lecture uses a 64×128 patch. Divide into 8×8 cells, group cells into overlapping 2×2 blocks, quantize orientations into 9 bins from 0° to 180° and concatenate.
• Dimensionality: 64×128 gives 8 cells across and 16 cells down. With 2×2 blocks sliding by one cell, there are 7×15 blocks. Each block has 4 cells × 9 bins = 36 values. Total descriptor = \(7\times15\times36=3780\) dimensions.
• Normalization: concatenate histograms in a block and normalize, e.g. \(v'=v/\sqrt{\lVert v\rVert_2^2+\varepsilon^2}\). This reduces sensitivity to illumination and contrast changes.
• Binning detail: a pixel angle can vote into neighboring bins by interpolation. Example from revision: 45° with magnitude 14.14 splits between 40° and 60° bins, weighted by distance to bin centers.
• Pipeline: compute \(G_x,G_y\) → magnitude/orientation → cell histograms → block normalization → concatenate → classifier. HoG is not a detector alone; it is a descriptor fed to a classifier.

SIFT: scale-invariant feature transform

CV7

• Motivation: panorama stitching, object search, visual SLAM, image alignment, 3D reconstruction, AR tracking and retrieval all need repeatable local features that can be detected and matched across images.
• Local feature components: detection identifies interest points, description extracts a vector around each point, matching finds correspondences between descriptors in two views.
• SIFT goal: distinctive invariant features. It is designed for invariance to scale and rotation and robustness to affine distortion, viewpoint change, noise and illumination change.
• Scale space: blur the image with Gaussians at increasing \(\sigma\) values. Structures appear strongest at their characteristic scale. Scale invariance comes from detecting features in \((x,y,\sigma)\), not only \((x,y)\).
• Difference of Gaussian: \(D(x,y,\sigma)=(G(x,y,k\sigma)-G(x,y,\sigma))*I(x,y)\). DoG approximates scale-normalized LoG and highlights blobs efficiently.
• Keypoint detection: find local extrema in DoG scale space by comparing each sample to neighbors in current, previous and next scales. Candidate must be extreme in position and scale.
• Keypoint localization: refine location, reject low-contrast points and remove edge-like unstable points. This prevents weak noise peaks or line responses from becoming features.
• Orientation assignment: compute local gradient orientations around the keypoint and assign dominant orientation(s). Rotating the descriptor frame to this orientation gives rotation invariance.
• Descriptor: take a local neighborhood, usually 4×4 cells, each with an 8-bin orientation histogram. Concatenate to get \(4\times4\times8=128\) dimensions. Normalize to reduce illumination effects.
• Matching: compare descriptors with Euclidean distance. Lowe's ratio test accepts a match when nearest neighbor distance is much smaller than second-nearest distance, filtering ambiguous matches. RANSAC is often used afterward to reject geometric outliers.

Numerical checklist before the mid-sem

• Histogram equalization: do not forget to normalize by total pixels \(MN\), compute cumulative probabilities, multiply by \(L-1\), round and remap the image.
• Convolution/filtering: align kernel center with target pixel. For true convolution flip the kernel; for symmetric smoothing masks the flip changes nothing. State your boundary assumption.
• Gradient masks: multiply elementwise with the 3×3 patch and sum for \(G_x\), then for \(G_y\). Magnitude is \(\sqrt{G_x^2+G_y^2}\), direction uses atan2. Edge direction is perpendicular to gradient direction.
• Canny: list the exact order: Gaussian smoothing, gradient magnitude/angle, NMS, double threshold, hysteresis. If asked why two thresholds: high starts reliable edges, low continues them through weak connected pixels.
• Hough line: for slope form, point \((x_0,y_0)\) gives \(b=-x_0m+y_0\). For polar form, plug into \(\rho=x\cos\theta+y\sin\theta\) and vote.
• Harris: compute products of gradients, sum/smooth inside window, form \(M\), get \(\det\), trace and \(R\). Interpret \(R\), not just calculate it.
• HoG: remember 64×128 → 8×16 cells → 7×15 blocks → 4 cells/block → 9 bins → 3780 features.
• SIFT: remember order: scale space, DoG extrema, localization, orientation, 128-D descriptor, matching.

References

• T1 — Richard Szeliski, Computer Vision: Algorithms and Applications, 2nd ed., free online at szeliski.org/Book.
• T2 — Milan Sonka, Vaclav Hlavac & Roger Boyle, Image Processing, Analysis, and Machine Vision, 4th ed.
• R1 — Rafael C. Gonzalez & Richard E. Woods, Digital Image Processing.
• Course: BITS Pilani WILP — Computer Vision (AIMLCZG525), CV1–CV7 lecture decks.