UDL slides explained -- lectures 1 to 8.

Lecture-by-lecture explanation of the local UDL PDFs/PPTX. For deeper continuous narrative, use the companion page: UDL from first principles.

Lecture 1 -- course map and why unsupervised learning matters

overview

The course starts by separating supervised learning from unsupervised representation and density learning. No labels does not mean no signal; it means the model must exploit structure inside \(x\) itself. The topic map sets the semester: PCA and variants, likelihood models, autoregressive models, normalizing flows, autoencoders/VAE, GANs, diffusion, energy-based models, vision/language models and semi-supervised learning.

The exam relevance is practical: know the model-family tradeoff. Autoregressive models give exact likelihood but slow sampling. Flows give exact likelihood and latent space but need invertibility. Autoencoders learn codes through reconstruction. PCA gives the simplest linear code.

Lecture 2 -- PCA, ICA, CCA and LLE

factor analysis

Factor analysis asks for hidden axes that explain observed data. PCA finds orthogonal directions that preserve variance. This is why covariance, eigenvectors and eigenvalues appear: covariance tells how dimensions move together; eigenvectors give principal directions; eigenvalues give how much variance those directions carry.

Randomized PCA and incremental PCA exist because exact PCA can be expensive. Kernel PCA exists because straight-line components fail on curved data. ICA changes the target from maximum variance to independent sources; CCA changes it to relationships between two views; LLE preserves local geometry.

Lecture 3 -- autoencoders

reconstruction

An autoencoder trains a neural network to output what it received. The useful part is the constraint. An undercomplete code must compress. A sparse code must choose a few active features. A denoising AE must reconstruct stable structure rather than noise. A convolutional AE exploits local image structure.

Exam questions turn this into activation/loss choices and shape/parameter calculations. Binary output needs sigmoid/BCE; real output usually linear/MSE. Conv layer parameters are \((k_hk_wc_{in}+1)c_{out}\). Pooling reduces spatial size and strengthens compression pressure.

Lectures 4-5 -- likelihood and autoregressive models

density modeling

Likelihood-based modeling asks for \(p_\theta(x)\), not just a code. Histograms fail in high dimensions, so neural parameterizations are used. The key architecture trick is autoregressive factorization: any joint distribution can be written as a product of conditionals.

MADE implements this with masks. WaveNet implements it with causal dilated convolutions. PixelCNN implements it with masked 2D convolution over raster order. Gated PixelCNN fixes blind spots, PixelCNN++ improves distribution and architecture, and PixelSNAIL adds attention for long-range context.

Lectures 6-7 -- normalizing flows

invertible density

Flows learn an invertible differentiable map between data \(x\) and simple latent \(z\). The entire method is the change-of-variables formula. If the map is invertible, sampling is possible. If the Jacobian determinant is tractable, likelihood training is possible.

RealNVP uses affine coupling: hold one part fixed, scale/shift the other part, and alternate masks. MAF and IAF trade off fast density vs fast sampling. Glow, Flow++ and FFJORD are later flow families. Dequantization is needed because continuous flows do not directly fit discrete pixel values cleanly.

Lecture 8 -- mid-sem tips

exam drill

Lecture 8 says to revise backpropagation, CNN/convolution operations, parameter calculations, autoencoders, autoregressive models, flows and assignment interpretation. It explicitly warns that most questions are quantitative. The drills shown match the previous papers: kernel-size changes, convolutional AE dimensions, AE loss/update, RealNVP change of variables and MADE mask design.