UDL question bank -- questions only, no answers.

Active-recall bank from UDL lectures 1-8, previous-year EC-2 papers and Lecture 8 tips. No answers here. Use solved paper only after writing an attempt.

Lecture 1 -- overview and generative-model map

1. Define unsupervised deep learning and distinguish representation learning, density estimation and generation. concept
2. Why does unlabeled data still contain learnable structure? Give image, text and time-series examples. concept
3. List the course model families: PCA variants, autoregressive models, normalizing flows, autoencoders/VAE, GANs, diffusion, energy-based models, self/semi-supervised learning. recall
4. Explain LeCun's "cake" view: why unsupervised/self-supervised learning matters for common sense. concept
5. Compare explicit likelihood, implicit generative and self-supervised objectives. compare

Lecture 2 -- PCA, ICA, CCA, LLE

1. What is factor/component analysis? Why do we seek hidden factors instead of raw dimensions? concept
2. Define covariance and explain positive vs negative covariance. define
3. Derive PCA as variance maximization / projection-error minimization. derive
4. How many principal components can an \(n\)-dimensional dataset have, and where does dimensionality reduction enter? concept
5. Explain eigenvalues/eigenvectors of covariance matrix in PCA. derive
6. Compare standard PCA, randomized PCA, incremental PCA and kernel PCA. compare
7. What are two disadvantages of incremental PCA? recall
8. Why does kernel PCA help nonlinear data? What is the cost? concept
9. Compare PCA and ICA. Why does ICA need non-Gaussian independent sources? compare
10. Give one ICA application. recall
11. Explain CCA/KCCA and one cross-modal retrieval use case. scenario
12. Explain LLE's local-neighbor reconstruction idea. concept
13. State computational complexity of standard PCA, randomized PCA and kernel PCA for \(N\) samples of dimension \(d\). numerical

Lecture 3 -- autoencoders

1. Define encoder, code, decoder and reconstruction loss. define
2. Why is an undercomplete autoencoder forced to learn useful features? concept
3. When is a linear autoencoder equivalent to PCA? concept
4. Why can an overcomplete autoencoder learn a trivial identity? How do regularizers prevent it? concept
5. Compare sparse, denoising, convolutional and deep autoencoders. compare
6. For binary reconstruction, what output activation and loss should be used? concept
7. For real-valued reconstruction, what output activation and loss are natural? concept
8. Compute BCE loss for a 2-output binary autoencoder when output is \((0.62,0.5)\) and target is \((1,0)\). numerical
9. Derive one-step backprop update for a simple 2D autoencoder weight. derive
10. Solve the 1D linear AE with \(X\sim U[-2,2]\), weights \(\alpha,\beta\), MSE plus \(0.5(\alpha^2+\beta^2)\). numerical
11. Calculate Conv2D output shapes and trainable parameters for a convolutional autoencoder. numerical
12. What happens when MaxPooling layers are removed from a convolutional autoencoder? scenario

Lectures 4-5 -- likelihood and autoregressive models

1. Why does histogram density estimation fail in high dimensions? Use binary MNIST count size. concept
2. Write maximum likelihood objective for \(p_\theta(x)\). derive
3. Why must a likelihood model define a valid normalized probability distribution? concept
4. Write chain-rule autoregressive factorization for \(D\) variables. derive
5. Why is autoregressive likelihood tractable but sampling slow? concept
6. Explain MADE masks for \(p(x_1)p(x_2|x_1)p(x_3|x_1,x_2)\). derive
7. Generate a binary sample from trained MADE step by step. procedure
8. Pros and cons of training MADE with more masks. compare
9. Explain WaveNet causal dilated convolution and receptive field growth. concept
10. How does Parallel WaveNet speed up WaveNet? concept
11. Explain PixelCNN raster ordering and masked convolution. concept
12. Why does standard PixelCNN have blind spots? How does Gated PixelCNN fix them? compare
13. What advantages does PixelCNN++ offer over PixelCNN/Gated PixelCNN? compare
14. What advantages does PixelSNAIL offer over PixelCNN++? compare

Lectures 6-7 -- normalizing flows

1. What do flows add that autoregressive models lack? concept
2. Write change-of-variables density formula for \(z=f_\theta(x)\). derive
3. Why must flow functions be invertible and differentiable? concept
4. How do you train, evaluate likelihood and sample from a flow? procedure
5. For \(z\sim U[0,1]\), \(x=z^2\), derive \(p_X(x)\). numerical
6. Check invertibility of \(x^3-x^5+2x^7\), \(\exp(-|x|)\), \(x^2-x\). numerical
7. Explain RealNVP affine coupling and log determinant. derive
8. What masking patterns are used in RealNVP? Why alternate masks? concept
9. Compare MAF and IAF for likelihood evaluation and sampling. compare
10. Explain Glow's invertible \(1\times1\) convolution at high level. concept
11. Why is dequantization needed for flow models on image data? concept
12. For a continuous Gaussian variable, what is \(P(x=0)\)? gotcha

Previous-paper style drills

1. Given a small image and a \(3\times3\) kernel, compute transposed convolution output for valid padding and stride 2. numerical
2. Given Laplace samples, find MLE \((\mu,\sigma)\). numerical
3. Given a RealNVP coupling equation and \(x=[2,3]^T\), compute \(y\) and log determinant. numerical
4. Given a MADE node-number diagram, design all masks and final connections. diagram
5. Given a convolutional AE Keras encoder, compute every layer shape and first-layer parameters. numerical
6. Given \(3\times3\to7\times7\) kernel change, discuss parameter count, training time, receptive field and reconstruction accuracy. scenario