Natural Language Processing — question bank, no answers.

Questions extracted and expanded from the BITS Natural Language Processing lecture notes NLP1–NLP7 — questions only, no answers. Use these for active recall before checking the NLP vault and the formula sheet.

NLP intro, applications, ambiguity & why language is hard

1. State the main goal of Natural Language Processing and explain what it means for a computer to analyze, understand and generate human language.
2. List the course objectives for the NLP mid-semester portion and convert each objective into one exam-ready skill.
3. Why is NLP considered a branch of Artificial Intelligence, and what does it borrow from linguistics, psycholinguistics, cognitive science and statistics?
4. Why is text described as one of the largest repositories of human knowledge? Give examples of text sources that an NLP system may need to process.
5. Explain why language technologies are valuable for companies that handle customer calls, web pages, journals, books, expert systems or translation.
6. List at least eight application areas of NLP from the lectures and state the central language task in each.
7. Compare question answering, text categorization, text mining, machine translation, language learning, usage checking and spelling correction as NLP applications.
8. What is the difference between text-to-speech and speech recognition as NLP-related technologies?
9. How can NLP support healthcare, information extraction, automatic summarization, fake-news detection, cyberbullying detection and social-media monitoring?
10. What roles do NLTK, spaCy, Stanford CoreNLP, Gensim, GPT-style models, AllenNLP and neural parsers play in the NLP ecosystem?
11. What preprocessing steps are commonly applied to raw text before modeling, and why might punctuation removal, tokenization, stop-word removal, stemming and lemmatization change downstream results?
12. Why is NLP a “big deal” even though language looks obvious to human speakers?
13. Explain the phrase \(L=\text{words}+\text{rules}+\text{exceptions}\) in the context of natural language difficulty.
14. Give examples of noisy NLP input sources, including electronic text, printed text and acoustic speech signals, and explain why each source can be hard to process.
15. Define structural ambiguity and analyze the ambiguity in “Visiting relatives can be a nuisance.”
16. Define grammatical ambiguity and explain why a word such as “can” may require contextual disambiguation.
17. Define lexical ambiguity and distinguish polysemy from homonymy using “understand” and “bank”.
18. Arrange morphological, lexical, syntactic, semantic, discourse and pragmatic analysis from lower-level to higher-level processing, and state what each level studies.
19. Why can a sentence be lexically correct but syntactically incorrect? Use “Large have green ideas nose” as the example.
20. Why can a sentence be syntactically correct but semantically odd? Use “Green ideas have large noses” as the example.
21. Why can a sentence be semantically possible but pragmatically strange? Use “She cuts banana with a pen” as the example.
22. How does discourse knowledge help interpret pronouns or later sentences such as “She is a Professor at BITS Pilani”?
23. What questions should be asked when evaluating language-understanding systems with complex outputs such as translations or summaries?
24. Why may human judgments for NLP evaluation be unreliable, and how can sample representativeness affect reported performance?

Lexical & vector semantics, TF-IDF, PPMI & cosine similarity

1. What is lexical semantics, and why are word senses more useful than treating words as raw strings or vocabulary indices?
2. Define lemma, sense and concept, and explain why a lemma such as “mouse” can have multiple senses.
3. What does it mean for words and concepts to have a many-to-many association?
4. Define synonymy and explain why most practical synonymy is approximate rather than perfect truth-preserving substitution.
5. Why are “big” and “large” not always interchangeable even though they are often treated as similar?
6. Differentiate word similarity from word relatedness using pairs such as coffee–tea and coffee–cup.
7. What is a semantic field, and how do hospital, restaurant and house vocabularies illustrate structured semantic relations?
8. Define antonymy and distinguish binary oppositions, scalar opposites and reversives.
9. What is connotation, and how do valence, arousal and dominance capture affective meaning?
10. State the distributional hypothesis and explain how it lets us infer the meaning of a new word such as “ongchoi”.
11. Why do words that occur in similar contexts tend to have similar meanings?
12. How can vector semantics help a question-answering system connect “tall” with “height”?
13. What is a term-document matrix, and how does each row or column represent a word or document?
14. How can document vectors make two comedies appear more similar to each other than to history plays?
15. What is a word-context or term-context matrix, and how is it different from a term-document matrix?
16. Why are raw co-occurrence counts useful but also problematic for representing meaning?
17. Define the dot product between two vectors and explain why it can act as a similarity score.
18. Why does raw dot product unfairly favor frequent words such as “the”, “of” and “you”?
19. Define vector length and explain why longer vectors tend to have larger dot products.
20. Write the cosine similarity formula \(\cos(\theta)=\frac{\vec a\cdot\vec b}{\|\vec a\|\|\vec b\|}\) and state what values near \(-1\), \(0\) and \(1\) mean.
21. Why does cosine similarity for non-negative count vectors usually range from \(0\) to \(1\)?
22. What is term frequency, and why might we use \(\operatorname{tf}_{t,d}=\log_{10}(\operatorname{count}(t,d)+1)\) instead of raw counts?
23. Define document frequency \(\operatorname{df}_t\) and explain why it is not the same as collection frequency.
24. Why should words appearing in nearly every document receive low inverse document frequency?
25. Write the idf idea using \(N\), the number of documents, and \(\operatorname{df}_t\), and explain why rare-but-useful terms get higher weights.
26. How does TF-IDF balance frequency inside one document with distinctiveness across the whole collection?
27. Why might a word like “good” receive TF-IDF weight zero if it appears in every document?
28. Compare count vectors, TF-IDF vectors and co-occurrence vectors as frequency-based embeddings.
29. Why are frequency-based vectors usually long and sparse?
30. What is Pointwise Mutual Information, and what intuition does \(\operatorname{PMI}(w,c)=\log_2\frac{P(w,c)}{P(w)P(c)}\) capture?
31. Why does Positive PMI use \(\operatorname{PPMI}(w,c)=\max(\operatorname{PMI}(w,c),0)\)?
32. When would PPMI be preferred to raw co-occurrence counts for semantic similarity?
33. How can sparse vectors fail to capture synonymy when “car” and “automobile” are separate dimensions?

Word embeddings, Word2Vec, CBOW, GloVe & t-SNE

1. Define a word embedding and explain why it is described as a point in semantic space.
2. Why does representing the previous word as an embedding help sentiment analysis generalize from “terrible” to a similar unseen word?
3. Compare frequency-based embeddings with prediction-based embeddings.
4. Why are dense vectors often easier to use as machine-learning features than sparse count vectors?
5. What are static embeddings, and how are they different from contextual embeddings such as ELMo or BERT-style representations?
6. What is the basic self-supervised idea behind Word2Vec?
7. Why does Word2Vec train a classifier for a task we do not directly care about?
8. In skip-gram training, what are the target word, context window and outside words?
9. For a \(\pm2\) window around “apricot” in “tablespoon of apricot jam a”, list the positive context positions that would be generated.
10. What does the skip-gram classifier estimate when it computes \(P(+\mid w,c)\)?
11. Why is a dot product converted through a sigmoid function in skip-gram with negative sampling?
12. What are negative examples in SGNS, and why are they sampled from the lexicon?
13. Why might \(K=2\) to \(5\) negatives work for large datasets, while more negatives may be used for smaller datasets?
14. What are the target embedding matrix \(W\) and context embedding matrix \(C\) in SGNS?
15. Why is a word sometimes represented by \(w_i+c_i\) after SGNS training?
16. State the learning goal of SGNS in terms of increasing similarity for positive pairs and decreasing similarity for negative pairs.
17. What happens conceptually during one stochastic-gradient step for target “apricot”, positive context “jam” and negative contexts such as “Tolstoy” and “matrix”?
18. Why are the classifier weights kept as embeddings after the Word2Vec training task is finished?
19. How does context-window size affect the kind of nearest neighbors learned by embeddings?
20. Why do small windows tend to find syntactically similar words, while larger windows tend to find semantically related field words?
21. How can embeddings trained on historical corpora be used to study semantic change over decades?
22. Why can embeddings reflect cultural, gender or ethnic bias from their training corpus?
23. Explain the analogy pattern “Paris : France :: Tokyo : x” and how vector arithmetic can support such reasoning.
24. What does the parallelogram method attempt to compute in analogical reasoning?
25. How does CBOW transform context words into a prediction task for the center word?
26. Compare skip-gram and CBOW in terms of training speed, rare-word quality and frequent-word accuracy.
27. What does GloVe stand for, and why is it described as an unsupervised method for global vectors?
28. How does GloVe combine global word-word co-occurrence statistics with local context information?
29. Define the GloVe co-occurrence matrix entry \(X_{ij}\), row total \(X_i\), and probability \(P_{ij}\).
30. Why does GloVe focus on ratios of co-occurrence probabilities rather than only raw probabilities?
31. What is t-SNE used for when visualizing high-dimensional embeddings?
32. Why can a two-dimensional t-SNE plot show clusters such as animals, fruits, numbers and gendered pairs while still losing some high-dimensional structure?
33. Compare count-based methods and direct-prediction methods in terms of training speed, statistics usage, corpus scaling and downstream task performance.

N-gram language models, smoothing, backoff, interpolation & perplexity

1. Define a language model as a model that computes \(P(W)\) or \(P(w_n\mid w_1,\ldots,w_{n-1})\).
2. Why can machine translation prefer “high winds tonight” over “large winds tonight” using language-model probabilities?
3. How does a language model help spell correction choose “about fifteen minutes from” over “about fifteen minuets from”?
4. How can speech recognition use language-model probabilities to prefer “I saw a van” over “eyes awe of an”?
5. State the chain rule for \(P(x_1,x_2,\ldots,x_n)\) and apply it to a sentence of words.
6. Why is the full chain-rule model impractical for long sentences?
7. Define the Markov assumption in language modeling.
8. What is a unigram language model, and what independence assumption does it make?
9. What is a bigram language model, and how do start and end symbols affect sentence probability?
10. How do trigram, 4-gram and 5-gram models extend the bigram idea?
11. Why are n-gram models insufficient for long-distance dependencies such as subject-verb agreement across a relative clause?
12. Define maximum likelihood estimation for a unigram or bigram probability.
13. How is \(P(w_i\mid w_{i-1})\) estimated from bigram counts?
14. Why do practical language models usually compute in log space?
15. What is underflow, and why does adding log probabilities help?
16. Why do n-grams overfit when the test corpus differs from the training corpus?
17. What happens to a full sentence probability if one test bigram has zero probability?
18. Explain the zero-count problem using “denied the offer” after seeing only “denied the allegations”, “denied the reports”, “denied the claims” and “denied the request”.
19. What is Laplace or add-one smoothing, and what does it mean to pretend every word was seen one more time?
20. Write the add-one bigram estimate for \(P(w_i\mid w_{i-1})\) using vocabulary size \(|V|\).
21. Why is add-one smoothing considered a blunt instrument for n-gram language models?
22. What are reconstituted counts after add-one smoothing, and why can they differ sharply from raw counts?
23. What is add-k smoothing, and how does it generalize add-one smoothing?
24. Distinguish backoff from interpolation in n-gram language modeling.
25. Why can using less context sometimes improve probability estimates?
26. How does linear interpolation combine unigram, bigram and trigram probabilities?
27. How are interpolation weights \(\lambda\) commonly chosen using a held-out corpus?
28. Distinguish open-vocabulary and closed-vocabulary language modeling tasks.
29. What is an out-of-vocabulary word, and why is an \(\langle\mathrm{UNK}\rangle\) token introduced?
30. How are \(\langle\mathrm{UNK}\rangle\) probabilities trained by replacing words outside a fixed lexicon?
31. Why are web-scale n-gram corpora often pruned by removing low-count n-grams?
32. What is “stupid backoff”, and why is it useful at web scale?
33. Distinguish extrinsic and intrinsic evaluation of language models.
34. Why is extrinsic evaluation often expensive but more meaningful for spelling correction, speech recognition or machine translation?
35. What is perplexity, and why is minimizing perplexity equivalent to maximizing test-set probability?
36. Explain perplexity as an average branching factor in the Shannon Game.
37. Why does lower perplexity usually indicate a better language model only when train and test distributions are comparable?

Neural networks, neural LMs, LLM/SLM & prompt engineering

1. What does a neural network unit compute from inputs, weights, bias and a nonlinear activation function?
2. Write the general unit computation \(z=\vec w\cdot\vec x+b\) followed by \(a=f(z)\).
3. Compare sigmoid, tanh and ReLU as activation functions.
4. Why is nonlinearity necessary in multilayer neural networks?
5. What is a perceptron, and why is it a linear classifier?
6. Why can a single perceptron compute AND or OR but not XOR?
7. Explain why XOR is not linearly separable in the \((x_1,x_2)\) plane.
8. How can a layered network solve XOR even when a single perceptron cannot?
9. What is a hidden representation, and why can hidden layers learn useful features?
10. Why are feedforward networks also called multilayer perceptrons?
11. How is binary logistic regression equivalent to a one-layer neural network with a sigmoid output?
12. How does a fully connected single-layer network with softmax output handle multiclass prediction?
13. Write the softmax form for output class \(i\), \(\operatorname{softmax}(z_i)=\frac{e^{z_i}}{\sum_j e^{z_j}}\).
14. What matrices and bias vectors are used in a two-layer network with hidden units and scalar output?
15. Why does adding a hidden layer to logistic regression allow nonlinear interactions between text features?
16. What is representation learning, and why are learned embeddings more powerful than hand-built binary word features?
17. How can a neural classifier handle documents of different lengths using padding, truncation, mean pooling or max pooling?
18. Define a neural language model as a model predicting \(w_t\) from previous words.
19. Why do simple feedforward neural language models use sliding windows for arbitrary-length sequences?
20. How does a neural language model use one-hot vectors and an embedding matrix to represent prior context words?
21. Why can neural LMs generalize from “cat gets fed” to “dog gets fed” better than n-gram LMs?
22. What does it mean to freeze a pretrained embedding layer during neural LM training?
23. When might it be better to learn embeddings simultaneously with the downstream neural model?
24. Compare n-gram LMs and neural LMs in terms of history length, generalization, accuracy, complexity, speed, energy use and interpretability.
25. What is cross-entropy loss, and why is it used to train logistic or softmax neural outputs?
26. How does gradient descent update neural-network weights using derivatives of the loss?
27. Why is backpropagation described as repeated use of the chain rule on a computation graph?
28. What is a Large Language Model, and how is it different from a smaller task-specific language model?
29. What are practical benefits and risks of using LLMs for NLP tasks compared with smaller language models?
30. What is prompt engineering, and why can phrasing affect completions from a generative language model?
31. Distinguish zero-shot and few-shot prompting with one example NLP task.
32. What is in-context learning, and how is it different from updating model weights?
33. How can prompt templates improve consistency for classification, summarization or extraction tasks?
34. Why are deep learning methods often data-hungry, and why can domain shift hurt NLP performance?
35. Define transfer learning and explain how pretrained models save time and computation for target NLP tasks.
36. Distinguish transductive transfer learning, inductive transfer learning, multitask learning, pretraining and fine-tuning.
37. Why does fine-tuning update pretrained layers, and when is it beneficial?

POS tagging, HMMs, Viterbi, MEMM & bidirectionality

1. Define part-of-speech tagging and write the input-output format for “the lead paint is unsafe”.
2. Why is POS tagging a sequence labeling problem rather than a set of independent word classifications?
3. List traditional parts of speech and explain why linguists disagree about the exact number and nature of categories.
4. Distinguish open-class words from closed-class words, with examples of each.
5. Why must a POS tagger choose a tagset before training or evaluation?
6. Compare a coarse tagset such as \(\{N,V,Adj,Adv\}\) with the Penn Treebank tagset of about 45 tags.
7. How are prepositions, subordinating conjunctions and the special word “to” treated in Penn Treebank-style tagging?
8. Use the word “back” to explain why one word form can have adjective, noun, adverb and verb tags.
9. Why does POS tagging matter for voice assistants, chatbots, search engines, information extraction and grammar checking?
10. How can POS tagging make an agentic AI system safer when interpreting “Set the timer for the oven to 10 minutes”?
11. Compare rule-based, statistical, neural, transformer-based and LLM-based POS tagging approaches in accuracy, speed, explainability and best use.
12. What is a Hidden Markov Model, and why is it called a generative sequence model?
13. Define states, observations, transition probabilities, emission probabilities and initial probabilities in an HMM for POS tagging.
14. What does the first-order Markov assumption say about \(P(q_i\mid q_1,\ldots,q_{i-1})\)?
15. How would a second-order HMM for POS tagging use the previous two tags?
16. Write the HMM joint probability idea for tags \(\mathbf t\) and words \(\mathbf w\).
17. What are tag transition probabilities \(P(t_i\mid t_{i-1})\), and why are \(P(NN\mid DT)\) and \(P(JJ\mid DT)\) often high?
18. What are word emission probabilities \(P(w_i\mid t_i)\), and why is \(P(\text{is}\mid VBZ)\) expected to be high?
19. How does an HMM disambiguate “race” in “to race tomorrow” using transition and emission probabilities?
20. Why can \(P(VB\mid TO)P(NR\mid VB)P(\text{race}\mid VB)\) beat the corresponding noun-tag path even if “race” is common as a noun?
21. What is the Viterbi algorithm used for in HMM POS tagging?
22. Why does Viterbi store backpointers as well as best path probabilities?
23. What does each column and row of a Viterbi table represent for a sentence of length \(T\) and \(N\) tags?
24. How is the first Viterbi column initialized using start-transition and emission probabilities?
25. Write the recurrence idea for computing the best score for tag \(t\) at position \(j\) from previous tags.
26. Why is Viterbi better than enumerating all possible tag sequences?
27. What are common HMM POS-tagging issues related to unknown words, limited context and sparsity?
28. How can smoothing help with unknown words or unseen transitions in HMM taggers?
29. What is a Maximum Entropy Markov Model, and how does it turn logistic regression into a discriminative sequence model?
30. How does an MEMM compute \(P(T\mid W)\) directly compared with an HMM using Bayes rule and \(P(W\mid T)\)?
31. Why are MEMMs attractive when many heterogeneous features are useful for tagging?
32. Give examples of features an MEMM might use when tagging “back” in “Janet will back the bill”.
33. What is greedy decoding in MEMMs, and why can hard left-to-right decisions reduce accuracy?
34. How can Viterbi decoding be used with MEMM scores to find a globally better tag sequence?
35. Why are purely left-to-right models limited for POS tagging?
36. How can bidirectional passes allow tags on the right to influence decisions about words on the left?
37. Why are modern taggers generally bidirectional?

Bi-LSTM-CRF, transformers, BERT/RoBERTa & LLM-agent POS tagging

1. What problem does a Bi-LSTM-CRF architecture solve for neural sequence tagging?
2. What information is provided by word embeddings and character embeddings at the input layer of a Bi-LSTM-CRF tagger?
3. Why are character embeddings useful for unknown words and morphologically rich words?
4. How does the forward LSTM differ from the backward LSTM in a bidirectional model?
5. Why can the tag for “bank” in “I walked to the river bank” require both left and right context?
6. What does the CRF layer add on top of Bi-LSTM token features?
7. Why can choosing the highest-probability tag independently at each word produce invalid or unlikely tag sequences?
8. What kind of tag-sequence constraints can a CRF learn, such as a verb being unlikely after a determiner?
9. What is the objective of the CRF path score numerator compared with all possible paths?
10. State typical Penn Treebank accuracy ranges for Bi-LSTM-CRF taggers and discuss why exact accuracy depends on dataset and setup.
11. List the main pros and cons of Bi-LSTM-CRF POS taggers.
12. What key idea lets transformers replace recurrence with self-attention?
13. Define self-attention in the context of POS tagging.
14. Why can self-attention connect distant words more directly than an RNN or LSTM chain?
15. What are multi-head attention and feed-forward networks inside transformer encoder blocks?
16. How is POS tagging performed by fine-tuning a pretrained language model such as BERT or RoBERTa?
17. What does pretraining teach a model before POS fine-tuning?
18. Why does a contextualized output vector for each token help solve lexical ambiguity?
19. What classification layer is added on top of BERT/RoBERTa outputs for POS tagging?
20. Why are transformer-based taggers highly parallelizable compared with recurrent taggers?
21. What accuracy range is commonly claimed for modern fine-tuned transformer POS taggers, and why should benchmark conditions be reported?
22. How does prompting frame POS tagging as a natural-language instruction for an LLM?
23. Write an exam-style prompt that asks an LLM to tag a sentence using the Penn Treebank tagset and output only word/tag pairs.
24. Distinguish zero-shot and few-shot POS tagging with LLMs.
25. What are risks of using an LLM for POS tagging without constrained output formatting?
26. What is an LLM agent, and how can it use a specialized POS tagger as an external tool?
27. In the request “Analyze the sentiment of all nouns in user reviews,” where does POS tagging fit inside an agent workflow?
28. Why might an agent combine LLM planning with a fast BERT POS tagger instead of asking the LLM to do everything?
29. If POS tagging has near-human performance, why does it still matter as an internal utility for larger AI systems?

Long & Numerical Exercises

Multi-part derivation, numerical and trace questions for subjective answers — still questions only. Show all intermediate steps when you attempt them.

1. L1 · TF-IDF and cosine similarity. Given documents \(D_1=\) “cat sat mat”, \(D_2=\) “dog sat mat”, and \(D_3=\) “cat chased dog”, build a term-document matrix for \(\{cat,dog,sat,mat,chased\}\), compute \(\operatorname{tf}_{t,d}=\log_{10}(c+1)\), compute \(\operatorname{idf}_t=\log_{10}(N/\operatorname{df}_t)\), form TF-IDF vectors for \(D_1\) and \(D_2\), and compute their cosine similarity.
2. L2 · Word-context cosine. For word vectors \(v_{cat}=[4,1,0,3]\), \(v_{dog}=[3,1,1,2]\), and \(v_{bank}=[0,4,5,0]\), compute all pairwise dot products, vector lengths and cosine similarities, then decide which pair is semantically closest by the numbers.
3. L3 · PPMI from a co-occurrence table. Given a word-context table with rows \(w\in\{cat,dog,bank\}\), columns \(c\in\{pet,river,money\}\), and counts \(\begin{bmatrix}8&0&0\\6&0&1\\0&5&7\end{bmatrix}\), compute \(P(w,c)\), \(P(w)\), \(P(c)\), \(\operatorname{PMI}(w,c)\), and \(\operatorname{PPMI}(w,c)\) for every nonzero cell.
4. L4 · Bigram MLE. From the corpus \(\langle s\rangle I am Sam \langle/ s\rangle\), \(\langle s\rangle Sam I am \langle/ s\rangle\), and \(\langle s\rangle I like ham \langle/ s\rangle\), compute unigram counts, bigram counts, \(P(I\mid\langle s\rangle)\), \(P(am\mid I)\), \(P(Sam\mid am)\), and the MLE probability of \(\langle s\rangle I am Sam \langle/ s\rangle\).
5. L5 · Add-one smoothing. Using the same corpus and vocabulary \(V=\{I,am,Sam,like,ham,\langle/ s\rangle\}\), compute add-one smoothed probabilities for \(P(like\mid I)\), \(P(Sam\mid I)\), and \(P(ham\mid am)\), then compare them with their unsmoothed MLE values.
6. L6 · Add-k smoothing. For context “want” with counts \(c(want,to)=608\), \(c(want,chinese)=6\), \(c(want,food)=0\), total \(c(want)=927\), vocabulary size \(|V|=1446\), and \(k=0.5\), compute the add-k estimates for the three next words.
7. L7 · Interpolated trigram probability. Given \(P(w_i\mid w_{i-2},w_{i-1})=0.18\), \(P(w_i\mid w_{i-1})=0.10\), \(P(w_i)=0.02\), and \((\lambda_3,\lambda_2,\lambda_1)=(0.6,0.3,0.1)\), compute the linearly interpolated probability and state which component contributes most.
8. L8 · Perplexity of a sentence. A bigram model assigns probabilities \(P(I\mid\langle s\rangle)=0.4\), \(P(want\mid I)=0.3\), \(P(food\mid want)=0.05\), and \(P(\langle/ s\rangle\mid food)=0.2\). Compute the sentence probability and perplexity for the four predicted tokens.
9. L9 · Skip-gram softmax step. Let target embedding \(w=[1,0]\), context embeddings \(c_{jam}=[1,1]\), \(c_{aardvark}=[0,1]\), \(c_{matrix}=[-1,1]\). Compute scores \(w\cdot c\), softmax probabilities over the three contexts, and the cross-entropy loss if the true context is “jam”.
10. L10 · SGNS sigmoid step. For target \(w=[0.4,0.2]\), positive context \(c^+=[0.5,0.1]\), negative context \(c^-=[-0.3,0.4]\), compute \(\sigma(w\cdot c^+)\), \(\sigma(-w\cdot c^-)\), and the negative-sampling loss for this one positive and one negative pair.
11. L11 · Feedforward unit forward pass. A neural unit has \(\vec w=[0.2,0.3,0.9]\), \(b=0.5\), and input \(\vec x=[0.5,0.6,0.1]\). Compute \(z=\vec w\cdot\vec x+b\), then compute the output under sigmoid, tanh and ReLU activations.
12. L12 · Softmax classifier. A three-class POS classifier produces logits \(z=[2.0,1.0,0.1]\) for tags \(NN,VB,JJ\). Compute the softmax probabilities and the cross-entropy loss if the correct tag is \(VB\).
13. L13 · HMM joint probability. For sentence “the dog barks” and tags \(DT,NN,VB\), with given values \(P(DT\mid *)=0.8\), \(P(NN\mid DT)=0.7\), \(P(VB\mid NN)=0.4\), \(P(STOP\mid VB)=0.6\), \(P(the\mid DT)=0.9\), \(P(dog\mid NN)=0.5\), and \(P(barks\mid VB)=0.3\), compute the full HMM joint probability.
14. L14 · Viterbi trace. Trace Viterbi for words “the dog runs” with tags \(D,N\). Use start probabilities \(P(D\mid S)=0.6\), \(P(N\mid S)=0.4\); transitions \(P(D\mid D)=0.1\), \(P(N\mid D)=0.9\), \(P(D\mid N)=0.2\), \(P(N\mid N)=0.8\); emissions \(P(the\mid D)=0.8\), \(P(dog\mid D)=0.05\), \(P(runs\mid D)=0.05\), \(P(the\mid N)=0.1\), \(P(dog\mid N)=0.6\), \(P(runs\mid N)=0.3\). Fill the dynamic-programming table, store backpointers and recover the best tag sequence.
15. L15 · MEMM feature reasoning. For the sentence “Janet will back the bill”, design MEMM features for tagging “back”, including current word, previous tag, suffix/prefix, capitalization, neighboring words and lexical cues, then explain how greedy decoding and Viterbi decoding would use the local classifier scores differently.
16. L16 · Bi-LSTM-CRF path scoring. Suppose a Bi-LSTM emits local scores for two tags \(N,V\) across three words and a CRF provides transition scores between tags. Set up the total score for paths \(N,N,V\), \(N,V,V\), and \(V,N,V\), then formulate the question of choosing the best path without independently maximizing each word.
17. L17 · Prompted POS tagging quality check. Design a zero-shot and a few-shot prompt for Penn Treebank POS tagging of “The old man the boats”, specify the exact output format constraints, and list the checks you would apply to detect malformed or hallucinated LLM output.