Textbooks, open notes, classic papers and practical companion resources for NLP fundamentals: vector semantics, TF-IDF, PPMI, embeddings, language modelling, smoothing, neural networks, POS tagging, HMMs, CRFs, transformers and modern tagging workflows.
Textbooks
- T1Daniel Jurafsky & James H. Martin — Speech and Language Processing, 3rd ed. draft. The best free reference for tokenization, vector semantics, embeddings, n-gram language models, neural networks, HMM tagging, CRFs, transformers and applications. web.stanford.edu/~jurafsky/slp3
- T2Christopher D. Manning & Hinrich Schütze — Foundations of Statistical Natural Language Processing. Classic statistical NLP reference for probability, corpus counts, tagging, parsing, information retrieval and lexical statistics. nlp.stanford.edu/fsnlp
- T3Jacob Eisenstein — Natural Language Processing. Modern, mathematically clear textbook covering classification, sequence models, parsing, semantics and neural methods. PDF draft
Open learning resources
- C1Stanford CS224n — Natural Language Processing with Deep Learning. Strong companion for word vectors, neural networks, recurrent models, attention, transformers, BERT and applied NLP. web.stanford.edu/class/cs224n
- C2James Allen — Natural Language Understanding materials. Useful for classical framing: language understanding, ambiguity, semantics, dialogue and symbolic/statistical foundations. cs.rochester.edu/u/james
- C3Stanford NLP Group resources. Papers, tools, datasets and notes around statistical and neural NLP. nlp.stanford.edu
- C4Hugging Face NLP tutorials. Practical guide for transformer token classification, fine-tuning, pipelines and model evaluation. huggingface.co/learn/nlp-course
Papers
| Topic | Reference | Why it matters |
|---|---|---|
| Word2Vec | Tomas Mikolov, Kai Chen, Greg Corrado & Jeffrey Dean (2013) — “Efficient Estimation of Word Representations in Vector Space.” arXiv:1301.3781 | Introduces efficient CBOW and skip-gram training that made dense word vectors widely practical. |
| Negative sampling | Tomas Mikolov et al. (2013) — “Distributed Representations of Words and Phrases and their Compositionality.” arXiv:1310.4546 | Develops negative sampling and phrase/vector regularities for scalable Word2Vec-style embeddings. |
| GloVe | Jeffrey Pennington, Richard Socher & Christopher D. Manning (2014) — “GloVe: Global Vectors for Word Representation.” nlp.stanford.edu/pubs/glove.pdf | Connects global co-occurrence matrix factorization with dense vector semantics. |
| Viterbi | Andrew J. Viterbi (1967) — “Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm.” doi.org/10.1109/TIT.1967.1054010 | Original dynamic-programming decoding algorithm later used for HMM tagging and many sequence models. |
| HMM tutorial | Lawrence R. Rabiner (1989) — “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition.” doi.org/10.1109/5.18626 | Canonical explanation of HMM evaluation, decoding, learning and speech/NLP applications. |
| Bi-LSTM-CRF | Zhiheng Huang, Wei Xu & Kai Yu (2015) — “Bidirectional LSTM-CRF Models for Sequence Tagging.” arXiv:1508.01991 | Shows the now-classic neural sequence-tagging architecture: contextual Bi-LSTM features plus CRF decoding. |
| Transformer | Ashish Vaswani et al. (2017) — “Attention Is All You Need.” arXiv:1706.03762 | Introduces the transformer architecture based on self-attention, the foundation of BERT, RoBERTa and many LLMs. |
| BERT | Jacob Devlin, Ming-Wei Chang, Kenton Lee & Kristina Toutanova (2018) — “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” arXiv:1810.04805 | Defines masked-language-model pretraining for bidirectional transformer encoders and strong fine-tuning for tagging/classification tasks. |
Tools & practical references
- P1NLTK Book. Gentle Python-first introduction to corpora, tagging, classification, parsing and lexical resources. nltk.org/book
- P2spaCy documentation. Practical reference for tokenization, POS tagging, named-entity recognition, pipelines and production NLP. spacy.io/usage
- P3Gensim documentation. Useful for Word2Vec, document vectors, similarity search and topic modelling workflows. radimrehurek.com/gensim
- P4scikit-learn text feature extraction. TF-IDF vectorizers, sparse matrices, cosine-like linear models and classification baselines. scikit-learn.org text features
- P5Natural Language Processing vault companions. Use the cheatsheet for compact revision and the formula sheet for equations such as \(\cos\theta=\frac{a\cdot b}{\|a\|\|b\|}\) and \(H=-\frac{1}{N}\sum_i\log P(w_i)\).