Prerequisite chain

Prerequisites for Transformer Architecture

Topics you need before working through Transformer Architecture. Direct prerequisites are listed first; transitive prerequisites (the chain reachable through them) follow.

Direct prerequisites (15)

Attention Mechanism Theorylayer 4, tier 2
Feedforward Networks and Backpropagationlayer 2, tier 1
Softmax and Numerical Stabilitylayer 1, tier 1
Adam Optimizerlayer 2, tier 1
Attention Mechanisms Historylayer 3, tier 2
Byte-Level Language Modelslayer 4, tier 3
Convolutional Neural Networkslayer 3, tier 2
Deep Learning (Goodfellow, Bengio, Courville)layer 0B, tier 1
Distributional Semanticslayer 2, tier 2
Linear Layer: Shapes, Bias, and Memorylayer 2, tier 1
Macroeconomic Time-Series Forecastinglayer 4, tier 3
Recurrent Neural Networkslayer 3, tier 2
RNNs for Signal Sequenceslayer 4, tier 3
Token Prediction and Language Modelinglayer 3, tier 2
Word Embeddingslayer 2, tier 2

Reachable through the chain (154)

These topics are not directly cited as prerequisites but are reached transitively by following the chain upward. Working through the direct prerequisites pulls these in.

Matrix Operations and Propertieslayer 0A, tier 1
Sets, Functions, and Relationslayer 0A, tier 1
Basic Logic and Proof Techniqueslayer 0A, tier 2
Linear Independencelayer 0A, tier 1
Vectors, Matrices, and Linear Mapslayer 0A, tier 1
Gram Matrices and Kernel Matriceslayer 1, tier 1
Inner Product Spaces and Orthogonalitylayer 0A, tier 1
Eigenvalues and Eigenvectorslayer 0A, tier 1
Matrix Normslayer 0A, tier 1
Distance Metrics Comparedlayer 1, tier 2
Metric Spaces, Convergence, and Completenesslayer 0A, tier 1
Non-Euclidean and Hyperbolic Geometrylayer 1, tier 2
Matrix Multiplication Algorithmslayer 1, tier 2
The Kernel Tricklayer 2, tier 1
Support Vector Machineslayer 2, tier 1
Convex Optimization Basicslayer 1, tier 1
Differentiation in Rⁿlayer 0A, tier 1
Continuity in Rⁿlayer 0A, tier 1
Common Inequalitieslayer 0A, tier 1
Common Probability Distributionslayer 0A, tier 1
Exponential Function Propertieslayer 0A, tier 1
Integration and Change of Variableslayer 0A, tier 2
Measure-Theoretic Probabilitylayer 0B, tier 1
Cardinality and Countabilitylayer 0A, tier 2
Kolmogorov Probability Axiomslayer 0A, tier 1
Random Variableslayer 0A, tier 1
Zermelo-Fraenkel Set Theorylayer 0A, tier 2
Dynamic Programminglayer 0A, tier 1
Graph Algorithms Essentialslayer 0A, tier 2
Greedy Algorithmslayer 0A, tier 2
GraphSLAM and Factor Graphslayer 3, tier 2
Inverse and Implicit Function Theoremlayer 0A, tier 2
The Jacobian Matrixlayer 0A, tier 1
Positive Semidefinite Matriceslayer 0A, tier 1
Submodular Optimizationlayer 3, tier 3
Taylor Expansionlayer 0A, tier 1
The Hessian Matrixlayer 0A, tier 1
Vector Calculus Chain Rulelayer 0A, tier 1
Convex Dualitylayer 2, tier 1
Subgradients and Subdifferentialslayer 1, tier 1
Logistic Regressionlayer 1, tier 1
Maximum Likelihood Estimation: Theory, Information Identity, and Asymptotic Efficiencylayer 0B, tier 1
Central Limit Theoremlayer 0B, tier 1
Law of Large Numberslayer 0B, tier 1
Expectation, Variance, Covariance, and Momentslayer 0A, tier 1
Joint, Marginal, and Conditional Distributionslayer 0A, tier 1
Triangular Distributionlayer 0A, tier 2
Borel-Cantelli Lemmaslayer 0B, tier 1
Modes of Convergence of Random Variableslayer 0B, tier 1
Characteristic Functionslayer 1, tier 1
Moment Generating Functionslayer 0A, tier 2
KL Divergencelayer 1, tier 1
Information Theory Foundationslayer 0B, tier 2
Total Variation Distancelayer 1, tier 1
Method of Momentslayer 0B, tier 2
Radon-Nikodym and Conditional Expectationlayer 0B, tier 1
Data Preprocessing and Feature Engineeringlayer 1, tier 1
Linear Regressionlayer 1, tier 1
The Elements of Statistical Learning (Hastie, Tibshirani, Friedman)layer 0B, tier 1
Naive Bayeslayer 1, tier 2
Perceptronlayer 1, tier 2
Loss Functionslayer 1, tier 2
Hypothesis Classes and Function Spaceslayer 2, tier 1
Empirical Risk Minimizationlayer 2, tier 1
Concentration Inequalitieslayer 1, tier 1
Martingale Theorylayer 0B, tier 2
Skewness, Kurtosis, and Higher Momentslayer 1, tier 1
High-Dimensional Probability (Vershynin)layer 2, tier 1
Cramér-Wold Theoremlayer 1, tier 2
Loss Functions Cataloglayer 1, tier 1
Robust Statistics and M-Estimatorslayer 3, tier 2
Minimax and Saddle Pointslayer 2, tier 2
Winsorizationlayer 1, tier 3
Order Statisticslayer 1, tier 2
Sequences and Series of Functionslayer 0A, tier 2
Understanding Machine Learning (Shalev-Shwartz, Ben-David)layer 1, tier 1
Ridge Regressionlayer 1, tier 1
Shrinkage Estimation and the James-Stein Estimator: Inadmissibility, SURE, and Brown's Characterizationlayer 0B, tier 1
Cramér-Rao Bound: Information Inequality, Achievability, and Sharper Variantslayer 0B, tier 1
Fisher Information: Curvature, KL Geometry, and the Natural Gradientlayer 0B, tier 1
Basu's Theoremlayer 0B, tier 3
Sufficient Statistics and Exponential Familieslayer 0B, tier 2
Minimax Lower Bounds: Le Cam, Fano, Assouad, and the Reduction to Testinglayer 3, tier 1
Empirical Processes and Chaininglayer 3, tier 2
Rademacher Complexitylayer 3, tier 1
VC Dimensionlayer 2, tier 1
Counting and Combinatoricslayer 0A, tier 2
PAC Learning Frameworklayer 1, tier 1
Uniform Convergencelayer 2, tier 1
Adaptive Learning Is Not IIDlayer 3, tier 2
Bernstein Inequalitylayer 2, tier 1
Bennett's Inequalitylayer 2, tier 1
Chernoff Boundslayer 1, tier 1
Hoeffding's Lemmalayer 1, tier 1
Realizability Assumptionlayer 2, tier 1
Slud's Inequalitylayer 2, tier 2
Bias-Complexity Tradeofflayer 2, tier 2
No-Free-Lunch Theoremlayer 2, tier 2
Glivenko-Cantelli Theoremlayer 2, tier 2
McDiarmid's Inequalitylayer 3, tier 1
Sub-Gaussian Random Variableslayer 2, tier 1
Epsilon-Nets and Covering Numberslayer 3, tier 1
Contraction Inequalitylayer 3, tier 2
Sub-Exponential Random Variableslayer 2, tier 1
Chi-Squared Concentrationlayer 2, tier 1
Symmetrization Inequalitylayer 3, tier 1
Asymptotic Statistics: M-Estimators, Delta Method, LANlayer 0B, tier 1
Measure Concentration and Geometric Functional Analysislayer 3, tier 1
Stochastic Processes for MLlayer 2, tier 2
Gauss-Markov Theoremlayer 2, tier 1
The Multivariate Normal Distributionlayer 0B, tier 1
Maximum A Posteriori (MAP) Estimationlayer 0B, tier 1
Bayesian Estimationlayer 0B, tier 2
Bayesian Linear Regressionlayer 2, tier 1
Conjugate Priorslayer 0B, tier 1
Matrix Calculuslayer 1, tier 1
Activation Functionslayer 1, tier 1
Automatic Differentiationlayer 1, tier 1
Decision Trees and Ensembleslayer 2, tier 2
Bias-Variance Tradeofflayer 2, tier 2
Elastic Netlayer 2, tier 2
Lasso Regressionlayer 2, tier 1
Generalized Additive Modelslayer 2, tier 2
MARS (Multivariate Adaptive Regression Splines)layer 2, tier 3
K-Nearest Neighborslayer 1, tier 2
Gradient Boostinglayer 2, tier 1
Gradient Descent Variantslayer 1, tier 1
AdaBoostlayer 2, tier 2
Cubist and Model Treeslayer 2, tier 3
Tensors and Tensor Operationslayer 0A, tier 1
Pandas and NumPy Fundamentalslayer 4, tier 3
Singular Value Decompositionlayer 0A, tier 1
Information Retrieval Foundationslayer 2, tier 1
Stochastic Gradient Descent Convergencelayer 2, tier 1
Coordinate Descentlayer 2, tier 2
Mirror Descent and Frank-Wolfelayer 3, tier 2
Online Convex Optimizationlayer 3, tier 2
No-Regret Learninglayer 3, tier 2
Projected Gradient Descentlayer 2, tier 2
Proximal Gradient Methodslayer 2, tier 1
Quasi-Newton Methodslayer 2, tier 1
Newton's Methodlayer 1, tier 1
Line Search Methodslayer 2, tier 2
Secant Methodlayer 1, tier 2
Fast Fourier Transformlayer 1, tier 2
Complex Numbers for Fourierlayer 0A, tier 2
Signals and Systems for MLlayer 1, tier 2
Skip Connections and ResNetslayer 2, tier 1
SVM for RF Classificationlayer 4, tier 3
Time Series Forecasting Basicslayer 2, tier 2
Time Series Foundationslayer 2, tier 2
Tokenization and Information Theorylayer 4, tier 3
NLP for Economic Text Analysislayer 4, tier 3
Natural Language Processing Foundationslayer 2, tier 2