Probability reinterpreted through linear algebra: random variables as vectors, expectation as a linear functional, and covariance as a weighted inner product.

Lesson 14

Covariance Geometry

Covariance as geometry: an inner-product viewpoint that turns correlation into cosine, PCA into diagonalization, and whitening into a rotate-and-rescale operation.

Lesson 15

Multivariate Gaussian

The multivariate Gaussian as geometry plus probability: covariance creates ellipsoids, the precision matrix weights distance, and whitening turns the distribution spherical.

Course section

Statistical Modeling for Machine Learning

4 lessons

Lesson 16

Maximum Likelihood Estimation

Maximum likelihood estimation as the rule for choosing parameters that make observed data most plausible.

Lesson 18

MAP Estimation

MAP estimation as maximum likelihood plus prior information, with Gaussian priors leading to L2 regularization.

Lesson 19

Bayesian Estimation Introduction

Bayesian estimation as maintaining a full posterior distribution over parameters instead of collapsing immediately to one best value.

Lesson 20

Bayesian Estimation Update

Bayesian updating as repeated evidence accumulation, where each posterior becomes the next prior.

Course section

Loss Functions and Optimization Foundations

1 lesson

Lesson 17

Loss Functions

Loss functions as negative log-likelihoods that turn probabilistic assumptions into optimization objectives.

Course section

Statistical Modeling and Generalization

5 lessons

Lesson 21

Bayesian Predictive Distributions

Bayesian predictive distributions as averaging future predictions over parameter uncertainty.

Lesson 22

Model Complexity

Model complexity as the tradeoff between expressive power, fitting the signal, and resisting overfitting.

Lesson 23

Regularization

Regularization as adding a preference for simpler parameter choices so models generalize instead of chasing noise.

Lesson 24

Early Stopping

Early stopping as choosing the checkpoint with the best validation performance instead of the final training checkpoint.

Lesson 25

Dropout

Dropout as training-time random neuron removal that discourages co-adaptation and improves generalization.

Course section

Optimization and Calculus for Machine Learning

8 lessons

Lesson 26

Gradient Descent

Gradient descent as a local movement rule for minimizing loss functions.

Lesson 27

Stochastic Gradient Descent

Stochastic gradient descent as a noisy but practical approximation to full gradient descent.

Lesson 28

Momentum

Momentum as gradient descent with velocity: smoothing noisy SGD updates by remembering previous directions.

Lesson 29

Adam

Adam as an adaptive optimizer that combines momentum-like gradient memory with per-parameter scaling from squared gradients.

Lesson 30

Learning Rate Schedules

Learning rate schedules as deliberate changes to the base step size during training.

Lesson 31

Hessians and Curvature

Second derivatives, Hessians, and curvature as tools for understanding optimizer behavior.

Lesson 32

Convexity

Convex sets and convex functions as the geometry that makes local optimization globally reliable.

Lesson 33

Constrained Optimization

Feasible sets, active constraints, and Lagrange multipliers as a geometric method for optimizing under restrictions.

Course section

Calculus and Matrix Calculus for ML

6 lessons

Lesson 34

Neural Networks

1 lesson

Lesson 40

Perceptrons and Linear Layers

A shape-aware introduction to perceptrons and linear layers, connecting affine maps, decision boundaries, activations, and trainable parameters.