Prerequisite material is overviewed in a series of videos that are available under the "Prerequisites" topic in the navigation bar, which is to the left of this text.

The lectures were produced using technology that is described in this video:
https://youtu.be/ltOxgb28ZKY

No.	PDF	Video
	Week   0
0	Class #00	Recall: Elementary Differential Calculus
1a		Calculus Overview
1b		Functions
1c		Basic Differential Calculus
1d		The Chain Rule
1e		Taylor Series
	Week   1
1	Class #01	Introduction To Optimization
1a		Course Overview
1b		Examples: Fermat's Problem
1c		Open Sets; Interior And Boundary Points
1d		Minimizers and Minima
2	Class #02	Minimizing By Approximation
2a		Approximation Of Data
2b		Polynomial Models
2c		Vandermonde Matrix For Quadratics
2d		Quadratic Examples
3	Class #03	Stationarity And Convexity
3a		Stationarity
3b		Conditions for Stationarity
3c		Convexity; Strict Convexity
3d		Gradient Inequality And Convexity
	Week   2
4	Class #04	Scalar Minimization
4a		Searching For A Scalar Minimizer
4b		Searching With A Fixed Stepsize
4c		Searching With a Variable Stepsize
4d		Armijo Backtracking
5	Class #05	Functions With A Vector Argument
5a		Functions of Multiple Variables
5b		The 1-Form as Linear Algebra
5c		Directional Derivative and the Chain Rule
5d		Gradient 1-Form And Jacobian Matrix
5e		Linear Forms And Quadratic Forms
5f		Level Curves
6	Class #06	Stationary Points
6a		Stationarity Example
6b		Conditions For Stationarity
6c		Second Derivative And Hessian Matrix
6d		Eigenvalues Of A Hessian Matrix
	Week   3
7	Class #07	Methods Using Steepest Descent
7a		Introduction To Descent
7b		Descent Directions
7c		Fixed-Stepsize Descent
7d		Backtracking Descent
8	Quiz #1	Basic Scalar Optimization
9	Class #09	Newton's Method
9a		Scaling Methods
9b		Manual Scaling
9c		Newton's Method
9d		Damped Newton's Method
	Week   4
10	Class #10	Nonlinear Least Squares
10a		GPS As Vector Optimization
10b		Descent Methods for NLS Problems
10c		The Levenberg-Marquardt Algorithm
10d		Fermat-Weber Problems
11	Test #1	Basic Scalar Optimization
12	Class #12	Convexity And Level Sets
12a		Monotonicity And Convexity
12b		Convex Functions
12c		Convex Sets
12d		Level Sets
12e		Gradient Inequality
	Week   5
		National Day For Truth And Reconciliation
13	Quiz #2	Adaptive Vector Optimization
14	Class #14	Artificial Neural Networks: Single Neuron
14a		Terms And Symbols For Neural Networks
14b		Activation Functions
14c		Steepest Descent For One Observation
14d		Batches Of Data
	Week   6
15	Class #15	Artificial Neural Networks: Hidden Layer
15a		Simple Neural Network
14b		Output Layer: Single Neuron
14c		Hidden Layer
14d		Example: Batch Data
16	Test #2	Adaptive Vector Optimization
17	Class #17	Back-Propagation Of Scale Factors
17a		Back-Propagation
17b		Recall: Computations
17c		Example: Single Observation
	Week   7
18	Class #18	Constrained Optimization
18a		Constraint Properties For A Minimizer
18b		Linear Constraints
18c		Linear Objective Functions
18d		Convex Problems
19	Quiz #3	Neural Networks and Back-Propagation
20	Class #20	Lagrange Multipliers
20a		Objective Gradient And Property Gradient
20b		Quadratic Objective And Quadratic Constraint
20c		Linear Objective And Non-Convex Constraint
20d		Existence Of Lagrange Multipliers
	Week   8
21	Class #21	The Lagrange Equations
21a		Single Linear Constraint
21b		Quadratic Objective With Linear Constraints
21c		Example Mechanical System
21d		Matrix Form Of Quadratic Problems
22	Test #3	Neural Networks and Back-Propagation
23	Class #23	Dual Formulation of Lagrange Equations
23a		Primal Form And Feasible Set
23b		Min-Max And Max-Min Expressions
23c		Closed Form For Minimization Step
23d		Dual Formulation For Quadratic Problems
	Week   9
24	Class #24	KKT Conditions For Constrained Optimization
24a		Linear Inequality Constraints
24b		Examples Of Linear Inequalities
24c		KKT Background And Examples
24d		Definition Of A KKT Point
25	Class #25	Geometry At KKT Points
25a		Algebraic Interpretation Of Lagrange Multipliers
25b		KKT Geometry Of Inactive Linear Inequalities
25c		KKT Geometry Of Active Linear Inequalities
25d		KKT Geometry Of Linear Equality
26	Class #26	Constrained Least Squares
26a		Ordinary Least Squares Problem
26b		Constrained Least Squares Problem
26c		KKT Conditions For Constrained Least Squares
	Week   9
27	Class #27	Tikhonov Regularization
27a		Objective With Quadratic Constraint
27b		Parameters For Tikhonov Regularization
27c		Discrete Total Variation Problem
27d		Tikhonov Regularization For Denoising
28	Quiz #4	Lagrange Multipliers and KKT Conditions
29	Class #29	The Lasso And Related Regularization
29a		Regularization For Standardized Data
29b		Ridge Regression And Lasso Regularization
29c		Lasso Selects Variables And Constrains Regression
29d		Elastic Net Blends Ridge Regression And Lasso
	Week   10
30	Class #30	The Support Vector Machine (SVM)
30a		Hyperplane Classification
30b		Hyperplane Optimization Using Support Vectors
30c		Hyperplane Margins
30d		Primal Formulation Of SVM
31	Test #4	Lagrange Multipliers and KKT Conditions
32	Class #32	Dual Formulation Of The SVM
32a		Design Matrix And Label Matrix
32b		Vectorization Of SVM Primal Formulation
32c		Dual Formulation Of SVM
	Week   11
33	Class #33	Slack Variables And Dual Formulations
33a		Separable Data Not Linearly Separable
33b		Slack Variables And Soft Margins
30c		Slack Variables In SVM Primal Formulation
33d		Slack Variables In SVM Dual Formulation
34	Class #34	Kernel Trick For Nonlinear SVM
34a		Nonlinearly Separable Data
34b		Kernel Functions
34c		Gram Matrix And The Kernel Trick
	Week   12
35	Class #35	Kernel Classification For SVM
35a		Convex Problem For Lagrange Multipliers
35b		Classifying Observations And The Kernel Trick
35c		Examples Using Gaussian Kernels
	Week   13
36	Class #36	Course Summary
	Symbols	Symbols used in these notes

Queen's University is situated on the territory of the Haudenosaunee and Anishinaabek.

Ne Queen's University e’tho nón:we nikanónhsote tsi nón:we ne Haudenosaunee táhnon Anishinaabek tehatihsnonhsáhere ne onhwéntsya.

Gimaakwe Gchi-gkinoomaagegamig atemagad Naadowe miinwaa Anishinaabe aking.