CISC271, Linear Data Analysis: Lectures

Description:

These lectures are approximately aligned with classes in the course notes. There may be differences between the notes and the videos because these evolve over time.

Prerequisite material is overviewed in a series of videos that are available under the "Prerequisites" topic in the navigation bar, which is to the left of this text.

The lectures were produced using technology that is described in this video:
https://youtu.be/ltOxgb28ZKY

No. PDF Video
Week   1
1 Class #01 Introduction To Linear Methods for AI
1a Course Overview
1b Course Organization
1c Matrix Columns
1d Eigenfacts
2 Class #02 Graphs: Adjacency Matrix and Laplacian Matrix
2a Introduction To Graphs
2b Relevant Definitions For Graphs
2c The Adjacency Matrix
2d Non-Bipartite Graphs
2e The Degree Matrix
2f A Laplacian Matrix
2g Properties Of A Laplacian Matrix
2h The Fiedler Vector
3 Class #03 Vector Spaces
3a Introduction To Vector Spaces
3b Block Partitioning A Matrix
3c Vector-Space Properties
3d The Column Space
3e The Null Space
Week   2
4 Class #04 Spanning Sets And Basis Vectors
4a Introduction To Vector Spaces
4b Relevant Definitions For Basis Vectors
4c Basis Vectors For A Column Space
4d Orthogonal Subspaces
5 Class #05 Diagonalizable Matrices
5a Similar Matrices
5b Diagonalizability
5c Examples of Diagonalizability
5d The Matrix Square Root
6 Class #06 Spectral Decomposition and Positive [Semi-]Definite Matrices
6a Real Normal Matrices
6b Real Orthogonal Matrices
6c Real Symmetric Matrices
6d The Spectral Theorem
6e Positive Definite Matrices
6f The Quadratic Form
6g Mean And Variance Of Data
6h The Covariance Matrix
Week   3
7 Class #07 Design Matrix And Standardized Data
7a Variables And Observations
7b Variables As Vectors
7c Zero-Mean Data
7d Unit-Variance Data
7e Standardized Data For Regression
7f Measuring Standard Deviations
8 Class #08 Orthogonal Projection
8a Concepts In Orthogonal Projection
8b Projecting A Vector To A Vector
8c Projecting A Vector To A Vector Space
8d The Normal Equation
8e Overdetermined Linear Equations
9 Quiz #1 Matrix Properties and Vector Spaces
Week   4
10 Class #10 Patterns - Linear Regression
10a Concepts In Statistical Regression
10b Concepts In Linear Regression
10c Examples Of Linear Regression
10d Data Standardization For Linear Regression
10e Residual Error In Linear Regression
11 Class #11 Cross-Validating Linear Regression
11a Validation Of Linear Regression
11b Training Sets And Testing Sets
11c K-Fold Cross-Validation Of Linear Regression
11d Examples Of 5-Fold Cross-Validation
12 Test #1 Matrix Properties and Vector Spaces
Week   5
13 Class #13 Singular Value Decomposition, Or SVD
13a Introduction To The SVD
13b The Left-Transpose Product
13c The Right-Transpose Product
13d Structure Of The SVD
14 Class #14 Orthonormal Basis Vectors And The SVD
14a I Did Not Shoot The SVD
14b Examples Of The SVD
14c Matrix Spaces And The SVD
14d The Null Space And The SVD
14e Orthonormal Basis Vectors And The SVD
15 Quiz #2 Matrices and Linear Regression
Week   6
16 Class #16 Principal Components Analysis, Or PCA
16a Introduction To PCA
16b PCA From Covariance Matrix
16c PCA As Spectral Decomposition
16d Computing Data Scores Using PCA
16e Matrix Norms
17f L2 Matrix Norm And Frobenius Matrix Norm
16g Matrix Series From The SVD
17 Class #17 PCA - Matrix Algebra and Dimensionality Reduction
17a Revisiting PCA
17b The Scatter Matrix Of Variables For PCA
17c PCA As Matrix Approximation
17d PCA As Dimensionality Reduction
17e Low-Rank Approximations
17f Scree Plot Of Singular Values
18 Test #2 Matrices and Linear Regression
Reading Week
Holiday Special
Week   7
19 Class #19 Unsupervised Learning - K-Means Clustering
19a A Conceptual Hierarchy Of Machine Learning
19b Hyperplane of Separation
19c Basics Of Vector Clustering
19d A K-Means Clustering Algorithm
19e Clustering The Iris Data
20 Class #20 Classification - Linear Separability
20a Separating Two Clusters
20b A Hyperplane From Cluster Centroids
20c Hyperplanes For Multiple Clusters
20d The Davies-Bouldin Index For Clusters
21 Quiz #3 The SVD, PCA, And Dimensionality Reduction
Week   8
22 Class #22 Classification - Assessment With Confusion Matrix
22a Data Labels As Dependent Variables
22b Confusion Matrix For Binary Labels
22c Example Of Confusion Matrix
23 Class #23 Classification - Assessment With ROC Curve
23a Receiver Operator Characteristic, Or ROC
23b ROC And The Confusion Matrix
23c Example ROC Curve For Fictitious Virus
24 Test #3 The SVD, PCA, And Dimensionality Reduction
Week   9
25 Class #25 Odds Of Occurrence And Probability
25a Odds Of Hyperplane Classification
25b Odds And Probability
25c The Logistic Function For Odds
25d Properties Of The Logistic Function
26 Class #26 Classification - Single Artificial Neuron
26a Artificial Neurons
26b Data Flow And Computations For Neurons
26c Hyperplane Separation For Neurons
27 Quiz #4 Classification Assessment, Odds Of Occurrence
Week   10
28 Class #28 Classification - Logistic Regression
28a Shortcomings Of Perceptrons
28b Scores From Logistic Activation
28c Residual Error Of Scores
28d Logistic Regression For Iris Data
29 Class #29 Nonlinear Separation - Embeddings And Gram Matrix
29a Some Data Are Nonlinearly Separable
29b Embedding A Vector Space
29c Gram Matrix For An Embedding
30 Test #4 Classification Assessment, Odds Of Occurrence
Week   11
31 Class #31 Nonlinear Separation - Kernel PCA
31a High-Dimensional PCA
31b Scatter Matrix Of Observations
31c Kernel PCA Using The Gram Matrix
31d Kernel PCA For Iris Data
32 Class #32 Spectral Clustering Of Data
32a Fielder Vector And Spectral Decomposition
32b Spectral Clustering Using Eigenvectors
32c Distance Matrix And Spectral Clustering
33 Quiz #5 Machine Learning
Week   12
34 Class #34 The Curse Of Dimensionality
34a The Curse of Dimensionality
34b Dimensionality And Hypercube Vertices
34c Dimensionality And Uniform Distributions
36d Dimensionality And Gaussian Distributions
35 Class #35 Course Summary
35a Course Summary
36 Test #5 Machine Learning
37b Unplanned Recording Events
Extra Material
Table of Contents
References


Queen's University is situated on the territory of the Haudenosaunee and Anishinaabek.

Ne Queen's University e'tho nón:we nikanónhsote tsi nón:we ne Haudenosaunee táhnon Anishinaabek tehatihsnonhsáhere ne onhwéntsya.

Gimaakwe Gchi-gkinoomaagegamig atemagad Naadowe miinwaa Anishinaabe aking.


Last updated