An orientation to the Fall 2024 Data Science Bootcamp.

We discussed a general supervised machine learning framework, simple linear regression, multiple linear regression, and data splits for predictive modeling. Note that we delayed kNN to next week.

Singular Value Decomposition and Principle Component Analysis

Hypothesis testing (null and alternative hypothesis, test size and power, p-values), confidence intervals, part of linear regression inference. Will finish linear regression inference next week.

Due to a technical glitch, the audio for the video Alec made to introduce himself and relevant information about projects didn't work in the orientation lecture. You can watch it now!

We derive the MLE estimates of multiple linear regression model parameters. This shows that OLS is MLE for linear regression. We derive the normal equation two ways: linear algebra and calculus.

Bias/Variance tradeoff, regularization, principle component analysis, and feature selection approaches.

We prove that the F statistic for comparing a full to reduced linear model is indeed F distributed under the null hypothesis that the reduced model is correct. Note: correction included at the end!

We discuss Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP) Estimation of model parameters. We address estimating the parameters of binomial and normal distributions from a sample.

k-Nearest Neighbors, data leakage, categorical features, non-linear transformations, pipelines, linear regression diagnostic plots.

Regularization as MAP estimation, Ridge regression using "pseudo-observations", understanding ridge regression as shrinking the OLS solution along principle component axes in a data adaptive manner.

Close to the end of Math Hour 5 I used the wrong definition of the F-Statistic. Essentially I used the squared cosine of an angle instead of the squared tangent. I correct that in this video.