Statistical Learning Videos Book: Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy

Supervised Learning

Regression and classification is classified by its output types. Regression when we predict quantitative outputs, and clas- sification when we predict qualitative outputs.

• Least squares assumes f(x) is well approximated by a globally linear function.

• k-nearest neighbors assumes f(x) is well approximated by a locally constant function.

• One fact should be clear by now. Any method that attempts to pro- duce locally varying functions in small isotropic neighborhoods will run into problems in high dimensions—again the curse of dimensionality. And conversely, all methods that overcome the dimensionality problems have an associated—and often implicit or adaptive—metric for measuring neighbor- hoods, which basically does not allow the neighborhood to be simultane- ously small in all directions.

Linear Regression

• The linear model either assumes that the regression function E(Y |X) is linear, or that the linear model is a reasonable approximation.

Poisson Distribution

test: (x^2) $$x^2$$ [x^2]