Linear Regression (Recap)
Linear regression (or Linear Models for Regression), is probably the simplest model in machine learning. Yet it can be still be powerful enough for some industrial level applications. And usually, it provides a great intuition towards models whose nonlinearity is not very strong. 2D line fitting Line fitting is its simplest form. Given a training data set1 $\left\{x^{(i)}, y^{(i)}\right\}_{i=1}^N$, where $x^{(i)}$ and $y^{(i)}$ are all real number, curve fitting requires us to build a model $$ y \approx w_0 + w_1 x = \begin{bmatrix} 1 & x \end{bmatrix} \begin{bmatrix} w_0\\ w_1 \end{bmatrix} \equiv \boldsymbol{x}^\top \boldsymbol{w}, $$ where we need to find the unknown parameters $\boldsymbol{w} = \begin{bmatrix} w_0 & w_1 \end{bmatrix}^\top$....