Machine Learning

Linear Regression

Objective

Fit the line that has the least sum of squares known as the Least Squares Regression Line.

The distance from a line to the data point is called a residual.

Types of Linear Regression

Simple Linear Regression

$$y={\beta }_0+{\beta }_{1}X$$

Multiple Linear Regression

$$y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+...+{\beta }_n{X}_n$$

Prediction Function

$$\hat{Y}={\theta }_1+{\theta }_{2}X$$

Loss Function

The error or difference between the predicted value $\hat{Y}$ and the true value $Y$.

In Linear Regression, the Mean Squared Error (MSE) is used which calculates the average of the squared errors between the predicted values $\widehat{y_i}$ and the actual values $y_i$.

$$Cost(J)=\frac{1}{n}{\Sigma }_n^i(\widehat{y_i}-y_i{)}^2$$

Gradient descent is utilized to find the optimal parameters i.e. coefficients that makes the line the most fit to the data.