202211201933
Status:
Tags: Statistics Mathematics
Types
Linear Regression
Assumptions
- Linearity
- Homoscedasticity
- Variance of residual is the same for any value of X
- AKA heterogeneity of variances
- Independence
- Observations are independent of each other
- Normality
Stepwise Regression
- Method of fitting regression models where the choice of predictor variables is carried out by an automatic procedure
Forward selection
- Start with no variables, test addition of each variable
- Add variable whose inclusion gives most statistically significant improvement of fit
- Repeat until none improve model to statistically significant extent
- Drawbacks:
- Variables whose statistical significance relies on a variable being held constant are not included
Backward elimination
- Start with all candidate variables
- Test deletion of each variable
- Delete if loss gives most statistically insignificant deterioration of model fit
- Repeat until no variables can be deleted without statistically insignificant loss of fit
Bidirectional elimination
- Combination of forward & backward
Criticisms
- Tests are biased, since they are based on the same data
- May oversimplify real models of the data
- Something to do with degrees of freedom