Statistics
Lecture 12
Simple Linear Regression
•David Bartl
•Statistics
•INM/BASTA

Outline of the lecture
•Introduction
•Least Squares Method
•An application: the trend of a time series
•Summary & Background
•The Classical Assumptions
•The Coefficient of Determination (R2)
•Further Theorems, Tests of Hypotheses and Confidence Intervals
•Two-sample t-test for the difference of the population means // σX=σY
•Simple linear regression without the intercept term

Simple Linear Regression:  Motivation



Simple Linear Regression:  Motivation



Simple Linear Regression:  Motivation



Simple Linear Regression:  Example



Simple Linear Regression:  Example
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10

Simple Linear Regression:  Least Squares Method



Simple Linear Regression:  Least Squares Method



Simple Linear Regression:  Least Squares Method
the normal equation


Simple Linear Regression:  Least Squares Method



An application:
the trend
of a time series
•
•

The trend of a time series



The trend of a time series:  Example



The trend of a time series:  Example
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10

Summary & Background
•
•

Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background
Regressand
Predicand
Explained variable
Dependent variable
Endogenous variable
Controlled variable
Response
Outcome
Predicted variable
Measured variable
Regressor
Predictor
Explanatory variable
Independent variable
Exogenous variable
Control variable
Stimulus
Covariate
The intercept term
Deviation
Error term
Disturbance
Noise
Parameters
Regression coefficients

Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



Simple Linear Regression:  Summary & Background



The Classical Assumptions
•
•

Simple Linear Regression:  Assumptions
linearity
homoskedasticity

Simple Linear Regression:  Assumptions



Further Theorems,
Tests
of Hypotheses
and
Confidence Intervals
•
•

Simple Linear Regression:  The Normal Equation
the normal equation


Simple Linear Regression:  The Normal Equation
the normal equation


Simple Linear Regression:  The Normal Equation



Simple Linear Regression:  The Normal Equation



Simple Linear Regression:  The Normal Equation



Simple Linear Regression



Simple Linear Regression:  Theorem 1



Simple Linear Regression



Simple Linear Regression:  Theorem 2



Simple Linear Regression



Simple Linear Regression



Simple Linear Regression:  Theorem 3



Simple Linear Regression:  Theorem 4



Simple Linear Regression



Simple Linear Regression:  Theorem 5



Simple Linear Regression:  Corollaries



Tests of hypotheses about the parameters  β0  and  β



t-test for the parameter  β0  or  β



t-test for the parameter  β0  or  β



t-test for the parameter  β0



t-test for the parameter  β0



t-test for the parameter  β0



t-test for the parameter  β0



t-test for the parameter  β0



t-test for the parameter  β0



Confidence interval for the parameter  β0



Confidence interval for the parameter  β0



Confidence interval for the parameter  β0



Confidence interval for the parameter  β0



t-test for the parameter  β



t-test for the parameter  β



t-test for the parameter  β



t-test for the parameter  β



t-test for the parameter  β



t-test for the parameter  β



Confidence interval for the parameter  β



Confidence interval for the parameter  β



Confidence interval for the parameter  β



Confidence interval for the parameter  β



Simple Linear Regression:  Theorem 6



Simple Linear Regression:  Theorem 7



Test of hypothesis about the variance  σ2



χ2-test for the variance  σ2



χ2-test for the variance  σ2



χ2-test for the variance  σ2



χ2-test for the variance  σ2



χ2-test for the variance  σ2



χ2-test for the variance  σ2



Confidence interval for the variance  σ2



Confidence interval for the variance  σ2



Confidence interval for the variance  σ2



Confidence interval for the variance  σ2



Simple Linear Regression:  Theorem 8



Hypotheses about all the parameters



The Coefficient of Determination  (R2)
•
•

The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2)



The Coefficient of Determination  (R2):  Theorem 9



F-test for the null hypothesis   H0:  β=0



Two-sample t-test for the difference of the population means // σX=σY
•
•

Two-sample t-test for the diff. of the pop. means // σX=σY



Two-sample t-test for the diff. of the pop. means // σX=σY
and


Two-sample t-test for the diff. of the pop. means // σX=σY



Simple linear regression without the intercept term
•
•

Simple linear regression without the intercept term



Simple linear regression without the intercept term



Simple linear regression without the intercept term



Simple linear regression without the intercept term
the normal equation