READING 20: LINEAR REGRESSION WITH ONE REGRESSOR

Regression

        Regression analysis attempts to measure the relationship between a dependent variable and one or more independent variables.
         A scatter plot (a.k.a. scattergram) is a collection of points on a graph where each point represents the values of two variables (i.e., an X/Y pair).

Population Regression

        A population regression line indicates the expected value of a dependent variable conditional on one or more independent variables: E(Y_i | X_i) = B₀ + B₁ × (X_i).
        The difference between an actual dependent variable and a given expected value is the error term or noise component denoted ε_i = Y_i − E(Y_i | X_i).

Sample Regression

        The sample regression function is an equation that represents a relationship between the Y and X variable(s) using only a sample of the total data. It uses symbols that are similar but still distinct from that of the population Y_i = b₀ + b₁ × X_i + e_i.

Linear Regression

        In a linear regression model, we generally assume that the equation is linear in the parameters, and that it may or may not be linear in the variables.

Ordinary Least Squares (OLS) Regression

        Ordinary least squares estimation is a process that estimates the population parameters Bi with corresponding values for bi that minimize Σei ² = Σ[Y_i − (b₀ + b₁ × X_i)]². The formulas for the coefficients are:

Simple Linear Regression

        Three key assumptions made with simple linear regression include:

• The expected value of the error term, conditional on the independent variable, is zero.
  
• All (X, Y) observations are independent and identically distributed (i.i.d.).
  
• It is unlikely that large outliers will be observed in the data.

Benefits of Using OLS Estimators

        OLS estimators are used widely in practice. In addition to practical benefits, OLS estimators exhibit desirable properties of an estimator.

Properties of OLS Estimators

        Since OLS estimators are random variables, they have their own sampling distributions. These sampling distributions are used to estimate population parameters. Given that the expected value of the estimator is equal to the parameter being estimated and the accuracy of the parameter estimate increases as the sample size increases, we can say that OLS estimators are both unbiased and consistent.

Measures Of Fit In Simple Linear Regression

        Explained sum of squares (ESS) measures the variation in the dependent variable that is explained by the independent variable.
         Total sum of squares (TSS) measures the total variation in the dependent variable. TSS is equal to the sum of the squared differences between the actual Y-values and the mean of Y.
         Sum of squared residuals (SSR) measures the unexplained variation in the dependent variable.
         The standard error of the regression (SER) measures the degree of variability of the actual Y-values relative to the estimated Y-values from a regression equation.
         The coefficient of determination, represented by R², is a measure of the “goodness of fit” of the regression.

The Results Of OLS Regression

        Assuming certain conditions exist, an analyst can use the results of an ordinary least squares regression in place of an unknown population regression function to describe the relationship between the dependent and independent variable.