Linear Regression Models Study Guide
LINEAR REGRESSION MODELS W4315
HOMEWORK 2 ANSWERS February 15, 2010
Instructor: Frank Wood 1. (20 points) In the file ”problem1.txt”(accessible on professor’s website), there are 500 pairs of data, where the first column is X and the second column is Y. The regression model is Y = β0 + β1 X + a. Draw 20 pairs of data randomly from this population of size 500. Use MATLAB to run a regression model specified as above and keep record of the estimations of both β0 and β1 . Do this 200 times. Thus you will have 200 estimates of β0 and β1 . For each parameter, plot a histogram of the estimations. b. The above 500 data are actually generated by the model Y = 3 + 1.5X + , where ∼ N (0, 22 ). What is the exact distribution of the estimates of β0 and β1 ? c. Superimpose the curve of the estimates’ density functions from part b. onto the two histograms respectively. Is the histogram a close approximation of the curve? Answer: First, read the data into Matlab. pr1=textread(’problem1.txt’); V1=pr1(1:250,1); V2=pr1(1:250,2); T1=pr1(251:500,1); T2=pr1(251:500,2); X=[V1;V2]; Y=[T1;T2]; Randomly draw 20 pairs of (X,Y) from the original data set, calculate the coefficients b0 and b1 and repeat the process for 200 times b0=zeros(200,1); b1=zeros(200,1); i=0 for i=1:200 indx=randsample(500,20); x=X(indx); 1
y=Y(indx); avg x = mean(x); avg y = mean(y); sxx = sum((x − avg x).2 ); sxy = sum((x − avg x). ∗ (y − avg y)); b1(i) = sxy/sxx; b0(i) = avg y − b1(i) ∗ avg x; end; Draw histograms of the coefficients b0 and b1 hist(b0) hist(b1)
Figure 1: Histogram of b0
Figure 2: Histogram of b1
2
i b. As we have known, b1 = i i(Xi −X)2 = i (Xii −X)2i = i Ki Yi whereKi = Xi −X¯ 2 ¯ ¯ i i i (Xi −X) So, b1 is a linear combination of Yi . Since Yi has a normal distribution, b1 also follows a normal distribution. E(b1 ) = i Ki E(Yi ) = i Ki (β0 + β1 Xi ) = i Ki β0 + ( i Ki Xi )β1 ¯ i (Xi −X) =0 ¯ i Ki = (Xi −X)2 i i i i i i i =1 ¯ 2 = ¯ 2 i Ki X i = i (Xi −X) i (Xi −X) E(b1 ) = 0 + 1 ∗ β1
HOMEWORK 2 ANSWERS February 15, 2010
Instructor: Frank Wood 1. (20 points) In the file ”problem1.txt”(accessible on professor’s website), there are 500 pairs of data, where the first column is X and the second column is Y. The regression model is Y = β0 + β1 X + a. Draw 20 pairs of data randomly from this population of size 500. Use MATLAB to run a regression model specified as above and keep record of the estimations of both β0 and β1 . Do this 200 times. Thus you will have 200 estimates of β0 and β1 . For each parameter, plot a histogram of the estimations. b. The above 500 data are actually generated by the model Y = 3 + 1.5X + , where ∼ N (0, 22 ). What is the exact distribution of the estimates of β0 and β1 ? c. Superimpose the curve of the estimates’ density functions from part b. onto the two histograms respectively. Is the histogram a close approximation of the curve? Answer: First, read the data into Matlab. pr1=textread(’problem1.txt’); V1=pr1(1:250,1); V2=pr1(1:250,2); T1=pr1(251:500,1); T2=pr1(251:500,2); X=[V1;V2]; Y=[T1;T2]; Randomly draw 20 pairs of (X,Y) from the original data set, calculate the coefficients b0 and b1 and repeat the process for 200 times b0=zeros(200,1); b1=zeros(200,1); i=0 for i=1:200 indx=randsample(500,20); x=X(indx); 1
y=Y(indx); avg x = mean(x); avg y = mean(y); sxx = sum((x − avg x).2 ); sxy = sum((x − avg x). ∗ (y − avg y)); b1(i) = sxy/sxx; b0(i) = avg y − b1(i) ∗ avg x; end; Draw histograms of the coefficients b0 and b1 hist(b0) hist(b1)
Figure 1: Histogram of b0
Figure 2: Histogram of b1
2
i b. As we have known, b1 = i i(Xi −X)2 = i (Xii −X)2i = i Ki Yi whereKi = Xi −X¯ 2 ¯ ¯ i i i (Xi −X) So, b1 is a linear combination of Yi . Since Yi has a normal distribution, b1 also follows a normal distribution. E(b1 ) = i Ki E(Yi ) = i Ki (β0 + β1 Xi ) = i Ki β0 + ( i Ki Xi )β1 ¯ i (Xi −X) =0 ¯ i Ki = (Xi −X)2 i i i i i i i =1 ¯ 2 = ¯ 2 i Ki X i = i (Xi −X) i (Xi −X) E(b1 ) = 0 + 1 ∗ β1