T2W2 Lab Linear Regression - Questions

1 . Prove that the Least Squared coefficient estimates (LSE) for and are: Solution 2 . Prove that the estimates in Q1 are unbiased. Solution 3 . Prove that the MLE estimates of and are equal to the ones given by LSE (from Q1 ). Solution 4 . Prove SST=SSE+SSM Solution 5 . Express SSM in terms of a) and b) Solution 6 . Prove the following variance formulas: Solution 7 . Prove , where . Solution 8 . Prove: Solution 9 . Forensic scientists use various methods for determining the likely time of death from post-mortem examination of human bodies. A recently suggested objective method uses the concentration of a compound (3-methoxytyramine or 3-MT) in a particular part of the brain. In a study of the relationship between post-mortem interval and the concentration of 3-MT, samples of the approximate part of the brain were taken from coroners cases for which the time of death had been determined form eye-witness accounts. The intervals ( ; in hours) and concentrations ( ; in parts per million) for 18 individuals who were found to have died from organic heart disease are given in the following table. For the last two individuals (numbered 17 and 18 in the table) there was no eye- witness testimony directly available, and the time of death was established on the basis of other evidence including knowledge if the individuals' activities. Observation number Interval ( ) Concentration ( ) 1 5.5 3.26 2 6.0 2.67 3 6.5 2.82 4 7.0 2.80 5 8.0 3.29 6 12.0 2.28 7 12.0 2.34 8 14.0 2.18 9 15.0 1.97 10 15.5 2.56 11 17.5 2.09 12 17.5 2.69 13 20.0 2.56 14 21.0 3.17 15 25.5 2.18 16 26.0 1.94 17 48.0 1.57 18 60.0 0.61 , , , , In this investigation you are required to explore the relationship between concentration (regarded the responds/dependent variable) and interval (regard as the explanatory/independent variable). a. Construct a scatterplot of the data. Comment on any interesting features of the data and discuss briefly whether linear regression is appropriate to model the relationship between concentration of 3-MT and the interval from death. b. Calculate the correlation coefficient for the data, and use it to test the null hypothesis that the population correlation coefficient is equal to zero. c. Calculate the equation of the least-squares fitted regression line and use it to estimate the concentration of 3-MT: i. after 1 day and ii. after 2 days. Comment briefly on the reliability of these estimates. d. Calculate a 99% confidence interval for the slope of the regression line. Using this confidence interval, test the hypothesis that the slope of the regression line is equal to zero. Comment on your answer in relation to the answer given in part (2) above. Solution 10 . A university wishes to analyse the performance of its students on a particular degree course. It records the scores obtained by a sample of 12 students at the entry to the course, and the scores obtained in their final examinations by the same students. The results are as follows: Student A B C D E F G H I J K L Entrance exam score (%) 86 53 71 60 62 79 66 84 90 55 58 72 Final paper score (%) 75 60 74 68 70 75 78 90 85 60 62 70 , , , , a. Calculate the fitted linear regression equation of on . b. Assuming the full normal model, calculate an estimate of the error variance and obtain a 90% confidence interval for . c. By considering the slope parameter, formally test whether the data is positively correlated. d. Find a 95% confidence interval for the mean finals paper score corresponding to an individual entrance score of 53. e. Test whether this data come form a population with a correlation coefficient equal to 0.75. f. Calculate the proportion of variance explained by the model. Hence, comment on the fit of the model. Solution 11 . Complete the following ANOVA table for a simple linear regression with observations: Source D.F. Sum of Squares Mean Squares F-Ratio Regression ____ ____ ____ Error ____ 8.2 Total Solution 12 . Suppose you are interested in relating the accounting variable EPS (earnings per share) to the market variable STKPRICE (stock price). Then, a regression equation was fitted using STKPRICE as the response variable with EPS as the regressor variable. Following is the computer output from your fitted regression. You are also given that: , , , and (Note that: and ) a. Calculate the correlation coefficient of EPS and STKPRICE. b. Estimate the STKPRICE given an EPS of $ 2. Provide a 95% confidence interval of your estimate. c. Provide a 95% confidence interval for the slope coefficient . d. Compute and . e. Describe how you would check if the errors have constant variance. f. Perform a test of the significance of EPS in predicting STKPRICE at a level of significance of 5%. g. Test the hypothesis against at a level of significance of 5%. Solution 13 . (Modified from an Institute of Actuaries exam problem) An insurance company issues house buildings policies for houses of similar size in four different post-code regions , , , and . An insurance agent takes independent random samples of house buildings policies for houses of similar size in each of the four regions. The annual premiums (in dollars) were as follows: Region 229 241 270 256 241 247 261 243 272 219 Region 261 269 284 268 249 255 237 270 269 257 Region 253 247 244 245 221 229 245 256 232 269 Region 279 268 290 245 281 262 287 257 262 246 Perform a one-way analysis of variance at the level to compare the premiums for all four regions. State briefly the assumptions required to perform this analysis of variance. Solution 14 . (Past Institute Exam) As part of an investigation into health service funding a working party was concerned with the issue of whether mortality could be used to predict sickness rates. Data on standardised mortality rates and standardised sickness rates collected for a sample of 10 regions and are shown in the table below: Region Mortality rate (per 100,000) Sickness rate (per 100,000) 1 125.2 206.8 2 119.3 213.8 3 125.3 197.2 4 111.7 200.6 5 117.3 189.1 6 100.7 183.6 7 108.8 181.2 8 102.0 168.2 9 104.7 165.2 10 121.1 228.5 Data summaries: , , , , and . a. Calculate the correlation coefficient between the mortality rates and the sickness rates and determine the probability-value for testing whether the underlaying correlation coefficient is zero against the alternative that it is positive. b. Noting the issue under investigation, draw an appropriate scatterplot for these data and comment on the relationship between the two rates. c. Determine the fitted linear regression of sickness rate on mortality rate and test whether the underlaying slope coefficient can be considered to be as large as 2.0. d. For a region with mortality rate 115.0, estimate the expected sickness rate and calculate 95% confidence limits for this expected rate. Solution 15 . (Past institute Exam) Consider the following data, which comprise of four groups sizes ( ), each comprising four observations. In scenario I, information is also given on the sum assured under the policy concerned - the sum assured is the same for all four policies in a group. In scenario II, we regard the policies in the different groups as having been issued by four different companies - the policies in a group are all issued the same company. All monetary amounts are in units of £ 10,000. Summaries of the claim sizes in each group are given in a second table. Group 1 2 3 4 Claim sizes 0.11 0.46 0.52 1.43 1.48 2.05 1.52 2.36 0.71 1.45 1.84 2.47 2.38 3.31 2.95 4.08 I: sum assured 1 2 3 4 II: Company A B C D Summaries of claim sizes: Group 1 2 3 4 2.73 6.26 9.22 10.91 2.8303 11.8018 23.0134 33.2289 a. In scenario I, suppose we adopt the linear regression model where is the claim size and is the corresponding sum assured, . i. Calculate the total sum of squares and its partition into the regression (model) sum of squares and the residual (error) sum of squares. ii. Fit the model and calculate the fitted values for the first claim size of group 1 (namely 0.11) and the last claim size of group 4 (namely 4.08). iii. Consider a test of the hypothesis against a two-sided alterative. By preforming appropriate calculations, assess the strength of the evidence against this "no linear relationship" hypothesis. b. In scenario II, suppose we adopt the analysis of variance model where is the claim size for company and is the company effect, and . i. Calculate the partition of the total sum of squared into the "between companies" (model) sum of squares and the "within companies" (residual/error) sum of squares. ii. Fit the model. iii. Calculate the fitted values for the first claim size of group 1 and the last claim size of group 4. iv. Consider a test of hypothesis , against a general alternative. By preforming appropriate calculations, assess the strength of the evidence against this "no company effects" hypothesis. Solution 1 . Describe the null hypotheses to which the p-values given in Table 3.4 correspond. Explain what conclusions you can draw based on these -values. Your explanation should be phrased in terms of sales , TV , radio , and newspaper , rather than in terms of the coefficients of the linear model. Solution 2 . Suppose we have a data set with five predictors, GPA, IQ, Level (1 for College and 0 for High School), Interaction between GPA and IQ, and Interaction between GPA and Level. The response is starting salary after graduation (in thousands of dollars). Suppose we use least squares to fit the model, and get , , , , , . a. Which answer is correct, and why ? i. For a fixed value of IQ and GPA, high school graduates earn more, on average, than college graduates. ii. For a fixed value of IQ and GPA, college graduates earn more, on average, than high school graduates. iii. For a fixed value of IQ and GPA, high school graduates earn more, on average, than college graduates provided that the GPA is high enough. iv. For a fixed value of IQ and GPA, college graduates earn more, on average, than high school graduates provided that the GPA is high enough. b. Predict the salary of a college graduate with IQ of 110 and a GPA of 4.0. c. True or false: Since the coefficient for the GPA/IQ interaction term is very small, there is very little evidence of an interaction effect. Justify your answer. Solution 3 . I collect a set of data ( observations) containing a single predictor and a quantitative response. I then fit a linear regression model to the data, as well as a separate cubic regression, i.e. . a. Suppose that the true relationship between X and Y is linear, i.e. . Consider the training residual sum of squares (RSS) for the linear regression, and also the training RSS for the cubic regression. Would we expect one to be lower than the other, would we expect them to be the same, or is there not enough information to tell ? Justify your answer. b. Answer (a) using test rather than training RSS. c. Suppose that the true relationship between X and Y is not linear, but we don't know how far it is from linear. Consider the training RSS for the linear regression, and also the training RSS for the cubic regression. Would we expect one to be lower than the other, would we expect them to be the same, or is there not enough information to tell ? Justify your answer. d. Answer (c) using test rather than training RSS. Solution a. 4 . Write down the design matrix for the simple linear regression model. b. Write out the matrix for the simple linear regression model. c. Write out the matrix for the simple linear regression model. d. Write out the matrix for the simple linear regression model. e. Calculate using your results above. Where is the vector of the response variable and is the vector of coefficients Solution 5 . The following model was fitted to a sample of supermarkets in order to explain their profit levels: where profits, in thousands of dollars food sales, in tens of thousands of dollars nonfood sales, in tens of thousands of dollars, and store size, in thousands of square feet. The estimated regression coefficients are given below: Which of the following is TRUE ? a. A dollar increase in food sales increases profits by 2.7 cents. b. A 2.7 cent increase in food sales increases profits by a dollar. c. A 9.7 cent increase in nonfood sales decreases profits by a dollar. d. A dollar decrease in nonfood sales increases profits by 9.7 cents. e. An increase in store size by one square foot increases profits by 52.5 cents. Solution 6 . In a regression model of three explanatory variables, twenty-five observations were used to calculate the least squares estimates. The total sum of squares and regression sum of squares were found to be and , respectively. Calculate the adjusted coefficient of determination (i.e adjusted ). a. 89.0% b. 89.4% c. 89.9% d. 90.3% e. 90.5% Solution 7 . In a multiple regression model given by: which of the following gives a correct expression for the coefficient of determination (i.e ) ? I. II. III. Options: a. I only b. II only c. III only d. I and II only e. I and III only Solution 8 . The ANOVA table output from a multiple regression model is given below: Source D.F. SS MS F-Ratio Prob( ) Regression 5 13326.1 2665.2 13.13 0.000 Error 42 8525.3 203.0 Total 47 21851.4 Compute the adjusted coefficient of determination (i.e adjusted ). a. 52% b. 56% c. 61% d. 63% e. 68% Solution 9 . You have information on 62 purchases of Ford automobiles. In particular, you have the amount paid for the car in hundreds of dollars, the annual income of the individuals in hundreds of dollars, the sex of the purchaser ( , 1=male and 0=female) and whether or not the purchaser graduated from college ( , 1=yes, 0=no). After examining the data and other information available, you decide to use the regression model: You are given that: and the mean square error for the model is Calculate . a. 0.17 b. 17.78 c. 50.04 d. 55.54 e. 57.43 Solution 10 . Suppose in addition to the information in question 21., you are given: Calculate the expected difference in the amount spent to purchase a car between a person who graduated from college and another one who did not. a. 233.5 b. 1604.3 c. 2195.3 d. 4920.6 e. 6472.1 Solution 11 . A regression model of on four independent variables and has been fitted to a data consisting of observations and the computer output from estimating this model is given below: Regression Analysis The regression equation is y = 3894 - 50.3 x1 + 0.0826 x2 + 0.893 x3 + 0.137 x4 Predictor Coef SE Coef T Constant 3893.8 409.0 9.52 x1 -50.32 9.062 -5.55 x2 0.08258 0.02133 3.87 x3 0.89269 0.04744 18.82 x4 0.13677 0.05303 2.58 Which of the following statement is NOT true ? a. All the explanatory variables have insignificant influence on . b. The variable is a significant variable. c. The variable is a significant variable. d. The variable is a significant variable. e. The variable is a significant variable. Where 's are vectors of explanatory variables and is the vector of response variable Solution 12 . The estimated regression model of fitting life expectancy from birth (LIFE_EXP) on the country's gross national product (in thousands) per population (GNP) and the percentage of population living in urban areas (URBAN%) is given by: For a particular country, its URBAN% is 60 and its GNP is 3.0. Calculate the estimated life expectancy at birth for this country. a. 49 b. 50 c. 57 d. 60 e. 65 Solution 13 . What is the use of the scatter plot of the fitted values and the residuals ? a. to examine the normal distribution assumption of the errors b. to examine the goodness of fit of the regression model c. to examine the constant variation assumption of the errors d. to test whether the errors have zero mean e. to examine the independence of the errors Solution 1 . Consider a -nearest neighbours model where , , and the estimated model is . The weight function is . Show that Where are 's -nearest neighbours. Note that: Regression Analysis The regression equation is STKPRICE = 25.044 + 7.445 EPS Predictor Coef SE Coef T p Constant 25.044 3.326 7.53 0.000 EPS 7.445 1.144 6.51 0.000 Analysis of Variance SOURCE DF SS MS F p Regression 1 10475 10475 42.35 0.000 Error 46 11377 247 Total 47 21851 ACTL3142 and ACTL5110 Lab 2: Linear Regression AUTHOR Questions Conceptual Questions ! Simple linear regression questions ˆ β 0 ˆ β 1 ˆ β 0 = - y - ˆ β 1 x - ˆ β 1 = ∑ n i =1 ( x i - x ) ⋅ ( y i - - y ) ∑ n i =1 ( x i - x ) 2 = S xy S xx - - ˆ β 0 ˆ β 1 β 1 β 2 1 V ( ˆ β 0 | X = x ) = σ 2 ( 1 n + x 2 S xx ) V ( ˆ β 1 | X = x ) = σ 2 S xx Cov ( ˆ β 0 , ˆ β 1 | X = x ) = - x σ 2 S xx - - - - - V (ˆ y 0 ) = ( 1 n + ( x - x 0 ) 2 S xx ) σ 2 - ˆ y 0 = E ( y | X = x 0 ) = ˆ β 0 + ˆ β 1 x 0 E [ Y i - ˆ y i | X = x , X = x i ] = 0 -- V ( Y i - ˆ y i | X = x , X = x i ) = σ 2 ( 1 + 1 n + ( x - x i ) 2 S xx ) - - - x y x y ∑ x = 337 ∑ x 2 = 9854.5 ∑ y = 42.98 ∑ y 2 = 109.7936 ∑ xy = 672.8 x y ∑ x = 836 ∑ y = 867 ∑ x 2 = 60, 016 ∑ y 2 = 63, 603 ∑ ( x - x )( y - - y ) = 1, 122 - y x σ 2 σ 2 60 ____ ____ ____ 639.5 x = 2.338 - - y = 40.21 s x = 2.004 s y = 21.56. s 2 x = S xx n - 1 s 2 y = S yy n - 1 β s R 2 H 0 : β = 24 H a : β > 24 A B C D 10 A : ( ∑ x = 2, 479, ∑ x 2 = 617, 163 ) B : ( ∑ x = 2, 619, ∑ x 2 = 687, 467 ) C : ( ∑ x = 2, 441, ∑ x 2 = 597, 607 ) D : ( ∑ x = 2, 677, ∑ x 2 = 718, 973 ) 5% m s ∑ m = 1136.1 ∑ m 2 = 129, 853.03 ∑ s = 1934.2 ∑ s 2 = 377, 700.62 ∑ ms = 221, 022.58 y y x ∑ y ∑ y 2 Y i = α + β x i + i Y i i th x i i = 1, ... , 16 H 0 : β = 0 Y ij = μ + τ i + e ij Y ij j th i τ i i th i = 1, 2, 3, 4 j = A , B , C , D H 0 : τ i = 0 i = A , B , C , D Multiple linear regression questions p X 1 = X 2 = X 3 = X 4 = X 5 = β 0 = 50 β 1 = 20 β 2 = 0.07 β 3 = 35 β 4 = 0.01 β 5 = - 10 n = 100 Y = β 0 + β 1 X + β 2 X 2 + β 3 X 3 + Y = β 0 + β 1 X + X X X Y - ( X X ) - 1 ˆ β = ( X X ) - 1 X Y - - Y - ˆ β - y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3 + ε y = x 1 = x 2 = x 3 = ˆ β 1 = 0.027 and ˆ β 2 = - 0.097 and ˆ β 3 = 0.525. 666.98 610.48 R 2 y = β 0 + β 1 x 1 + ... + β p - 1 x p - 1 + ε , R 2 SSM SST SST - SSE SST SSM SSE > F R 2 y x 1 x 2 x 3 y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3 + ε . ( X X ) - 1 = 0.109564 - 0.000115 - 0.035300 - 0.026804 - 0.000115 0.000001 - 0.000115 - 0.000091 - 0.035300 - 0.000115 0.102446 0.023971 - 0.026804 - 0.000091 0.023971 0.083184 s 2 = 30106. SE( ˆ β 2 ) X Y = . - 9 558 4 880 937 7 396 6 552 y x 1 , x 2 , x 3 x 4 212 y - x 1 - x 2 - x 3 - x 4 - x i - y - LIFE_EXP = 48.24 + 0.79 GNP + 0.154 URBAN%. KNN question k Y = f ( X ) + E ( ) = 0, V ( ) = σ 2 ^ f ( x ) 1 k EPE k ( x 0 ) = σ 2 + f ( x 0 ) - 1 k ∑ l ∈ N ( x 0 ) f ( x ( l ) ) 2 + σ 2 k N ( x 0 ) x 0 k EPE k ( x 0 ) = E [( Y - ^ f ( x 0 )) 2 | X = x 0 ] Questions Conceptual Questions Simple linear regression questions Multiple linear regression questions KNN question Applied Questions Solutions Conceptual Questions Simple linear regression questions Multiple linear regression questions KNN question Applied Questions ! PDF Table of contents Other Formats

Solution 1 . (ISLR2, Q3.8) This question involves the use of simple linear regression on the Auto data set. a. Use the lm() function to perform a simple linear regression with mpg as the response and horsepower as the predictor. Use the summary() function to print the results. Comment on the output. For example: i. Is there a relationship between the predictor and the response ? ii. How strong is the relationship between the predictor and the response ? iii. Is the relationship between the predictor and the response positive or negative ? iv. What is the predicted mpg associated with a horsepower of 98 ? What are the associated 95% confidence and prediction intervals ? b. Plot the response and the predictor. Use the abline() function to display the least squares regression line. c. Use the plot() function to produce diagnostic plots of the least squares regression fit. Comment on any problems you see with the fit. Solution 2 . (ISLR2, Q3.10) This question should be answered using the Carseats data set. a. Fit a multiple regression model to predict Sales using Price , Urban , and US . b. Provide an interpretation of each coefficient in the model. Be careful—some of the variables in the model are qualitative ! c. Write out the model in equation form, being careful to handle the qualitative variables properly. d. For which of the predictors can you reject the null hypothesis ? e. On the basis of your response to the previous question, fit a smaller model that only uses the predictors for which there is evidence of association with the outcome. f. How well do the models in (a) and (e) fit the data ? g. Using the model from (e), obtain 95% confidence intervals for the coefficient(s). h. Is there evidence of outliers or high leverage observations in the model from (e) ? Solution 3 . (ISLR2, Q3.11) In this problem we will investigate the -statistic for the null hypothesis in simple linear regression without an intercept. To begin, we generate a predictor and a response as follows. a. Perform a simple linear regression of onto , without an intercept. Report the coefficient estimate , the standard error of this coefficient estimate, and the -statistic and -value associated with the null hypothesis . Comment on these results. (You can perform regression without an intercept using the command lm(y ~ x+0) .) b. Now perform a simple linear regression of onto without an intercept, and report the coefficient estimate, its standard error, and the corresponding -statistic and -values associated with the null hypothesis . Comment on these results. c. What is the relationship between the results obtained in (a) and (b) ? d. For the regression of onto without an intercept, the -statistic for takes the form , where is given by (3.38), and where (These formulas are slightly different from those given in Sections 3.1.1 and 3.1.2, since here we are performing regression without an intercept.) Show algebraically, and confirm numerically in R, that the t-statistic can be written as e. Using the results from (d), argue that the -statistic for the regression of onto is the same as the -statistic for the regression of onto . f. In R, show that when regression is performed with an intercept, the -statistic for is the same for the regression of onto as it is for the regression of onto . Solution 4 . (ISLR2, Q3.13) In this exercise you will create some simulated data and will fit simple linear regression models to it. Make sure to use set.seed(1) prior to starting part (a) to ensure consistent results. a. Using the rnorm() function, create a vector, x , containing 100 observations drawn from a distribution. This represents a feature, . b. Using the rnorm() function, create a vector, eps , containing 100 observations drawn from a distribution—a normal distribution with mean zero and variance 0.25. c. Using x and eps, generate a vector y according to the model What is the length of the vector y ? What are the values of and in this linear model ? d. Create a scatterplot displaying the relationship between x and y. Comment on what you observe. e. Fit a least squares linear model to predict y using x . Comment on the model obtained. How do and compare to and ? f. Display the least squares line on the scatterplot obtained in (d). Draw the population regression line on the plot, in a different color. Use the legend() command to create an appropriate legend. g. Now fit a polynomial regression model that predicts y using x and . Is there evidence that the quadratic term improves the model fit ? Explain your answer. h. Repeat (a)-(f) after modifying the data generation process in such a way that there is less noise in the data. The model ( [eq:3.39] ) should remain the same. You can do this by decreasing the variance of the normal distribution used to generate the error term in (b). Describe your results. i. Repeat (a)-(f) after modifying the data generation process in such a way that there is more noise in the data. The model ( [eq:3.39] ) should remain the same. You can do this by increasing the variance of the normal distribution used to generate the error term in (b). Describe your results. j. What are the confidence intervals for and based on the original data set, the noisier data set, and the less noisy data set ? Comment on your results. Solution 5 . (ISLR2, Q3.14) This problem focuses on the collinearity problem. a. Perform the following commands in R : The last line corresponds to creating a linear model in which y is a function of x1 and x2 . Write out the form of the linear model. What are the regression coefficients ? b. What is the correlation between x1 and x2 ? Create a scatterplot displaying the relationship between the variables. c. Using this data, fit a least squares regression to predict y using x1 and x x2 . Describe the results obtained. What are , , and ? How do these relate to the true , , and ? Can you reject the null hypothesis ? How about the null hypothesis ? d. Now fit a least squares regression to predict y using only x1 . Comment on your results. Can you reject the null hypothesis ? e. Now fit a least squares regression to predict y using only x1 . Comment on your results. Can you reject the null hypothesis ? f. Do the results obtained in (c)-(e) contradict each other ? Explain your answer. g. Now suppose we obtain one additional observation, which was unfortunately mismeasured. Re-fit the linear models from (c) to (e) using this new data. What effect does this new observation have on the each of the models ? In each model, is this observation an outlier ? A high-leverage point ? Both ? Explain your answers. Solution 1 . We determine and by minimizing the error. Hence, we use least squares estimates (LSE) for and : The minimum is obtained by setting the first order condition (FOC) to zero: The LSE and are given by setting the FOC equal to zero: So we have Next step: Rearranging so that and become functions of , , , and * . And 's in line (6) was subbed into in line (7). At this point, is done. So we'll continue with . From the previous steps we have: thus: * . Using the notations, we have an easier way to write : * 2 . For , using the equation in line (10) in Q1: For , using equation in line (13) from Q1: 3 . In the regression model there are three parameters to estimate: , , and . Joint density of -under the (strong) normality assumptions- is the product of their marginals (independent by assumption) so that the likelihood is: Taking partial derivatives of the log-likelihood with respect to : Equate the above to 0 and solve for should give Similarly, taking partial derivatives of the log-likelihood with respect to : The last line was derived using the fact that Equate the above equation to 0 and solve for , we'll get: The rest is the same as the derivation from Q1 * using , continue: ** uses (which is self-explanatory) and (We'll prove this at the end of this question), we have the following results *Proof of :* Using the estimates of and , We have: *** uses equation (13) from Q1 5 . The SSM is using . 6 . *For :* Note that we have: *uses: And because , where the are constant and is given, hence *For :* Using that: *For :* we have: 8 . Expectation *uses the fact that the expected value of the random error is 0 Variance ** uses as it contains only constants, and the covariance is 0 because the observed point is not used in making predictions, and should be independent of the predicted point, and the conditional variance of was derived in earlier questions a. 9 . Interesting features are that, in general, the concentration of 3-MT in the brain seems to decrease as the post mortem interval increases. Another interesting feature is that we observe two observations with a much higher post mortem interval than the other observations. The data seems to be appropriate for linear regression. The linear relationship seems to hold,especially for values of interval between 5 and 26 (we have enough observations for that). Care should be taken into account when evaluating for lower than 5 and larger than 26 (only two observations) because we do not know whether the linear relationship between and still holds then. b. We test: The corresponding test statistic is given by: We reject the null hypothesis for large and small values of the test statistic. We have and the correlation coefficient is given by: Thus, the value of our test statistic is given by: From Formulae and Tables page 163 we observe , * using symmetry property of the student- distribution. We observe that the value of our test statistic (-5.89) is smaller than -4.015, thus our -value should be smaller than . Thus, we can reject the null hypothesis even at a significance level of 0.1%, hence we can conclude that there is a linear dependency between interval and concentration. Note that the alternative hypothesis is here a linear dependency and not negative linear dependency, so you do accept the alternative by rejecting the null hypothesis. Although, when you would use as alternative hypothesis negative dependency, you would accept this alternative, due to the construction of the test we have to use the phrase "a linear dependency" and not "a negative linear dependency". c. The linear regression model is given by: The estimate of the slope is given by: The estimate of the intercept is given by: Thus, the estimate of given a value of is given by: i. One day equals 24 hours, i.e., , thus Applied Questions H 0 : β j = 0 t H 0 : β = 0 x y set.seed ( 1 ) x <- rnorm ( 100 ) y <- 2 * x + rnorm ( 100 ) y x ^ β t p H 0 : β = 0 x y t p H 0 : β = 0 Y X t H 0 : β = 0 ^ β /SE( ^ β ) ^ β SE( ^ β ) = ∑ n i ( y i - x i ^ β ) 2 ( n - 1) ∑ n i ′ =1 x 2 i ′ ( √ n - 1) ∑ n i =1 x i y i √ ( ∑ n i =1 x 2 i )( ∑ n i ′ =1 y 2 i ′ ) - ( ∑ n i ′ =1 x i ′ y i ′ ) 2 t y x t x y t H 0 : β 1 = 0 y x x y N (0, 1) X N (0, 0.25) Y = - 1 + 0.5 X + . β 0 β 1 ^ β 0 ^ β 1 β 0 β 1 x 2 β 0 β 1 set.seed ( 1 ) x1 <- runif ( 100 ) x2 <- 0.5 * x1 + rnorm ( 100 ) / 10 y <- 2 + 2 * x1 + 0.3 * x2 + rnorm ( 100 ) ^ β 0 ^ β 1 ^ β 2 β 0 β 1 β 2 H 0 : β 1 = 0 H 0 : β 2 = 0 H 0 : β 1 = 0 H 0 : β 1 = 0 set.seed ( 1 ) x1 <- c (x1, 0.1 ) x2 <- c (x2, 0.8 ) y <- c (y, 6 ) Solutions Conceptual Questions Simple linear regression questions ˆ β 0 ˆ β 1 ˆ β 0 ˆ β 1 min β 0 , β 1 { S ( ˆ β 0 , ˆ β 1 )} = min β 0 , β 1 { n ∑ i =1 ( y i - ( ˆ β 0 + ˆ β 1 x i )) 2 } . ∂ S ( ˆ β 0 , ˆ β 1 ) ∂ ˆ β 0 = - 2 n ∑ i =1 y i - ( ˆ β 0 + ˆ β 1 x i ) ∂ S ( ˆ β 0 , ˆ β 1 ) ∂ ˆ β 1 = - 2 n ∑ i =1 x i ( y i - ( ˆ β 0 + ˆ β 1 x i )) . ˆ β 0 ˆ β 1 n ∑ i =1 y i = n ˆ β 0 + ˆ β 1 n ∑ i =1 x i n ∑ i =1 x i y i = ˆ β 0 n ∑ i =1 x i + ˆ β 1 n ∑ i =1 x 2 i . ˆ β 0 = ∑ n i =1 y i - ˆ β 1 ∑ n i =1 x i n = - y - ˆ β 1 x ˆ β 1 = ∑ n i =1 x i y i - ˆ β 0 ∑ n i =1 x i ∑ n i =1 x 2 i - ˆ β 0 ˆ β 1 ∑ n i =1 y i ∑ n i =1 x i ∑ n i =1 x 2 i ∑ n i =1 x i y i ˆ β 0 = ∑ n i =1 y i - ( ∑ n i =1 x i y i - ˆ β 0 ∑ n i =1 x i ∑ n i =1 x 2 i ) ∑ n i =1 x i n ( 1 - ( ∑ n i =1 x i ) 2 n ∑ n i =1 x 2 i ) ˆ β 0 = ∑ n i =1 x 2 i ∑ n i =1 y i - ( ∑ n i =1 x i y i ) ∑ n i =1 x i n ∑ n i =1 x 2 i ˆ β 0 * = ∑ n i =1 y i ( ∑ n i =1 x 2 i ) - ∑ n i =1 x i y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 . (1 - a / b ) c = d / b → ( bc - ac )/ b = d / b → c = d /( b - a ) ˆ β 0 ˆ β 1 ˆ β 0 ˆ β 1 ˆ β 0 = ∑ n i =1 y i - ˆ β 1 ∑ n i =1 x i n ˆ β 1 = ∑ n i =1 x i y i - ˆ β 0 ∑ n i =1 x i ∑ n i =1 x 2 i . ˆ β 1 = n ∑ n i =1 x i y i - ( ∑ n i =1 y i - ˆ β 1 ∑ n i =1 x i ) ∑ n i =1 x i n ∑ n i =1 x 2 i ( 1 - ( ∑ n i =1 x i ) 2 n ∑ n i =1 x 2 i ) ˆ β 1 = n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i ˆ β 1 * = n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 . (1 - a / b ) c = d / b → ( bc - ac )/ b = d / b → c = d /( b - a ) ˆ β 1 ˆ β 1 = n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = n ( ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i ⋅ n n 2 ) n ( ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 ⋅ n n 2 ) = ∑ n i =1 x i y i - nx - y ∑ n i =1 x 2 i - nx 2 * = ∑ n i =1 x i y i - ∑ n i =1 x i - y - ∑ n i =1 y i x + nx - y ∑ n i =1 x 2 i + ∑ n i =1 x 2 - 2 ∑ n i =1 x i x = ∑ n i =1 ( x i - x ) ⋅ ( y i - - y ) ∑ n i =1 ( x i - x ) 2 = S xy S xx - - -- -- - - ∑ n i =1 x i - y = ∑ n i =1 x i ∑ n i =1 y i n = ∑ n i =1 y i ∑ n i =1 x i n = ∑ n i =1 y i x = n ∑ n i =1 x i n ∑ n i =1 y i n = nx - y -- ˆ β 0 E [ ˆ β 0 | X = x ] = E [ ∑ n i =1 y i ( ∑ n i =1 x 2 i ) - ∑ n i =1 x i y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 ] - = ∑ n i =1 E [ y i ] ( ∑ n i =1 x 2 i ) - ∑ n i =1 x i E [ y i ] ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = ∑ n i =1 ( β 0 + β 1 x i ) ( ∑ n i =1 x 2 i ) - ∑ n i =1 x i ( β 0 + β 1 x i ) ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = n β 0 ( ∑ n i =1 x 2 i ) - ∑ n i =1 ( β 0 x i ) ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = β 0 . ˆ β 1 E [ ˆ β 1 | X = x ] = E [ n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 ] = n ∑ n i =1 x i E [ y i ] - ∑ n i =1 E [ y i ] ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = n ∑ n i =1 x i ( β 0 + β 1 x i ) - ∑ n i =1 ( β 0 + β 1 x i ) ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = β 1 ∑ n i =1 x 2 i + n ∑ n i =1 x i β 0 - ∑ n i =1 β 0 ∑ n i =1 x i - β 1 ( ∑ n i =1 x i ) ( ∑ n i =1 x i ) n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 = β 1 . - β 0 β 1 σ 2 Y 1 , Y 2 , ... , Y n L ( y - ; β 0 , β 1 , σ ) = n ∏ i =1 1 √ 2 πσ exp ( - ( y i - ( β 0 + β 1 x i )) 2 2 σ 2 ) = 1 (2 π ) n /2 σ n exp ( - 1 2 σ 2 n ∑ i =1 ( y i - ( β 0 + β 1 x i )) 2 ) ℓ ( y - ; β 0 , β 1 , σ ) = - n log ( √ 2 πσ ) - 1 2 σ 2 n ∑ i =1 ( y i - ( β 0 + β 1 x i )) 2 . β 0 ∂ l ∂ β 0 = n ∑ i =1 ( y i - β 0 - β 1 x i ) = n ∑ i =1 y i - n β 0 - β 1 n ∑ i =1 x i β 0 ˆ β 0 = - y - ˆ β 1 x - β 1 ∂ l ∂ β 1 = n ∑ i =1 2 x i ( y i - ( β 0 + β 1 x i )) = 2 ( n ∑ i =1 x i y i - n ∑ i =1 x i β 0 - n ∑ i =1 β 1 x 2 i ) ) = 2 ( n ∑ i =1 x i y i - n ∑ i =1 x i ( - y - β 1 x ) - n ∑ i =1 β 1 x 2 i ) = n ∑ i =1 x i y i - n ∑ i =1 x i - y - β 1 n ∑ i =1 x 2 i + β 1 n ∑ i =1 x i x = n ∑ i =1 x i y i - 1 n n ∑ i =1 x i n ∑ i =1 y i - β 1 n ∑ i =1 x 2 i + β 1 1 n n ∑ i =1 x i n ∑ i =1 x i - - x = ∑ n i =1 x i n - β 1 ˆ β 1 = n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 4 . SST = n ∑ i =1 ( y i - - y ) 2 = n ∑ i =1 y 2 i + - y 2 - 2 - yy i SSE + SSM = n ∑ i =1 ( y i - ˆ y i ) 2 + n ∑ i =1 (ˆ y i - - y ) 2 = n ∑ i =1 ( y 2 i + ˆ y 2 i - 2 y i ˆ y i + ˆ y 2 i + - y 2 - 2 - y ˆ y i ) = n ∑ i =1 ( y 2 i + 2ˆ y 2 i - 2 y i ˆ y i + - y 2 - 2 - y ˆ y i ) * = n ∑ i =1 ( y 2 i + 2ˆ y 2 i - 2(ˆ y i + ˆ i )ˆ y i + - y 2 - 2 - y ( y i - ˆ i ) ) ˆ i = y i - ˆ y i SSE + SSM = n ∑ i =1 ( y 2 i + 2ˆ y 2 i - 2(ˆ y i + ˆ i )ˆ y i + - y 2 - 2 - y ( y i - ˆ i ) ) = n ∑ i =1 ( y 2 i - 2ˆ y i ˆ i + - y 2 - 2 - yy i + 2 - y ˆ i ) ** = n ∑ i =1 ( y 2 i + - y 2 - 2 - yy i ) = SST ∑ ˆ i = 0 ∑ x i ˆ i = 0 n ∑ i =1 2 - y ˆ i = 2 - y n ∑ i =1 ˆ i = 0 n ∑ i =1 2ˆ y i ˆ i = n ∑ i =1 2( ˆ β 0 + ˆ β 1 x i )ˆ i ∑ x i ˆ i = 0 ˆ β 0 ˆ β 1 n ∑ i =1 x i ˆ e i = n ∑ i =1 x i ( y i - ( ˆ β 0 - ˆ β 1 x 1 ) ) = n ∑ i =1 x i y i - ˆ β 0 n ∑ i =1 x i - ˆ β 1 n ∑ i =1 x 2 i = n ∑ i =1 x i y i - ( n ∑ i =1 y i n - ˆ β 1 n ∑ i =1 x i n ) n ∑ i =1 x i - ˆ β 1 n ∑ i =1 x 2 1 = n ∑ i =1 x i y i - ∑ n i =1 x i ∑ n i =1 y i n + ˆ β 1 ( ( ∑ n i =1 x i ) 2 n - n ∑ i =1 x 2 i ) *** = n ∑ i =1 x i y i - ∑ n i =1 x i ∑ n i =1 y i n + n ∑ n i =1 x i y i - ∑ n i =1 y i ∑ n i =1 x i n ∑ n i =1 x 2 i - ( ∑ n i =1 x i ) 2 ( ( ∑ n i =1 x i ) 2 n - n ∑ i =1 x 2 i ) = n ∑ i =1 x i y i - ∑ n i =1 x i ∑ n i =1 y i n - ( n ∑ i =1 x i y i - ∑ n i =1 x i ∑ n i =1 y i n ) = 0 SSM = n ∑ i =1 ( ˆ y i - - y ) 2 = n ∑ i =1 ( ˆ β 0 + ˆ β 1 ⋅ x i - - y ) 2 = n ∑ i =1 ( ( - y - ˆ β 1 ⋅ x ) + ˆ β 1 ⋅ x i - - y ) 2 = n ∑ i =1 ˆ β 2 1 ⋅ ( x i - x ) 2 b ) = ˆ β 2 1 ⋅ S xx a ) = ˆ β 1 ⋅ S xy - - ˆ β 1 = S xy S xx V ( ˆ β 1 | X = x ) - ˆ β 1 = ∑ n i =1 ( x i - x ) ( y i - - y ) ∑ n i =1 ( x i - x ) 2 * = ∑ n i =1 ( x i - x ) y i ∑ n i =1 ( x i - x ) 2 . - - - - n ∑ i =1 ( x i - x ) - y = - y n ∑ i =1 x i - n ∑ i =1 x . - y = - y . xn - - y . xn = 0 -- -- V ( ˆ β 1 | X = x ) = V ( ∑ n i =1 ( x i - x ) y i ∑ n i =1 ( x i - x ) 2 X = x ) = ∑ n i =1 ( x i - x ) 2 V ( y i | x ) ( ∑ n i =1 ( x i - x ) 2 ) 2 = σ 2 ∑ n i =1 ( x i - x ) 2 ( ∑ n i =1 ( x i - x ) 2 ) 2 = σ 2 S xx - - - - - - - - - V ( y i | x = σ 2 ) - y i = β 0 + β 1 x i + i β ′ s x V ( y i | x = σ 2 ) = V ( ) = σ 2 - V ( ˆ β 0 | X = x ) - ˆ β 0 = - y - ˆ β 1 x , - V ( ˆ β 0 | X = x ) = V ( - y - ˆ β 1 x | X = x ) = V ( n ∑ i =1 y i / n | X = x ) + x 2 V ( ˆ β 1 | X = x ) = n ∑ i =1 V ( y i | x ) / n 2 + x 2 σ 2 / S xx = σ 2 ( 1 n + x 2 S xx ) . - - - - - - - - - Cov ( ˆ β 0 , ˆ β 1 | X = x ) - ˆ β 0 = - y - ˆ β 1 x , - Cov ( ˆ β 0 , ˆ β 1 | X = x ) =Cov ( - y - ˆ β 1 x , ˆ β 1 | X = x ) =Cov ( - ˆ β 1 x , ˆ β 1 | X = x ) = - x ⋅ Cov ( ˆ β 1 , ˆ β 1 | X = x ) = - x ⋅ V ( ˆ β 1 | X = x ) = - x σ 2 S xx - - - - - - - - - - 7 . V ( ˆ y 0 ) = V ( ˆ β 0 + ˆ β 1 x 0 ) = V ( ˆ β 0 ) + x 2 0 V ( ˆ β 1 ) + 2 x 0 Cov ( ˆ β 0 , ˆ β 1 ) = ( 1 n + x 2 S xx ) σ 2 + x 2 0 σ 2 S xx + 2 x 0 ( - x σ 2 S xx ) = ( 1 n + x 2 - 2 x 0 x + x 2 0 S xx ) σ 2 = ( 1 n + ( x - x 0 ) 2 S xx ) σ 2 . -- -- - E [ Y i - ˆ y i | X = x , X = x i ] = E [ Y i | X = x i ] - E [ ˆ y i | X = x ] = E [ β 0 + β 1 x i + i | X = x i ] - E [ ˆ β 0 + ˆ β 1 x i | X = x ] * = β 0 + β 1 x 0 - β 0 + β 1 x i = 0. ---- - - V ( Y i - ˆ y i | X = x , X = x i ) = V ( Y i | X = x i ) + V ( ˆ y i | X = x ) - 2Cov ( Y i , ˆ y i | X = x , X = x i ) = V ( β 0 + β 1 x i + i | X = x i ) + σ 2 ( 1 n + ( x - x i ) 2 S xx ) - 0 ** = σ 2 + σ 2 ( 1 n + ( x - x i ) 2 S xx ) - 0 = σ 2 ( 1 + 1 n + ( x - x i ) 2 S xx ) . - --- - - - - - V ( β 0 + β 1 x 1 ) = 0 ˆ y i y x x y H 0 : ρ = 0 v.s. H 1 : ρ ≠ 0 T = R √ n - 2 √ 1 - R 2 ∼ t n - 2 . n = 18 r = ∑ x i ⋅ y i - nx - y √ ( ∑ x 2 i - nx 2 )( ∑ y 2 i - n - y 2 ) = 672.8 - 18 ⋅ 337/18 ⋅ 42.98/18 √ (9854.5 - 337 2 /18) ⋅ (109.7936 - 42.98 2 /18) = - 0.827 - - T = - 0.827 √ 16 √ 1 - ( - 0.827) 2 = - 5.89. P ( t 16 ≤ - 4.015) * = P ( t 16 ≥ 4.015) = 0.05% t p 2 ⋅ 0.05% = 0.1% y = α + β x + ˆ β = ∑ x i y i - n ∑ x i / n ∑ y i / n ∑ x 2 i - n ( ∑ x i / n ) 2 = 672.8 - 337 ⋅ 42.98/18 9854.4 - 334 2 /18 = - 0.0372008 ˆ α = - y - ˆ β x =42.98/18 + 0.0372008 ⋅ 337/18 = 3.084259 - y x ˆ y = ˆ α + ˆ β x =3.084259 - 0.0372008 x x = 24

ii. Two day equals 48 hours, i.e., , thus The data set contains accurate data up to 26 hours, as for observations 17 and 18 (at 48 hour and 60 hours respectively) there was no eye-witness testimony direct available. Predicting 3- MT concentration after 26 hours may not be advisable, even though is within the range of the -values (5.5 hours to 60 hours). d. The pivotal quantity is given by: We have: From Formulae and Tables page 163 we have . Using the test statistic, the 99% confidence interval of the slope is given by: Thus the 99% confidence interval of is given by: . Note that in not within the 99% confidence interval, therefore we would reject the null hypothesis that equals zero and accept the alternative that at a 1% level of significance. This confirms the result in (2) where the correlation coefficient was shown to not equal zero at the 1% significance level. a. 10 . The linear regression model is given by: where i.i.d. distributed for . The fitted linear regression equation is given by: The estimated coefficients of the linear regression model are given by (see Formulae and Tables page 25): Thus, the fitted linear regression equation is given by: b. The estimate for is given by: *uses the result from Q5 for SSM We know the pivotal quantity: Note: we have the degree of freedom of because we have to estimate two parameters form the data ( and ). We have that . Thus we have that the 90% confidence interval is given by: Thus the 90% confidence interval of is given by . c. i) We test the following: with a level of significance . ii. The test statistic is: iii. The rejection region of the test is given by: iv. The value of the test statistic is given by: v. The value of the test statistic is in the rejection region, hence we reject the null hypothesis of a zero correlation. d. We have that has a student- distribution: The predicted value is given by: The estimated variance of the observation is give by: Thus, the 95% confidence interval for the value of given that is given by: Thus the 95% confidence interval of given is . e. i) We test the following hypothesis: ii. The test statistic is given by: iii. The critical region is given by: iv. The value of the test statistic is given by: where v. We have that . Thus, the -value is given by . The value of the test statistic is not in the critical region if the level of significance is lower than 0.34212 (which is normally the case). Hence, for reasonable values of the level of significance we would not reject the null hypothesis. f. The proportion of the variability explained by the model is given by: Hence, a large proportion of the variability of is explained by . 11 . The completed ANOVA table is given below: Source D.F. Sum of Squares Mean Squares F-Ratio Regression 639.5-475.6=163.9 163.9 19.99 Error 8.2*58=475.6 8.2 Total 12 . A simple linear regression problem: a. Since we know that , then where are sample standard deviations. Alternatively, you can use the fact that , so that from 4. below, . You take the positive square root because of the positive sign of the coefficient of . b. Given , we have: A 95% confidence interval of this estimate is given by: where is the sample variance of . c. A 95% confidence interval for is: d. and . e. A scatter plot or diagram of the fitted values against the residuals (standardised) will provide us an indication of the constancy of the variation in the errors. f. To test for the significance of the variable EPS, we test against . The test statistic is: This is larger than and therefore we reject the null. There is evidence to support the fact that the EPS variable is a significant predictor of stock price. g. To test against , the test statistic is given by: Thus, since this test statistic is smaller than do not reject the null hypothesis. 13 . The grand total sum is so that the grand mean is Also, Therefore the total sum of squares is: The sum of squares between the regions is: The difference gives the sum of squares within the regions: The one-way ANOVA table is then summarised below: ANOVA Table for the One-Way Layout Source d.f. Sum of Squares Mean Square F-Statistic Between Within Total Thus, to test the equality of the mean premiums across the regions, we test: against the alternative: using the -test. Since (approximately), we therefore reject . There is evidence to support a difference in the mean premiums across regions. The one-way ANOVA model assumptions are as follows: each random variable is observed according to the model where refers to the random error in the observation of the treatment which satisfies: and for all . The are independent and normally distributed (normal errors), and where is the overall mean and is the effect of the treatment with: a. 14 . We have the estimated correlation coefficient: i. We have the hypothesis: ii. The test statistic is: iii. The critical region is given by: iv. The value of the test is: v. We have . Thus the -value is 0.005 and we reject the null hypothesis of a zero correlation for level of significance less than 0.005 (usually it is larger, thus then we reject the null). b. Given the issue of whether mortality can be used to predict sickness, we require a plot of sickness against mortality: There seems to be an increase linear relationship such that mortality could be used to predict sickness. c. We have the estimates: i. Hypothesis: ii. Test statistic: iii. Critical region: iv. Value of statistic: v. We have from Formulae and Tables page 163: and . Thus the -value (using symmetry) is between 0.2 and 0.25. Thus, we accept the null hypothesis if the level of significance is smaller than the -value (which is usually the case). Note: exact -value using computer package is 0.2402. d. For a region with we have the estimated value: with corresponding variance: The corresponding 95% confidence limits are and . 1 . a. 15 . We have: Thus, the fitted model is given by . For we have: For we have: iii. We have . i) Hypothesis: ii) Test statistic: iii) Critical region: iv) Value of statistic: v) We have and . Thus the -value is between 0.1% and 0.2%. Accept the null hypothesis if the level of significance is lower than the - value (which is usually not the case). Hence, we have strong evidence against the "no linear relationship" hypothesis. Note: exact -value using computer package is 0.00070481. 1 . b. Calculating the sums of squares in this question is done similarly to question 13. We have: iii. Company A: fitted value Company D: fitted value iv. Observed statistic is on (3,12) d.f.. v. From Formulae and Tables page 173 and 174 we observe that and . Thus the -value is between 0.025 and 0.01, so we have some evidence against the "no company effects" hypothesis. Note: exact -value using computer package is 0.0213. 1 . In Table 3.4, the null hypothesis for TV is that in the presence of radio ads and newspaper ads, TV ads have no effect on sales. Similarly, the null hypothesis for radio is that in the presence of TV and newspaper ads, radio ads have no effect on sales. (And there is a similar null hypothesis for newspaper .) The low -values of TV and radio suggest that the null hypotheses are false for TV and radio. The high -value of newspaper suggests that the null hypothesis is true for newspaper. 2 . The fitted model is given by For males, Gender = 0, so For females, Gender = 1, so a. False. For a fixed value of IQ and GPA, if GPA , males earn less on average than females. b. False. For a fixed value of IQ and GPA, if GPA , females earn less on average than males. c. True. For a fixed value of IQ and GPA, if GPA , males earn more on average than females. d. False. See above. a. 3 . I would expect the polynomial regression to have a lower training RSSthan the linear regression because it could make a tighter fit against data that matched with a wider irreducible error . b. I would expect the polynomial regression to have a higher test RSS as the overfit from training would have more error than the linear regression. c. Polynomial regression has lower train RSS than the linear fit because of higher flexibility: no matter what the underlying true relationship is the more flexible model will closer follow points and reduce train RSS. An example of this behaviour is shown on Figure 2.9 from Chapter 2. d. There is not enough information to tell which test RSS would be lower for either regression given the problem statement is defined as not knowing "how far it is from linear" If it is closer to linear than cubic, the linear regression test RSS could be lower than the cubic regression test RSS. Or, if it is closer to cubic than linear, the cubic regression test RSS could be lower than the linear regression test RSS. It is dues to bias-variance trade-off: it is not clear what level of flexibility will fit data better. a. 4 . The design matrix is b. The matrix is c. The matrix is d. Note: the inverse of a matrix is given by: Using this and the result from 2. we have: e. Using the result of 3. and 4. we have: 5 . Statement **(E)** is correct. Note that statement (A) is incorrect because, if food sales increases with one, the expected profit increases with (note the difference in the scale of profit (thousands) and food sales (in ten thousands). Similarly, (B), (C) and (D) are incorrect. 6 . Statement **(D)** is correct.We have observations, parameters (three explanatory variables and the constant), SST , and SSM . Thus we have: ˆ y = ˆ α + ˆ β 24 = 3.084259 - 0.0372008 ⋅ 24 = 2.19 x = 48 ˆ y = ˆ α + ˆ β 24 = 3.084259 - 0.0372008 ⋅ 48 = 1.30 x = 48 x β - ˆ β SE( ˆ β ) ∼ t n - 2 . SE( ˆ β ) = √ ˆ σ 2 ∑ x 2 i - nx 2 = √ ˆ σ 2 9854.5 - 337 2 /18 ˆ σ 2 = 1 n - 2 ( ∑ y 2 i - ( ∑ y i ) 2 / n - ( ∑ x i y i - ∑ x i ∑ y i / n ) 2 ∑ x 2 i - ( ∑ x i ) 2 / n ) = 1 16 ( 109.7936 - 42.98 2 /18 - (672.8 - 337 ⋅ 42.98/18) 2 9854.5 - 337 2 /18 ) = 0.1413014 SE( ˆ β ) = √ 0.1413014 9854.5 - 337 2 /18 = 0.00631331 - t 16,1 - 0.005 = 2.921 ˆ β - t 16,1 - α /2 SE( ˆ β ) < β < ˆ β + t 16,1 - α /2 SE( ˆ β ) - 0.0372008 - 2.921 ⋅ 0.00631331 < β < - 0.0372008 + 2.921 ⋅ 0.00631331 - 0.055641979 < β < - 0.0188 β ( - 0.0372008, - 0.0188) β = 0 β β ≠ 0 y i = α + β x i + i , i ∼ N (0, σ 2 ) i = 1, ... , n ˆ y = ˆ α + ˆ β x . ˆ β = s xy s xx = 1122 ∑ n i =1 x 2 i - nx 2 = 1122 60016 - 12 ⋅ ( 836 12 ) 2 = 1122 1774.67 = 0.63223 ˆ α = - y - ˆ β x = ∑ n i =1 y i n - ˆ β ∑ n i =1 x i n = 867 12 - 0.63223 ⋅ 836 12 = 28.205. - - ˆ y = 28.205 + 0.63223 ⋅ x . σ 2 ˆ σ 2 = 1 n - 2 n ∑ i =1 ( y i - ˆ y i ) 2 = 1 n - 2 SSE = 1 n - 2 ( SST - SSM ) * = 1 n - 2 ( n ∑ i =1 ( y i - - y ) 2 - ˆ β 2 1 ⋅ S xx ) = 1 n - 2 ( n ∑ i =1 y 2 i - n ⋅ - y 2 - ( ∑ n i =1 ( x i - x )( y i - - y ) ) 2 ∑ n i =1 x 2 i - nx 2 ) = 1 10 ⋅ ( 63603 - 1122 2 60016 - 836 2 /12 ) = 25.289 - - s 2 σ 2 /( n - 2) ∼ χ 2 n - 2 n - 2 ˆ α ˆ β s 2 = ˆ σ 2 10 ˆ σ 2 χ 2 0.95,10 < σ 2 < 10 ˆ σ 2 χ 2 0.05,10 10 ⋅ 25.289 18.3 < σ 2 < 10 ⋅ 25.289 3.94 13.8 < σ 2 < 64.2 σ 2 (13.8, 64.2) H 0 : β = 0 v.s. H 1 : β > 0, α = 0.05 T = ˆ β - β √ ˆ σ 2 / ( ∑ n i =1 ( x i - x ) 2 ) ∼ t n - 2 - C = {( X 1 , ... , X n ) : T ∈ ( t 10,1 - 0.05 , ∞ )} = {( X 1 , ... , X n ) : T ∈ (1.812, ∞ )} T = 0.63223 - 0 √ 25.289/( ∑ n i =1 x 2 i - nx 2 ) = 0.63223 - 0 √ 25.289/(60016 - 836 2 /12) = 5.296. - ( y i | x i ) - (ˆ y | x i ) √ V ( y i | x i ) t y i | x i - ˆ y | x i √ V ( y i | x i ) ∼ t n - 2 ˆ y | x i = ˆ α + ˆ β x i = 28.205 + 0.63223 ⋅ 53 = 61.713. x = 53 V ( y i | x i = 53) = ( 1 n + ( x - x ) 2 ∑ n i =1 ( x i - x ) 2 ) ˆ σ 2 = ( 1 12 + (53 - 836/12) 2 60016 - 836 2 /2 ) ˆ σ 2 = 6.0657. - - y x = 53 ˆ y - t 1 - 0.05/2 ⋅ √ V ( y i | x i = 53) < y | x = 53 < ˆ y + t 1 - 0.05/2 ⋅ √ V ( y i | x i = 53) 61.713 - 2.228 ⋅ √ 6.0657 < y | x = 53 < 61.713 + 2.228 ⋅ √ 6.0657 56.2 < y | x = 53 < 67.2 y x = 53 (56.2, 67.2) H 0 : ρ = 0.75 v.s. H 1 : ρ ≠ 0.75 T = Z r - z ρ √ 1 n - 3 ∼ N (0, 1) C = {( X 1 , ... , X n ) : T ∈ {( -∞ , - z 1 - α /2 ) ∪ ( z 1 - α /2 , ∞ )}} Z r - z ρ √ 1 9 = 3( z r - z ρ ) = 3(1.2880 - 0.97296) = 0.94512 z r = 1 2 log ( 1 + 0.85860 1 - 0.85860 ) = 1.2880 z ρ = 1 2 log ( 1 + 0.75 1 - 0.75 ) = 0.97296 r = ∑ n i =1 ( x i - x )( x i - - y ) √ ∑ n i =1 ( x i - x ) 2 ∑ n i =1 ( y i - - y ) 2 = 1122 ( ∑ n i =1 y 2 i - n - y 2 i )( ∑ n i =1 x 2 i - nx 2 ) = 1122 √ 962.25 ⋅ 1774.667 = 0.85860 - - - z 0.82894 = 0.95 p 2 ⋅ (1 - 0.82894) = 0.34212 R 2 = SSM SST = 1 - SSE SST =1 - ∑ n i =1 ( y i - ˆ y i ) 2 ∑ n i =1 ( y i - - y i ) 2 =1 - ∑ n i =1 y 2 i - n - y 2 - ( ∑ n i =1 ( x i - x )( y i - - y ) ) 2 ∑ n i =1 x 2 i - nx 2 ∑ n i =1 y 2 i - n - y 2 i = ( ∑ n i =1 ( x i - x )( y i - - y ) ) 2 ( ∑ n i =1 y 2 i - n - y 2 i )( ∑ n i =1 x 2 i - nx 2 ) = 1122 2 962.25 ⋅ 1774.667 = 0.737193. - - - - Y X 1 163.9 8.2 = 58 59 639.5 ˆ β = r s y s x r = ˆ β s x s y = 7.445(2.004/21.56) = 69.2%. s x , s y R 2 = r 2 r 2 = 0.4794 r = + √ 0.4794 = 69.2% EPS EPS = 2 ˆ STKPRICE = 25.044 + 7.445 (2) = 39.934. ( ˆ α + ˆ β x 0 ) ± t 1 - α /2, n - 2 × s ( 1 n + ( x - x 0 ) 2 ( n - 1) s 2 x ) = (39.934) ± t 1 - 0.025,46 =2.012896 × √ 247 ( 1 48 + (2.338 - 2) 2 (47) ( 2.004 2 ) ) = 39.934 ± 4.636 = (35.298, 44.570). - s 2 x X β ˆ β ± t 1 - α /2, n - 2 ⋅ SE( ˆ β ) = 7.445 ± 2.0147 × √ 247 2.004 √ 47 = 7.445 ± 2.305 = (5.14, 9.75). s = √ 247 = 15.716 R 2 = SSM SST = 10475 21851 = 47.94% H 0 : β = 0 H a : β ≠ 0 t ( ˆ β ) = ˆ β SE( ˆ β ) = 7.445 1.144 = 6.508. t 1 - α /2, n - 2 = 2.0147 H 0 : β = 24 H a : β > 24 t ( ˆ β ) = ˆ β - β 0 SE( ˆ β ) = 7.445 - 24 1.144 = - 14.47. t 1 - α , n - 2 = t 0.95,46 = 1.676, ∑ x = 2479 + 2619 + 2441 + 2677 = 10216 x = 10216/40 = 255.4. - - ∑ x 2 = 617163 + 687467 + 597607 + 718973 = 2621210. SST = ∑ ( x - x ) 2 = ∑ x 2 - Nx 2 = 2621210 - (40)(255.4) 2 = 12043.6. - - - - SSM = ∑ n i ( x i . - x ) 2 = 10 ( (247.9 - 255.4) 2 + (261.9 - 255.4) 2 + (244.1 - 255.4) 2 + (267.7 - 255.4) 2 ) = 3774.8. - - - SSE = SST - SSM = 12043.6 - 3774.8 = 8268.8. 3 3774.8 1258.27 1258.27 229.69 = 5.478 36 8268.8 229.69 39 12043.6 H 0 : α A = α B = α C = α D = 0 all variances are equal H a : at least one α is not zero all variances are equal F F = 5.478 > F 0.95 (3, 36) = 2.9 H 0 x ij x ij = μ + α i + ε ij , for i = 1, ... , I , and j = 1, 2, ... , n i ε ij j th i th E [ ε ij ] = 0 V ( ε ij ) = σ 2 i , j ε ij μ α i i th I ∑ i =1 α i = 0. r = s ms √ s mm ⋅ s ss = ∑ ms - nms √ ( ∑ m 2 - nm 2 ) ⋅ ( ∑ s 2 - n - s 2 ) = 221, 022.58 - 1136.1 ⋅ 1934.2/10 √ (129, 853.03 - 1136.1 2 /10) ⋅ (377, 700.62 - 1934.2 2 /10) = 0.764. - - H 0 : ρ = 0 v.s. H 1 : ρ > 0 T = r √ n - 2 √ 1 - r 2 ∼ t n - 2 C = {( X 1 , ... , X n ) : T ∈ ( t n - 2,1 - α , ∞ )}) T = r √ n - 2 √ 1 - r 2 = 0.764 √ 10 - 2 √ 1 - 0.764 2 = 3.35 t 8,1 - 0.005 = 3.35 p ˆ β = s ms s mm = ∑ ms - nms ∑ m 2 - nm 2 = 221, 022.58 - 1136.1 ⋅ 1934.2/10 129, 853.03 - 1136.1 2 /10 = 1.6371 ˆ α = - y - ˆ β x = 1934.2 10 - 1.6371 1136.1 10 = 7.426 ˆ σ 2 = 1 n - 2 n ∑ i =1 ( y i - ˆ y i ) 2 = 1 n - 2 ( s ss - s 2 ms s mm ) = 1 8 ( ( ∑ s 2 - n - s 2 ) - ( ∑ ms - nms ) 2 ( ∑ m 2 - nm 2 ) ) = 1 8 ( 3587.656 - (1278.118) 2 780.709 ) = 186.902 V ( ˆ β ) = ˆ σ 2 / s mm = 186.902/780.709 = 0.2394 - - - - - H 0 : β = 2 v.s. H 1 : β < 2 T = ˆ β - β √ ˆ σ 2 / s xx ∼ t n - 2 C = {( X 1 , ... , X n ) : T ∈ ( -∞ , - t n - 2,1 - α )} T = ˆ β - β √ ˆ σ 2 / s xx = 1.6371 - 2 √ 0.2394 = - 0.74 t 8,1 - 0.25 = 0.7064 t 8,1 - 0.20 = 0.8889 p p p m = 115 ˆ s = 7.426 + 1.6371 ⋅ 115 = 195.69 ˆ σ 2 ( 1 n + ( x 0 - x ) 2 s mm ) = 186.902 ( 1 10 + (115 - 113.61) 2 780.709 ) = 19.1528 - 195.69 - t 8,1 - 0.025 ⋅ SE( s | m = 115) = 195.69 - 2.306 ⋅ √ 19.1528 = 185.60 195.69 + t 8,1 - 0.025 ⋅ SE( s | m = 115) = 195.69 + 2.306 ⋅ √ 19.1528 = 205.78 SST = ∑ y 2 - ( ∑ y ) 2 / n = 70.8744 - 29.12 2 /16 = 17.8760 ∑ x =4 ⋅ (1 + 2 + 3 + 4) = 40 ∑ x 2 = 4 ⋅ (1 2 + 2 2 + 3 2 + 4 2 ) = 120 ∑ xy =1 ⋅ 2.73 + 2 ⋅ 6.26 + 3 ⋅ 9.22 + 4 ⋅ 10.91 = 86.55 s xy = ∑ xy - ∑ x ∑ y / n = 86.55 - 40 ⋅ 29.12/16 = 13.75 SSM = = ˆ β 2 1 ⋅ s xx = ( 13.75 20 ) 2 ⋅ 20 = 9.453125 SSE =SST - SSM = 17.8760 - 9.453125 = 8.422875. ii. ˆ β = s xy s xx = 13.75 20 = 0.6875 ˆ α = - y - ˆ β x = 29.12 - 0.6875 ⋅ 40/16 = 0.1012 - ˆ y = ˆ α + ˆ β x = 0.1012 + 0.6875 x x = 1 ˆ y = ˆ α + ˆ β x = 0.1012 + 0.6875 ⋅ 1 = 0.7887 x = 4 ˆ y = ˆ α + ˆ β x = 0.1012 + 0.6875 ⋅ 4 = 2.8512 SE( ˆ β ) = √ 8.4229/14 20 = 0.1734 H 0 : β = 0 v.s. H 1 : β ≠ 0 T = ˆ β - β SE( ˆ β ) ∼ t n - 2 C = {( X 1 , ... , X n ) : T ∈ {( -∞ , - t n - 2,1 - α /2 ) ∪ ( t n - 2,1 - α /2 , ∞ )}} T = ˆ β - β SE( ˆ β ) = 0.6875 - 0 0.1734 = 3.965 t 14,1 - 0.001 = 3.787 t 14,1 - 0.0005 = 4.140 p p p SST =17.8760 SSM = ∑ n i ( - y i . - - - y ) 2 = 4 ∑ ( - y i - - - y ) 2 = 9.6709 SSE =SST - SSM = 17.8760 - 9.6709 = 8.2051 ii. ˆ μ =29.12/16 = 1.82 ˆ τ 1 =2.73/4 - 1.82 = - 1.1375 ˆ τ 2 =6.26/4 - 1.82 = - 0.255 ˆ τ 3 =9.22/4 - 1.82 = 0.485 ˆ τ 4 =10.91/4 - 1.82 = 0.9075 = 2.73/4 = 0.6825 = 10.91/4 = 2.7275 F (9.6709/3)/(8.2051/12) = 4.715 F 3,12 (4.474) = 2.5% F 3,12 (5.953) = 1% p p Multiple linear regression questions p p Y = 50 + 20 GPA + 0.07 IQ + 35 Gender + 0.01GPA × IQ - 10 GPA × Gender. Y = 50 + 20 GPA + 0.07 IQ + 0.01GPA × IQ. Y = 85 + 10 GPA + 0.07 IQ + 0.01GPA × IQ. < 3.5 > 3.5 > 3.5 V ( ) X = [1 - n x ] = - 1 x 1 1 x 2 1 x n X X X X = [ ] = [ ] = n [ ] 1 1 ... 1 x 1 x 2 ... x n 1 x 1 1 x 2 1 x n n ∑ n i =1 x i ∑ n i =1 x i ∑ n i =1 x 2 i 1 x x 1 n ∑ n i =1 x 2 i - - X Y - X Y = [ ] = [ ] - 1 1 ... 1 x 1 x 2 ... x n y 1 y 2 y n ∑ n i =1 y i ∑ n i =1 x i y i 2 × 2 M - 1 = [ ] - 1 = 1 det ( M ) ⋅ [ ] = 1 ad - bc ⋅ [ ] a b c d d - b - c a d - b - c a ( X X ) - 1 = 1 n ∑ n i =1 x 2 i - n 2 x 2 ⋅ [ ] = 1 s xx ⋅ [ ] - ∑ n i =1 x 2 i - nx - nx n - - 1 n ∑ n i =1 x 2 i - x - x 1 - - ˆ β =( X X ) - 1 X Y = 1 s xx ⋅ [ ] [ ] = 1 s xx [ ] = [ ] = = = [ ] - - 1 n ∑ n i =1 x 2 i - x - x 1 - - ∑ n i =1 y i ∑ n i =1 x i y i ∑ n i =1 y i ⋅ 1 n ∑ n i =1 x 2 i - ∑ n i =1 x i y i ⋅ x - ∑ n i =1 y i ⋅ x + ∑ n i =1 x i y i ⋅ 1 - - - y ∑ n i =1 x 2 i - ∑ n i =1 x i y i ⋅ x ∑ n i =1 x i y i - nx - y - - - y ( ∑ n i =1 x 2 i - nx 2 ) - ( ∑ n i =1 x i y i ⋅ x - nx 2 - y ) ∑ n i =1 x i y i - nx - y --- - - y ( ∑ n i =1 x 2 i - nx 2 ) - x ( ∑ n i =1 x i y i - nx - y ) ∑ n i =1 x i y i - nx - y --- - - y - s xy s xx x s xy s xx - ˆ β 1 ⋅ 10 n = 25 p = 3 + 1 = 4 = 666.98 = 610.48 SSE =SST - SSM = 666.98 - 610.48 = 56.5 SS /( ) 56.5/(25 4) 56.5/21