Chapter 9 Notes

1 Notes for STA 210 Chapter 9 Hypothesis Testing: Concepts Hypothesis testing is an inferential technique in which we test a claim about an unknown population parameter using sample data. We begin a hypothesis test by stating two hypotheses: Null Hypothesis (H 0 ) - The status quo. Things are the same way they have always been. Nothing to see here. Alternative Hypothesis (H a ) - Usually what we suspect, or hope is true. Something interesting is happening. Example: A pharmaceutical company develops a new drug to reduce acne. They must perform a hypothesis test to see if it really works. Null Hypothesis - The drug has no effect on acne. Alternative Hypothesis - The drug effectively reduces acne. Example: A child welfare agency plans to conduct a hypothesis test to see if the number of cases of child abuse and neglect have increased since the pandemic started. Null Hypothesis - The number of cases of child abuse and neglect is the same as it was before the pandemic. Alternative Hypothesis - The number of cases of child abuse and neglect has increased since the pandemic started. Example 1: In an effort to improve your mood, you recently started limiting your screen time to 2 hours per day. State the null and alternative hypotheses for a hypothesis test to see if your strategy is working. Once a person has set the null and alternative hypotheses, they collect data that will help them decide which hypothesis is likely true. Just like a court case where the defendant is considered "innocent until proven guilty," with hypothesis testing, we assume the null hypothesis is true until we see convincing evidence that the alternative hypothesis is true. We say that the results of a hypothesis test are statistically significant if the observed data is so inconsistent with the null hypothesis that the difference is unlikely to have happened by chance. For example, if almost all of the test subjects saw a visible improvement in their acne when taking an experimental medication, and almost no one in the placebo group, that probably wasn't a fluke. Their clear skin was probably due to the medication , and so the results would be considered statistically significant. We side with the alternative hypothesis when the results are statistically significant.

2 So how much evidence is considered "enough" to reject the null hypothesis and say that we have statistically significant results? Our decision is determined by the p-value. The p-value is the probability, assuming the null hypothesis is true , that we would obtain a sample statistic at least as extreme as we did. • Large p-value → This means that if the null hypothesis is true, there is a strong chance that we would have observed this data. A large p-value supports the null hypothesis (H 0 ). We fail to reject the null hypothesis. The results are not statistically significant. Nothing to see here. • Small p-value → This means that if the null hypothesis is true, we were unlikely to have observed this data. A small p-value supports the alternative hypothesis (H a ). We reject the null hypothesis. The results are statistically significant. Woo hoo! We have found something interesting! We set a threshold for the p-value before we perform the hypothesis test. This cut-off value, called α (pronounced "alpha"), is used to determine when the p-value is small enough for us to reject the null hypothesis, thus concluding that the results are statistically significant. The significance level, α , is the probability of mistakenly rejecting the null hypothesis when it is actually true. We want to keep our chances of making such a mistake very low, so we set the significance level low, often using α = 0.05 as our significance level. Speaking of mistakes, there is always a chance that we will make the wrong decision when performing a hypothesis test. There are four possible scenarios for the correctness, or incorrectness, of a hypothesis test decision: The null hypothesis is true, and we fail to reject it. That is, H 0 is true, and we choose H 0 as the "correct" hypothesis. We have made the right decision! The alternative hypothesis is true, and we reject the null hypothesis. That is, H a is true, and we choose H a as the "correct" hypothesis. We have made the right decision! The null hypothesis is true, but we reject it. That is, H 0 is true, but we choose H a as the "correct" hypothesis. This the wrong decision! Siding with the alternative hypothesis when the null hypothesis is actually true is a Type I Error . The alternative hypothesis is true, but we fail to reject the null hypothesis. That is, H a is true, but we choose H 0 as the "correct" hypothesis. This is the wrong decision! Siding with the null hypothesis when the alternative hypothesis is actually true is a Type II Error . Note that we never know if we made a correct decision or an error when we perform a hypothesis test. However, we do know that if we perform many hypothesis tests using a significance level of α = 0.05, then over the long run we would make a Type I error about 5% of the time. We control for the Type I error because it typically has more serious consequences (for example, saying that a pharmaceutical drug works when it really doesn't).

3 Example: A pharmaceutical company develops a new drug to reduce acne. They must perform a hypothesis test to see if it really works. H 0 : The drug has no effect on acne. H a : The drug effectively reduces acne. a) The researchers collect data and obtain a p-value of 0.023. Using a significance level of α = 0.05, make a decision about whether to reject the null hypothesis or not. SOLUTION: Since the p-value of 0.023 is less than α = 0.05, we reject the null hypothesis. b) Interpret the conclusion in the context of this problem. SOLUTION: The drug effectively reduces acne. c) Are these results statistically significant? SOLUTION: Yes. Since we found a small p-value and thus sided with the alternative hypothesis, our results are statistically significant. d) Which type of error might we have made, a Type I or Type II error? SOLUTION: It is possible that we made a Type I error because we sided with H a , but it is possible that H 0 was actually correct. Example 2: A child welfare agency plans to conduct a hypothesis test to see if the number of cases of child abuse and neglect increased since the pandemic started. H 0 : The number of cases of child abuse and neglect is the same as it was before the pandemic. H a : The number of cases of child abuse and neglect has increased since the pandemic started. a) The researchers collect data and obtain a p-value of 0.11. Using a significance level of α = 0.05, make a decision to reject the null hypothesis or not. b) Interpret the conclusion in the context of this problem. c) Are these results statistically significant? d) Which type of error might we have made, a Type I or Type II error?