TS - Introduction to Significance Tests Lesson
Introduction to Significance Tests Lesson
In the last unit we studied Confidence Intervals. In this unit we study the second type of statistical inference about a population based on information from a sample which is called performing a "TEST of SIGNIFICANCE." These tests have a goal of assessing evidence provided by data about "some claim" about the population. Lots of people and businesses have a vested interest in making a claim and convincing the world that it is true. Think about commercials that you have heard and advertising that you have seen for all different types of products.
Claim: "I make 80% of my free throw attempts in basketball."
"Cola drinks lose sweetness over time."
"Students over the age of 30 have better attitudes toward school."
"This shampoo will rid you of dandruff."
Validating these statements (or refuting them) is done through a significance test.
Significance tests answer two questions:
a) does the sample result (however small) reflect the true population parameter
b) would the outcome easily be explained by chance or is there really something responsible
Procedures:
1) careful statement of alternatives
2) identification of the parameter of interest (mean or proportion)
3) clear statement of the alternatives - "null hypothesis" and "alternative hypothesis"
We need to be cautious when believing an unsupported claim or even a claim that "appears" to have some basis for belief. Employing the use of a test on any hypothesis is the way to rule out the possibility that the results stated are due to chance rather than to the suggested cause.
"Does lowering the price of a product really lead to increased sales?" How can we be sure that the change in sales is due to the price reduction and not to chance? How much change would we expect to see before we declare that chance cannot explain the change? When is the change large enough to rule out chance? All of these are reasonable questions.
Here is how it works...
The statement (claim, hypothesis) being tested is called the null hypothesis.
We don't really believe the null hypothesis and are hoping to PROVE IT WRONG.
By rejecting the null hypothesis we are forced to conclude that the alternative must be true. Usually the null hypothesis states "NO effect" or "NO difference", a statement of mathematical equality.
The alternative hypothesis is the claim about the population that we are trying to find evidence to support. This will be a mathematical statement of inequality based on the question wording.
When the alternative hypothesis cites a specific change (> or <) we have a ONE-SIDED test.
When the alternative hypothesis cites a change but doesn't specify direction (≠) we have a TWO-SIDED test.
More Important Points
Hypotheses are ALWAYS talking about the POPULATION so population parameters are used in their statement.
Tests of significance assess the strength of evidence against the null hypothesis by assigning a P-value indicating the probability that the outcome would occur by chance if the null hypothesis were true. VERY unlikely or very low P-values provide evidence against the null hypothesis.
Using significance tests with an alpha level selected in advance suggests a DECISION will be made based on the outcome - alpha being the standard against which the P-value is measured for decision purposes.
Acceptance sampling is a procedure used by manufacturers, such as Lays Potato Chips and many others, who inspect a sample of products and will either accept or reject an entire batch. Acceptance sampling calls for slight adjustments to our tests of significance.
There are only two outcomes - accept or reject. In this case the hypotheses are about meeting some pre-established quality control standard.
H0 implies meeting a standard
Ha indicates not meeting a standard.
For more detail and examples please download the lesson below.
Test of Significance Links to an external site.
IMAGES CREATED BY GAVS