Hypothesis Tests: Hypothesis Formulation, Understanding P-Value, and the Level of Significance Test
Understanding the problem statement is always the first point in figuring out which statistical test to carry out on provided data and parameters — sample or population.
Below is a guide on figuring out what test to perform —
Source: Statistical Tests with Python — Source: https://plainenglish.io/
Null vs Alternative Hypothesis
A null hypothesis represents the claim which is a well-established or justified belief. In addition to well-established beliefs, it represents the default state of the claim in the real world.
An alternate hypothesis represents the new claim that needs to be established.
While Designing or creating a hypothesis test, we must make a selection for Null or Alternative Hypotheses based on the following:
- The theory/evidence we plan on nullifying or invalidating will be used as our Null Hypothesis; thus we move towards designing or selecting the appropriate statistical tests or significance test(based on a provided sample or population data) to evaluate the strength of the evidence against the nullifiable theory also called the null hypothesis (H_0)
- The alternative hypothesis (H_a ) is the claim we are trying to find evidence in favour of.
The null hypothesis should always contain a statement of equality or maintenance of a status quo. Another way of thinking of it is that the null hypothesis is a statement of “no difference.” The reason we have equality in all null hypothesis formulas (=, >= or <=:: equal, less than or equal to and less than or equal to).
Like the Null hypothesis, The alternative hypothesis could take one of three forms, depending on the context of the test — (≠, < or > Not equal to, greater than or less than)
Remember, a significance test is designed to evaluate the strength of the evidence against some null hypothesis parenthesis is the claim we are trying to find evidence in favour of.
P-Value
With these established, we move on to computing the p-value against some Level of Significance to reject or not reject the null hypothesis above.
When we reject the null hypothesis (p<LOS — we have enough statistical evidence to disagree with the status quo), we are invariably stating the alternative hypothesis is more accurate: we are accepting the alternative theory as against the status quo.
Computing P-Value and Comparing with the Significance Benchmark
P >= LOS: we don’t have enough statistical evidence to reject the null hypothesis.
Remember, the primary purpose of any given hypothesis test is to nullify a theory or invalidate a hypothesis or disprove the status quo. A P-value greater than the threshold means we don’t have enough evidence to reject or void the status quo(null hypothesis).
Level of Significance and Significance Testing
The reason for this is simply — The P value is defined as the probability under the assumption of no effect or no difference (null hypothesis), that is status quo is maintained — the higher the value the more status quo should be maintained and a lower value means evidence to reject the status quo getting stronger. The higher the p-value, the more there is not enough evidence to reject the status quo or null hypothesis. The lower the P-value, the more statistical evidence exists to reject the status quo or null hypothesis. We are in fact validating or maintaining that the status quo is accurate or not accurate with our P-value after a defined threshold of 10%, 5% or 1%.
Thus, if we computed a p-value or probability value based on a specified Significance test’s threshold (10%, 5% or 1% — Level of Significance or LOS) — we can conclude there is enough evidence to reject (p < LOS) the status quo or to state there is not enough evidence to reject the status quo(p>=LOS).
I like to know your thoughts in the comment session.
