We begin with the assumption that the null hypothesis is true, and then proceed to test this assumption, but researchers are usually interested in rejecting the null.Normally we believe a difference exists; a decision to reject the null is usually the desired outcome (we want a low p-value).We are using a logic of proof by con- tradiction: we want to find support for the alternative hypothesis by showing that there is no support for its opposite, the null hypothesis.
Does this mean that if we fail to reject the null, the difference we are searching for does not exist?
Not necessarily: failing to reject the null hypothesis of no dif- ference simply means there is no evidence to think that the null hypoth- esis is wrong. This does not necessarily mean, however, that it is the right. There might actually be a difference out there but on the basis of the sample result such a difference has not been detected.This like the presumption of innocence in criminal law. A defendant is presumed not guilty unless the evidence is strong enough to justify a verdict of guilty.However, when someone has been found not guilty on the strength of the available evidence, it does not mean that the person is in fact innocent: all it means is that, given that either verdict is possible, we do not choose ‘guilty’ unless stronger evidence comes to light.Similarly, with a verdict of ‘no difference’, failing to reject the null hypothesis does not mean the alterna- tive is wrong.It simply means that on the basis of the information available, the null can explain the sample result without stretching our notion of reasonable probability.
Therefore, failing to find a significant difference should not be seen as conclu- sive.If we have good theoretical grounds for suspecting that a difference really does exist, even though a test suggests that it does not, this can be the basis of future research.Perhaps he variable has not been operationalized effectively, or the level of measurements does not provide sufficient information, or the sample what not appropriately chosen or was not large enough.In the context of research, inference tests do not prove anything; they are usually evidence in an ongoing discussion or debate that rarely reaches a decisive conclusion.