Inferential Statistics 

Inferential statistics are mathematical calculations performed to determine if the results from a small sample of data are likely due to chance or are a true representation of the larger population. Inferential statistics are used to determine the probability of chance alone leading to the sampled results.  

The phrase "statistically significant" is a key concept in inferential analysis. Unlike the common use of the word significant, calling a result statistically significant does not mean that the result is important or momentous. Instead, statistical significance is an estimate of the probability that the observed association or difference is due to chance. In other words, tests of statistical significance describe the likelihood that an observed result would be observed even if there were no real association or difference actually present. 

• “Significant” = the results are very unlikely to have occurred due to chance.
• “Not significant” = the results could have to have occurred due to chance.

A few other key terms for inferential statistics:

• Significance level = the probability of saying a result is significant (not due to chance) when it actually was due to chance.  A significance level of 0.05 (the standard used in biology) indicates a 5% risk of concluding that a result is significant when it was actually due to chance.
• p value = the probability value (p-value) is the probability of obtaining the sampling results due to chance.  
  • The smaller the p-value, the stronger the evidence that the results are significant (not due to chance).   
  • In biology, a p-value of less than 0.05 is considered significant. A p-value greater than 0.05 is considered non-significant. 
  • P values are expressed as decimals although it may be easier to understand what they are if converted to a percentage. For example, a p value of 0.0254 is 2.54%. This means there is a 2.54% chance the results could be random (i.e. happened by chance). Since 2.54% (0.0254) is less than 5.00% (0.05), this result would be considered "significant."

​The following factors contribute to determining if a statistical test is significant:

1. How large is the difference between the means/medians of the groups? Other factors being equal, the greater the difference between central tendency, the greater the likelihood that a statistically significant difference exists. If the means/medians of groups are far apart, we can be fairly confident that there is a real difference between them.
2. How much overlap is there between the groups? This is a function of the dispersion of the data within the groups. Other factors being equal, the smaller the variances of the groups under consideration, the greater the likelihood that a statistically significant difference exists. We can be more confident that two groups differ when the data points within each group are close together.
3. How many measures (trials or data points) are in the samples? The size of the sample (n) is extremely important in determining the significance. With increased sample size, measurements of the sample tend to become more stable representations of the population because larger samples reduce the impact of random variation. If the result we find remains constant as we collect more and more data, we become more confident that we can trust the effect that we are finding.
4. The significance level being used to test the significance determines how confident the conclusion needs to be. A larger significance level requires less confidence in the conclusion. It is much harder to conclude a result is significant when results are only allowed to occur by chance 1 out of 100 times (p < .01) as compared to 5 out of 100 times (p < .05).​

In most research settings, there are two very distinct types of hypotheses: 

• the research/experimental hypothesis - a statement of an expected or predicted relationship between two or more variables. It’s what the experimenter believes will happen in the research study. This hypothesis sets the stage to design a study to collect data to test its truth or falsity.  
• the statistical hypotheses - the null and alternative hypothesis of inferential statistical tests. The statistical hypotheses are statements about whether a pattern/trend/difference is SIGNIFICANT (meaning, likely due to more than chance in sampling).  The statistical hypotheses do not necessarily provide support for or against the research hypothesis that was tested. They just indicate whether chance alone likely could be the reason for the results. 

 
​The statistical hypotheses are:

•  Null hypothesis (H0).  The null hypothesis states that the results could be due to chance, that there is no significant relationship/difference compared to what could have resulted from random chance in sampling.
•  Alternative Hypothesis (H1).  The alternative hypothesis states that results are not likely due to chance,  that there is a significant relationship/difference compared to what could have resulted from random chance in sampling.

Selecting an Inferential Test

The type of inferential analysis to perform depends on the research question and the type of data collected.  It is often easiest to select the inferential analysis tool based on the type of graph!  ​

Bar chart:  
When the manipulated variable is qualitative and data are not skewed (meaning they fit a normal distribution), the MEAN with STANDARD DEVIATION should be graphed using a bar chart. Both the T-test and ANOVA are used to determine if there is a significant difference between the means for the different levels of the manipulated variable.

• T-Test is used when there are two levels of manipulation
  • Paired T-test - used when the two levels are from "before and after" data
  • Unpaired T-test - used when the two levels are not collected from the same sample before and after a treatment
• ANOVA is used when there are two levels of manipulation

​Box and Whisker: 
The MEDIAN with QUARTILE bars should be graphed using a box and whisker chart when data are skewed (meaning they do not fit a normal distribution curve). The Kruskal-Wallis  test is used to determine if there is a significant difference between the medians of the three or more levels of the manipulated variable.

Stacked-bar Chart:
A stacked bar chart should be created when there are levels of manipulation and a qualitative responding variable. The Chi-Square Test for Independence determines if a difference exists between observed counts and the counts that would be expected if there were no relationship at all between the variables in the population. ​This test compares observed frequencies in a contingency table to expected frequencies that would occur if the variables were completely independent. 

Pie Chart:
A pie chart should be created when there are no levels of manipulation and there is a qualitative responding variable. The Chi-Square Goodness of Fit test determines if a difference exists between observed counts and the counts that would be expected based on a theoretical distribution or null hypothesis. 

Scatterplots and Line Graphs with a LINEAR relationship:
A scatter plot OR a line graph showing the mean with standard deviation bars should be created when the manipulated variable is quantitative. A LINEAR trendline should be added.  Pearson’s Correlation is the test statistic that measures the statistical relationship, or association, between two continuous variables.​

Scatterplots and Line Graphs with a NON-LINEAR but MONOTONIC relationship:
A scatter plot OR a line graph showing the mean with standard deviation bars should be created when the manipulated variable is quantitative. A NON-LINEAR trendline should be added when the data is MONOTONIC (meaning X and Y have a consistent relationship, positive, negative or inverse). Spearman's Rank Correlation is a measure of correlation for data that is non-linear. It assesses how well the relationship between two variables can be described using a monotonic function.​

Scatterplots and Line Graphs with a NON-LINEAR and NON-MONOTONIC relationship
A scatter plot OR a line graph showing the mean with standard deviation bars should be created when the manipulated variable is quantitative. A NON-LINEAR trendline should be added when the data is NON-MONOTONIC (meaning sometimes Y increases and sometimes Y decreases with X). ANOVA is used to determine if there is a significant difference between different levels of the manipulated variable. ANOVA is suitable when the quantitative manipulated variable has been divided into discrete levels or groups for experimental purposes. For example, testing specific concentrations (0, 5, 10, 15 mg/L) or time points (0, 24, 48, 72 hours) where each level is treated as a separate category.​