Hypothesis Testing

Statistical vs. Machine learning Hypothesis

Even though most of the concepts we will cover in this article are predominantly statistical, it is important to understand how the term hypothesis is perceived from either a purely statistical or machine learning perspective.

Steps to test a hypothesis

A hypothesis test evaluates two statements about a population. The statements are mutually exclusive. The test concludes which statement best reflects the sample data. A hypothesis test helps us determine the statistical significance of a finding.

Establish hypotheses

The first step in testing a hypothesis is first defining the hypothesis. This is done by establishing both a null and alternative hypothesis. A null hypothesis can be thought of as a statement claiming no relationship between two measured events. It is an assumption made, which may be based on domain experience.

Significance Level

  • It’s the degree of significance within which we accept or reject the null hypothesis. 100% accuracy is not possible for accepting or rejecting a hypothesis, therefore we select a level of significance that is usually 5%.
  • This is usually denoted with alpha and generally, it is 0.05 or 5%, which suggests your output ought to be 95% confident to present a similar kind of result in each sample.
  • Type I error: When we reject the null hypothesis, though that hypothesis was true. Type I error is denoted by alpha. In hypothesis testing, the normal curve that represents the critical region is known as the alpha region.
  • Type II error: When we accept the null hypothesis but it is false. Type II error is denoted by beta. In hypothesis testing, the normal curve that represents the acceptance region is known as the beta region.


The P-value or calculated probability is the probability of finding the observed or more extreme results when the null hypothesis (H 0) of a study question is true — the definition of ‘extreme’ depends on how the hypothesis is being tested.


The t-test is defined as the statistical test that examines whether the population means of two samples greatly differ from one another, using t-distribution which is used when the standard deviation is not known and the sample size is small. It is a tool to analyze whether the two samples are drawn from the same population.


Analysis of variance (ANOVA) can be defined as the statistical technique which is used to check if the means of two or more groups are significantly different from each other by analyzing variance. ANOVA checks the impact of one or more factors by comparing the means of various samples.

Types of ANOVA

1. One-way ANOVA

One-way ANOVA is a hypothesis test within which only one categorical variable or single factor is taken into consideration. With the help of F-distribution, it enables us to compare the means of three or more samples. The Null hypothesis (H 0) is the equity in all population means while an Alternative hypothesis is a difference in at least one mean.

2. Two-way ANOVA

Two-way ANOVA examines the result of two independent factors on a dependent variable. It also studies the inter-relationship between independent variables influencing the values of the dependent variable, if any.

Compare p-value to the significance level to retain or reject the null hypothesis

To know whether to keep or reject the null hypothesis, we can compare our significance level to the p-value. Let’s assume our significance level is 5% (or 0.05). The smaller the p-value, the greater the evidence is favoring the alternative hypothesis.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sunilkumar Prajapati

Sunilkumar Prajapati

Data Science enthusiast | Machine Learning | Deep Learning | Data Analyst | Software Support Engineer