Bayes Theorem Or The Probability Of Causes

Bayes’ theorem is one of the pillars of probability calculus. It is a theory put forward by Thomas Bayes (1702-1761) in the 18th century. But what is the purpose of the research of this famous scientist? The probability expresses, in a random process, the ratio between the number of “favorable” cases and the number of “possible” cases.

Many theories of probability have been developed that govern our existence today. When we go to the doctor, he prescribes the drug that will most likely prove useful in our case, just as advertisers dedicate their campaigns to the people who are most likely to acquire the product they want to promote or, again, the tourists and travelers who they choose the path where there is likely to be less queue.

The law of total probability is one of the most famous, so before talking about Bayes’ theorem , we will have to dedicate a few lines to the explanation of the first. To try to understand it, just give an example. Let’s say that, in a random country, 39% of the population is made up only of women. We also know that 22% of women and 14% of men are unemployed.

What is the probability (P) that a person chosen at random from the working population in this country is unemployed?

According to probability theory, the data would be expressed as follows:

• The probability that the person is female: P (M)
• The probability that the person is male: P (H)

Knowing that 39% of the population is made up of women, we deduce that: P (M) = 0.39.

It is therefore clear that: P (H) = 1 – 0.39 = 0.61. The problem posed at the beginning also gives us conditional probabilities:

• Probability that a person is unemployed, knowing that he is a woman -> P (P | M) = 0.22
• Probability that a person is unemployed, knowing that he is male – P (P | H) = 0.14

Using the law of total probability we will have:

P (P) = P (M) P (P | M) + P (H) P (P | H)

P (P) = 0.22 × 0.39 + 0.14 × 0.61

P (P) = 0.17

The odds of a randomly chosen person being unemployed will be 0.17. We observe that the result is halfway between the two conditional probabilities (0.22 <0.17 <0.14). Moreover, it is closer to the value of men because, in the population of this imaginary country, they are the majority.

Let’s discover Bayes’ theorem

Now suppose that an adult is chosen at random to fill out a form and it is observed that he has no job. In this case, and taking into account the previous example, what is the probability that this randomly chosen person is a woman -P (M | P) -?

To solve this problem we will apply Bayes’ theorem, which is used to calculate the probability of an event by having information about it in advance. We can calculate the probabilities of an event A knowing that it satisfies certain characteristics (B).

In this case, we are talking about the likelihood that the person randomly chosen to fill out a form is a woman. But it will not be independent of whether the selected person is unemployed or not.

The formula of Bayes’ theorem

Like any other theorem, we need a formula.

It sounds complicated, but everything has an explanation. We think in parts. What does each letter mean?

• B is the event on which we have preliminary information.
• The letter A (n) refers to the different conditioned events.
• In the numerator part we have the conditional probability. This refers to the probability that something (an event A) will occur, knowing that another event (B) also occurs. It is defined as P (A | B) and is expressed as: The probability of A given B.

• In the denominator, we have the equivalent of P (B) and the same explanation as the previous point follows.

An example

Returning to the previous example, suppose that an adult is chosen at random to fill in a questionnaire and it is observed that he is unemployed. What are the chances that this chosen person is female?

We know that 39% of the active population is made up of women, while the rest are men. Furthermore, we know the percentage of unemployed women, 22%, and that of men, 14%.

Finally, we also know that the odds of a randomly chosen person being unemployed are 0.17. If we apply the formula of Bayes’ theorem, the result we will obtain is that there is a probability of 0.5 that a person chosen at random from the unemployed will be a woman.

P (M | P) = (P (M) * P (P | M) / P (P)) = (0.22 * 0.39) / 0.17 = 0.5

Bayes’ theorem derives from the conjunction of the compound and absolute probability theorem, which we explained at the beginning. Its main feature is that it works in all interpretations of probability.

Since it can be used to calculate the probability of a cause, which triggered the event, its importance lies in the way it has historically affected the study of statistics. Today, in fact, two main schools are known (one frequentist and the other, in fact, Bayesian) which oppose each other starting from the interpretation that is given to this theory.

Let’s close with a curiosity: did you know that electronic spam (Internet spam, emails, advertisements) works thanks to Bayes’ theorem?