Probability Fundamentals

Jul 1

Overview

In this worksheet, you will...

Review key laws and formulas for finding probabilities, including the Inclusion-Exclusion Rule, the Multiplication Rule, and the Complement Rule.
Examine the concepts of independent events and mutually exclusive/disjoint events, including how to use these to solve problems.
Practice solving problems by applying these key rules and concepts.

Some of these building-block concepts in probability will likely be familiar to you, while others are new or unintuitive to grasp. Our goal is to become fluent in using all of these with confidence, and being able to state how each rule and concept comes up in problems which may use them at multiple steps along the way.

Later on, it won't always be obvious which rule of probability is most useful for a problem-- often, the hardest point of problems in probability theory is deciding how to get started. We want to know how to use each law so we can more easily decide between them and tackle more challenging problems which present multiple possible approaches.

Key Rules of Probability

For all the rules below, let $\Omega$ represent the outcome space we are working on: the set of all possible events. $A$ and $B$ are subsets of $\Omega$, describing events that may (or may not) occur.

For instance: if we roll a six-sided die and look at the result, then $\Omega=\{ 1,2,3,4,5,6\}$. The event $A$ might represent the event that the roll is even, e.g. $A=\{ 2,4,6\}$, while $B=\{ 1,2\}$ might represent the event that the roll is less than or equal to 2.

Axioms of Probability:

$0 \leq P(A) \leq 1$: Probabilities are always between 0 and 1. In particular, $P(\emptyset) = 0$, while $P(\Omega) = 1$.
$P(\emptyset)=0$: An impossible event, one whose entire possibilities are the empty set, $\emptyset$, has probability 0.
$P(\Omega)=1$: Since we're guaranteed that something will happen, the probability of some event in $\Omega$ occurring is $P(\Omega)=1$.
If $A$ and $B$ are disjoint events, then $P(A\cup B)=P(A\text{ or } B) = P(A)+P(B)$. Disjoint events, also known as mutually-exclusive events, are non-overlapping and cannot occur at the same time, i.e. $P(A\cap B) = P(A\text{ and }B)=0$, or more formally, $A\cap B = \emptyset$. (We'll soon discuss what to do if $A$ and $B$ are not disjoint events.)

Equally Likely Events

The simplest rule for computing probabilities is as follows: if in an outcome space $\Omega$, all of the possible outcomes are equally likely, then the probability of any particular event $A$ is:

$$P(A) = \frac{\text{number of events in } A}{\text{number of total events in } \Omega}$$

This allows us to get started with quantifying probabilities, such as the possible outcomes of a dice roll, or likelihood of a person showing up in a random sample from a population.

A quick note: when a problem mentions that something is "chosen at random", you may take that to assume all possible choices are equally likely.

The Inclusion-Exclusion Rule: "OR" Statements

As we saw above, there is a special way to compute $P(A \text{ or } B)$ when the two events $A$ and $B$ are mutually exclusive: we simply add the probabilities! However, what happens when there's some overlap between the two? Consider the Venn diagram below, where the area of each region corresponds to its probability:

Two overlapping circles, labeled A and B, within a rectangle labeled Omega.

We can see that if we were to find $P(A \text{ or } B)$ by adding the areas of the circles $A$ and $B$, we'd be double-counting the overlapping region. To correct for this overcounting, we have the Inclusion-Exclusion Rule:

$$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$$

Example: About 45% of Americans own a dog, 30% own a cat, and 17% own a dog and a cat. To find the chance that a randomly selected American owns either a cat or a dog, we would use the Inclusion Exclusion Rule:

$$P(\text{own a cat OR dog}) =$$ $$P(\text{cat}) + P(\text{dog}) - P(\text{cat AND dog})$$ $$= 0.45 + 0.3 -0.17 = 0.58$$

Keep in mind that our use of "or" here is not exclusive: "or" allows for the possible of both conditions to be met at once!

Also note that the Inclusion-Exclusion Rule does still work for mutually exclusive events, but since $P(A \text{ and }B)= 0$ for mutually exclusive events, it reduces to the shorter $P(A) +P(B)$ formula we have from the Axioms.

The Multiplication Rule: "AND" Statements

The Multiplication Rule is arguably the most useful, and yet most complicated, of the key probability rules. For now, we will restrict ourselves to looking at the Multplication Rule for Independent Events.

Two events $A$ and $B$ are independent if knowing whether or not $A$ occurred has no effect on the chances of the event $B$ occurring (and vice versa).

If (and only if) two events $A$ and $B$ are independent, then

$$P(A \text{ and } B) = P(A)\cdot P(B)$$

Examples of independent events: If we toss two coins, the event that the first coin is heads and the event that the second is heads are independent.

NON-examples of independent events: From the previous example, dog and cat ownership in American families are NOT independent. Intuitively, some people like having lots of pets, so they may be more likely to get both a dog and a cat. Without having the data, you may make a different assumption that people have a preference for cats OR for dogs, and are less likely to own one of each. In either case, if we have reason to suspect that two events may not be independent, we must be careful!

What if the events aren't independent? In general, if events are not independent, then $P(A\cap B) = P(A) \cdot P(B|A)$, where $P(B|A)$ is read as "the probability of B, given A". This is called a conditional probability, and will be the subject of a later lesson.

In general, you must identify some reasoning for why two events should be independent. Oftentimes a problem will give to you that you should treat things as independent events. Without that, we should not automatically assume that we can treat events as independent! It can be surprising to discover over the course of a problem whether or not events are independent; this is part of what makes probability such an interesting subject!

Notation: $P (A\cap B)$ is often written simply as $P(AB)$, in part because $AB$ looks like a product of terms!

The Complement Rule: "NOT" Statements

A motivating example: Suppose you roll a six-sided die three times. To find the chance of rolling at least one 6, we could rewrite this as the pro that the 1st OR 2nd OR 3rd roll is a 6. However, this requires a more complex version of the Inclusion-Exclusion Rule, which ends up having 8 terms to compute (some of which are identical). You're welcome to try this one out: drawing a 3-way Venn diagram helps. If you do, you'll find it is cumbersome keeping track of the overlapping terms. We can imagine that if we were to roll the die 4, 5, or a general $n$ times, it would be even more laborious to work out all these terms. There must be a better way to approach these types of problems!

When finding the probability of an event is hard to approach directly, consider finding its complement. Notated $A^c$, "A complement" is the set of all outcomes in $\Omega$ not in $A$. We can then find the probability of $A$ using the Complement Rule:

$$P(A) = 1 - P(A^c)$$

This works because the union of $A$ and $A^c$ is $\Omega$, and $A$ and $A^c$ are mutually exclusive (basically by definition). Thus, $P(A) + P(A^c) =1$, and subtracting $P(A^c)$ from both sides gives us the Complement Rule.

Returning to the above example of rolling three dice, we have to consider what the complement of "at least one 6" is. Since "at least one" translates to "1, 2, or 3, sixes", the complement is "no sixes". Thus, we can find the probability as follows:

$$P(\text{roll at least one 6})$$ $$= 1 - P(\text{no 6s rolled})$$ $$=1 - P(\text{1st not 6, 2nd not 6, AND 3rd not 6})$$ $$= 1 - \left(\frac{5}{6}\right)^3$$

Here, the last part was obtained by noting that the rolls are independent, and thus we can multiply the probability that each of the rolls is not a 6 together.

Note: A common mistake on this type of problem is to say that the complement of "no 6s" is "all 6s" (or similar). Be careful; while these feel like "opposites", these two are not complements! A complement is simply all the ways a particular desired event doesn't happen, not that the polar opposite of the event happens.

Problem-Solving Strategies

When you're stumped on how to compute the probability of an event, try to break it down in terms of "OR", "AND", and "NOT" statements.
For computing probabilities involving "OR" statements, we use addition, but check whether the events are mutually exclusive:

$$ P(A \text{ or }B) = P(A) + P(B) - P(A \text{ and }B) $$

If the events are mutually exclusive, $P(A \text{ or }B) = P(A) + P(B)$.

For computing probabilities involving "AND" statements, we use multiplication, but check whether the events are independent.

$$ P(A \text{ and } B) = P(A) \cdot P(B|A) $$

If the events are independent, $P(A \text{ and } B) = P(A) \cdot P(B)$. For now, all the practice problems below requiring "and" statements can be done with independent events.

For "NOT" statements, use the complement rule (which uses subtraction). If an event as written seems difficult to compute its probability, consider finding its complement instead!

$$ P(A) = 1 - P(A^c) $$

Practice

For all of the problems below, note which laws of probability you are using as you solve them. Try rewording these problems in terms of "AND", "OR", and "NOT" statements if you are stumped! Visual aids are also very useful; for instance, for a set of two dice, you may wish to draw a 6 by 6 grid to visualize all of the possible (equally likely) outcomes.

Answers are available here.

Essential Problems

Select a word chosen at random from this eleven word sentence. What is the chance the selected word has two or more vowels (A,E,I,O, or U)?
Roll two six-sided dice. What is the chance that:
- (a) Both results are 4 or less;
- (b) The smaller (minimum) of the results is 4 or less?
Toss three fair coins. What is the chance that their results are not all the same? (E.g., HHT)
About 10% of Americans are left-handed. If you pick 5 Americans at random, what is the chance that:
- (a) at least one of them is left-handed?
- (b) all of them have the same handed-ness? (For simplicity, let's assume the remaining 90% of Americans are right-handed.)
Continued from the cats and dogs example above (see the Inclusion-Exclusion Rule): What is the probability that a randomly chosen family owns neither a cat nor a dog?