Conditional Probability (and Independent Events) #

1.1 Conditional probability #

Conditional probability answers: What is the probability of event \( B \) , given that \( A \) has occurred?

How to think about it: Meaning:

• \( P(A\cap B) \) is the probability that both events happen together (joint probability).
• Conditioning on \( A \) means: we restrict attention to the outcomes where \( A \) is true.

Conditioning “shrinks the universe”: once \( A \) is known to have happened, we only count outcomes inside \( A \) .

Conditioning on \( A \) means we restrict our attention to outcomes inside \( A \) .

1.2 Multiplication rule (joint probability) #

Start from the definition:

Rearranging the definition gives the multiplication rule:

Similarly:

For three events (chain form):

This chain form is the backbone of many probability models (including ML sequence models).

This is the bridge between: conditional probability and joint probability. In ML, this “chain idea” shows up when modelling sequences and joint distributions.

1.3 Conditional probability of independent events #

Two events \( A \) and \( B \) are independent if knowing one does not change the probability of the other.

Equivalent tests:

Common confusion: Mutually exclusive events cannot be independent unless one of them is impossible (unless one has probability 0). If two events cannot happen together, learning one occurred forces the other to be false.

If \( A\cap B = \varnothing \) , then \( P(A\cap B)=0 \) , but \( P(A)P(B) \) is positive if both events can occur.

1.4 A quick “independent events” example (engineering style) #

If a washer needs service with probability 0.30 and a dryer needs service with probability 0.10, and they are assumed independent, then the probability both need service is found by multiplying the probabilities.

1.5 When independence fails (why conditional probability matters) #

If \( P(B\mid A) eq P(B) \) , then A changes B and the events are dependent.

flowchart LR
  A["Know A happened"] --> B{"Does B change?"}
  B -->|No| C["Independent: use multiplication P(A∩B)=P(A)P(B)"]
  B -->|Yes| D["Dependent: compute P(B|A) using joint/total probability"]

Home | Conditional Probability & Bayes’ Theorem