The Simpson Paradox

The Simpson's paradox is one of the easily understandable and at the same time astonishing phenomena in statistics. It occurs whenever groups of data show a particular trend, but that trend is reversed when the groups are combined. With the help of a simple example, the paradox can be understood immediately.

We consider the two disjoint sets \(\#1\) and \ (\#2\) as well as \(G = \#1 \cup \#2\) and test within these sets the success rate of \(A\) and \(B\):

\(\#1 \cup \#2\)\(\frac{3}{6}=50\%\)\(\frac{4}{7}=57\%\)\(B\)

It turns out that \(A\) is more successful in both \ (\#1\) and \(\#2\) than \(B\), but surprisingly \(B\) is more successful in \ (G\) than \( A\). This example is also among those with the smallest amount of \(G\) with \(|G|=13\). There is no \(G\) with \(|G|<13\) ( proof by brute force).

We now subdivide the set \(G\) instead of \(2\) into \(3\) disjoint subsets \(\#1, \, \#2, \, \#3\) with \(\#1 \cup \#2 \cup \#3 = G\) . Then we construct the exciting case that for every element \(e_k \neq \emptyset\) the power set \(P(G)\) of \(G\) following applies: $$\forall e_1, e_2 \in P(G): |e_1| \neq |e_2| \Rightarrow win(e_1) \neq win(e_2) \land |e_1| = |e_2| \Rightarrow win(e_1) = win(e_2)$$ $$\forall e_1, e_2 \in P(G): |e_1| \neq |e_2| \Rightarrow win(e_1) \neq win(e_2) \land |e_1| = |e_2| \Rightarrow win(e_1) = win(e_2)$$

After a few hours of brute force on a standard Core i7, the following example can be found:

\(\#1\)\(\frac{6}{7}=85,71\%\)\(\frac{12}{15}=80,00\%\) \(\frac{22}{37}=59,46\%\) \(A\)
\(\#2\)\(\frac{95}{167}=56,89\%\) \(\frac{48}{88}=54,55\%\) \(\frac{38}{67}=56,72\%\) \(A\)
\(\#3\)\(\frac{48}{144}=33,33\%\) \(\frac{16}{50}=32,00\%\) \(\frac{2}{20}=10,00\%\) \(A\)
\(\#1 \cup \#2\)\(\frac{101}{174}=58,05\%\) \(\frac{60}{103}=58,25\%\) \(\frac{60}{104}=57,69\%\) \(B\)
\(\#1 \cup \#3\)\(\frac{54}{151}=35,76\%\) \(\frac{28}{65}=43,08\%\) \(\frac{24}{57}=42,11\%\) \(B\)
\(\#2 \cup \#3\)\(\frac{143}{311}=45,98\%\) \(\frac{64}{138}=46,38\%\) \(\frac{40}{87}=45,98\%\) \(B\)
\(\#1 \cup \#2\cup \#3\)\(\frac{149}{318}=46,86\%\) \(\frac{76}{153}=49,67\%\) \(\frac{62}{124}=50,00\%\) \(C\)

If such cases occur in reality, any conclusions on a recommendation of the success of a group are both meaningful and meaningless.

At this point, we recommend the exciting reading Causality: Models, Reasoning and Inference by Judea Pearl .