Survivorship Bias

Here's an interesting problem. Supposedly, during the Second World War, analysis was performed on aircraft returning from their sorties. Analysts wanted to know where to place additional armour on aircraft so as to protect aircraft crew.

The picture below displays an aircraft representing the sum of all returning aircraft. The parts of this summated aircraft in black represent areas which suffered hits from bullet and cannon rounds. The white areas suffered little to no hits. From this evidence where do you place the additional armour?

The answer is that you place additional armour in the white areas and not the areas that have suffered the most damage. The reason being that only aircraft that survived and returned to base are represented in the return data. Aircraft that were shot down did not make it home to enter the dataset.

The white areas represent the pilot's cockpit and the aircraft's rudder. If either of these two areas are hit then the aircraft won't be returning home to be entered into the analyst's datasheet. A dead pilot or an aircraft that can't yaw is the worst that can happen to an aircraft and it won't be returning home to be logged.

Those of you who thought of reinforcing the wings or the engine area were caught out by survivorship bias. You only considered the data in front of you and not other data that would have been pertinent to the question.

We see examples of survivorship bias amongst tipsters. A tipster who has a lucky season will eventually regress to the mean simply because they were lucky once. By only taking tips from winning tipsters you have ignored the fact that they could quite have easily have given losing tips. The population of all tipsters do worse than break even, as proven by Joseph Buchdahl in Squares & Sharps, Suckers & Sharks. Tipping is a classic example of suriviorship bias in sports trading.