At the start of each football game a coin is tossed to determine who will receive the opening kick-off. Let’s imagine if instead of just one team calling heads or tails the fans in attendance were also asked to make a call. And let’s further imagine that if you make the correct call, you get to stay. But if you are wrong, you have to leave.
Okay, now let’s do a bit more imagining. Let’s say 80,000 fans are in attendance – and since fans know it is a fair coin (equally likely to be heads or tails) — about 40,000 make the wrong call. So these fans exit the building. After they are gone, let’s imagine we play the same game again. This time, about 20,000 fans are incorrect and they depart. And then we play it again, and again, and again… After three tosses we are left with about 10,000 fans. After seven tosses there are about 625 fans. After twelve tosses we should still have about 40 people left in the stands.
Now what have these 40 people learned? These people have just called a coin flip correctly twelve consecutive times. Clearly these people are incredible at this game.
If we play the game one more time, though, we should expect about 20 more to depart. What will these departing fans have learned? Well, clearly they just didn’t match-up with the 20 who got the 13th call correctly. And they better go home and figure out why that particular match-up didn’t work if they ever wish to see another football game.
The above scenario was adapted from Nassim Nicholas Taleb’s book Fooled by Randomness [(2005): pp. 165]. This book argues that people often have problems understanding randomness. And what it says is relevant to how people see the NBA playoffs.
Orlando Better than Cleveland?
Consider the Eastern Conference Finals.
In the regular season the Cleveland Cavaliers had the best won-loss record and posted the highest efficiency differential (offensive efficiency minus defensive efficiency) in the league. In fact their mark of 9.74 was the fifth best in the NBA since 1973-74.
In the first two rounds of the playoffs the Cavaliers performed as expected by sweeping both the Pistons and Hawks. And then in the Eastern Conference finals they met the Orlando Magic.
The Magic took six games to eliminate the 76ers in the first round. And in the second round – against a Celtic team that did not have Kevin Garnett – the Magic took seven games. Entering the series against the Cavs, few people expected the Magic to win. And when we look at these two teams, no one should have expected a Magic victory. Certainly everyone should have thought it was possible for Orlando to win. But the expected outcome –given the quality of the two teams – is that Cleveland should win.
Despite this expectation, though, the Magic did win. Consequently we are now hearing explanations for how this happened. One specific story is that the Magic match-up so well against the Cavs. After all, in the regular season the Magic beat the Cavs twice.
When we look at the Cavs season, though, we see that the Washington Wizards – the very worst team in the Eastern Conference – also beat the Cavaliers twice. Other than Cleveland, the Wizards only had two victories against Sacramento, Minnesota, and New Jersey. So Cleveland was the only winning team the Wizards beat more than once. From this evidence I guess we should now conclude that the Wizards match-up well with Cleveland as well.
Of course, there’s another possibility. The Magic won the first game of the Eastern Conference Finals by one point. In the fourth game the Magic prevailed in overtime. Had the Cavs won either game this series would have had probably had a Game Seven in Cleveland. Now it’s possible the Magic could have won seventh game as well. But that would not have been the likely outcome.
Perhaps another way to think about this is to imagine Cleveland and Orlando played this series 10,000 times. Given this number of trials we would expect to see every possible scenario appear at least once. And given the quality of the teams at the onset, the most frequent outcome in our 10,000 trials is Cleveland winning. But Orlando winning in six games would also show up in our trials. If we considered all 10,000 outcomes, though, we would end up concluding Cleveland was the better team. In reality, though, we didn’t get 10,000 trials. All we got was one. And that one trial had Orlando winning. From this one trial people are concluding that Orlando – who was clearly not as good during the 82 game regular season – is actually better than Cleveland.
It’s important to remember that despite what you hear on television, the playoffs are not really designed to identify the best team. The Drunkard’s Walk: How Randomness Rules Our Lives by Leonard Mlodinow (a wonderful book) contains the following passage relevant to any discussion of predicting the winner in a best-of-seven playoff series.
“…if one team is good enough to warrant beating another in 55% of its games, the weaker team will nevertheless win a 7-game series about 4 times out of 10. And if the superior team could beat its opponent, on average, 2 out of 3 times they meet, the inferior team will still win a 7-game series about once every 5 match-ups. There is really no way for a sports league to change this. In the lopsided 2/3-probability case, for example, you’d have to play a series consisting of at minimum the best of 23 games to determine the winner with what is called statistical significance, meaning the weaker team would be crowned champion 5 percent or less of the time. And in the case of one team’s having only a 55-45 edge, the shortest significant “world series” would be the best of 269 games, a tedious endeavor indeed! So sports playoff series can be fun and exciting, but being crowned “world champion” is not a reliable indication that a team is actually the best one.” (p. 70-71).
The small sample we see in the playoffs imposes an element of randomness on the outcome. This randomness makes it interesting, but it also makes decision-making complicated. Already we are hearing how the Cavaliers need to change their roster to compete with the Magic in the future. In fact, the people who are running the Cavaliers are making such statements.
But I don’t think this is the lesson to be learned from the Eastern Conference Finals. The lesson to be learned is that in a small sample of games an upset can occur. It’s possible for a good team like the Magic to defeat a great team like the Cavs. Just as it was possible for a very bad team like the Wizards to win two games against Cleveland in the regular season. And just as it is possible –given the set-up of the imaginary scenario outline above – for 20 people to call a coin toss correctly 13 times in a row.
The True Hoop Stat Geek Smackdown
Okay, just to be clear. The playoffs are not a scientific test that will tell us the identity of the best team.
That being said… heading into the NBA Finals I am still holding on to the top spot in the True Hoop Stat Geek Smackdown. For the Finals I am taking the Lakers, and if LA does win then I am the winner.
If Orlando wins, though, Jeff Ma finishes in first. Ma has picked Orlando to win the Finals in seven games. Does this mean that Ma believes Orlando is the best team? Not according to Henry Abbott. As Abbott notes, “In essence, every single stat expert thinks the Lakers will win this series.” Abbott goes on to note, “As Hollinger and Berri have both picked the Lakers, Jeffrey Ma is the only person besides Berri with a chance to win. Once again, he (Ma) writes, “I find myself in the unenviable position of making a pick simply to be contrarian and hoping for the best.”
Let’s summarize. We have a contest to see who has the best approach to picking winners in the NBA playoffs. It’s possible for Jeff Ma to win this contest, but only by going against his approach. So if Orlando wins, does this mean Ma’s approach is the best? How can that be when Ma’s approach clearly says before the series starts that LA is the better choice?
Given these questions – and with tongue firmly planted in cheek – I am going to have to lodge an official protest with the True Hoop Smackdown management. If Orlando does prevail (and as we saw in the Eastern Conference Finals, that’s certainly possible) I will follow the lead of Norm Coleman (who is apparently willing to go to the Supreme Court to prevent the funniest senator in the history of Congress from taking his seat) and do whatever I can to prevent Ma from being declared the winner.
Let me close by making two observations. First, much of what I just said about small samples and randomness was noted in the comments on the last post. Secondly, if the Lakers do prevail and I am the winner, then I take back everything I just said about randomness and the playoffs. Clearly, if you have the right model (and of course I do), then the playoffs are very predictable (by the way, that was a joke also).
The WoW Journal Comments Policy
Our research on the NBA was summarized HERE.
Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:
Finally, A Guide to Evaluating Models contains useful hints on how to interpret and evaluate statistical models.