Wayne Winston has a short post on John Hollinger’s Player Efficiency Rating.
Here is Wayne’s entire post (in case you didn’t feel like clicking over):
I have had a great time teaching a sports and math class to Kelley School of Business students at Indiana University. One of my ace students Paul Aynilian did a study trying to estimate John Hollinger’s famous ESPN PER Ratings based on box score statistics. We found that
45.75*(Points/Minute)+22.55*(Rebounds/Minute)+32.8*(Assists/Minute)+58.2*(Steals/Minute)-48.65*(Turnovers/Minute) -39.73*(Missed FG’s per minute) -20.6*(Missed FT per minute)+38.37*(Blocked Shots Per Miute)-18.68*(Personal Fouls Per Minute)
explains over 99% of the variation in this season’s PER rankings and is off by an average of .37 in estimating the PER of the top 200 NBA players whose stats are on Yahoo.com. So basically our simple formula virtually duplicates the PER rating without a lot of mumbo jumbo.
The disturbing thing about these weights is that if an NBA player shot 33 .33% (1-3) then the more shots they take the higher their PER because shooting 1 for 3 gives you a net contribution of 2(45.75)-2(39.73)>0!! Clearly this is bad because a 33% shooter is not a good shooter and with these weights the more shots a bad shooter takes, the higher his PER rating.
Let me add a bit to this story. We can also see how well PER explains wins with a simple regression of team wins on a team’s PER. The results indicate that PER only explains 33% of team wins. You can do better if you add the team defensive measures used in Wins Produced. Those factors increase explanatory power to 55% (if you weigh the team defensive factors as they are in Wins Produced) or 82% (if you allow the weights of each individual factor to vary). Either way, PER does not explain 95% of team wins (as we see with Wins Produced).
In sum, PER is a model that is not theoretically sound (yes, it over-values inefficient scoring) or empirically sound (it doesn’t explain wins). So why is it so popular? I am going to offer four explanations:
1. PER certainly looks like a very complicated formula. And so it is easy for people who are not mathematically inclined to think it is “advanced” (although it is highly correlated with the much simpler NBA Efficiency metric – which few people think is “advanced”).
2. Measures like PER and NBA Efficiency are not about “efficiency”. But they do explain – as noted in Stumbling on Wins — player evaluations. In other words, these “advanced” measures are consistent with popular perceptions.
3. Hollinger writes at ESPN, which is a very large platform. Clearly this gives his work wide exposure. And this exposure has led writers who are not at ESPN to quote PER often.
4. And finally, Hollinger doesn’t respond much to criticism. Back in 2006 he responded to something I said once. But that was it.
Hollinger simply doesn’t spend much time addressing the problems with his model. And I think that might be an effective strategy. Addressing your critics is something we encourage in academia. At least, at our academic meetings, discussants are assigned to each paper and these discussants are supposed to critique your work. This process is supposed to make the work better. But outsider of academia, I am not convinced that addressing critics has the effect that academics suspect.
Paul Krugman, for example, frequently addresses his critics. But he clearly is not well-loved by these same critics. The problem is that although Krugman addresses his critics, he clearly doesn’t agree with these people. And that is the problem for these critics. People don’t want their criticisms addressed. They want people to agree with these criticisms. When that doesn’t happen, well…these critics get very angry.
So I think Hollinger has probably taken the correct path. By essentially ignoring his critics he has defused a great deal of hostility. And that might have had a small impact on the popularity of PER.
However we explain the popularity, it is clear that – no matter how often Wayne and I point to its flaws — PER is not going away. As long as Hollinger and the other people at ESPN (and other people in the sports media) want to use this measure to evaluate basketball players, this measure — with all its flaws — will remain part of the discussion of player value in the NBA.