What is the value of a manager or coach?
Adam Smith argued (in An Inquiry into the Nature and Causes of the Wealth of Nations, published in 1776) that daily operations of a firm are run by “principal clerks” and such clerks are essentially homogenous. Or as Smith put it “their labour of inspection and direction may be either altogether or very nearly the same”.
The view put forward by Smith can be contrasted with the high regard many members of the media (and fans) have for coaches and managers in professional sports. There is a belief that certain coaches can – in the words of Bum Phillips (in describing Don Shula) — “He can take his’n and beat your’n, and then he can take your’n and beat his’n”
Coaches or Players?
It is these two perspectives that inspired a study conducted by Michael Leeds, Eva Marikova Leeds, Michael Mondello, and myself. Utilizing data from the NBA, we wished to see if coaches can actually impact the performance of the players. Specifically, we wished to know if a statistical relationship existed between player performance in the NBA and the identity of the player’s coach.
Our work has been detailed in an unpublished paper (that is currently under review at refereed journal). This work will also be detailed in our forthcoming book [Stumbling on Wins from Wharton School Publishing/Financial Times Press].
Normally I don’t go out of my way to discuss my unpublished papers in the forum. And I have definitely gone out of my way to avoid discussing the many new stories we plan for the next book.
But a few weeks ago I got a call from Ryan McCarthy. Ryan was working on a story for Slate.com looking at the value of NBA coaches. Given where we are at in writing the book, I thought I would share our basic findings.
From McCarthy’s article you get the general story. Here are a few excerpts (click on the previous link for the entire article, which is well worth reading):
According to a new study co-authored David Berri, an economist who runs the sports blog Wages of Wins, most NBA coaches are similar to company managers. In the study, Berri and his colleagues sought to investigate whether Adam Smith’s theory that workers make up the value of an organization-and that managers are nothing more than “principal clerks“-applies to the NBA. The economists looked at a group of 19 longtime NBA coaches that had helmed multiple teams, using a Bill Jamesian statistic called Win Score to evaluate how players performed under their tutelage. Only eight of the 19 coaches had any statistically discernible effect on team performance. Seven had a positive impact, with Phil Jackson topping the chart. Next on the list: Rick Adelman, Rudy Tomjanovich, Rick Carlisle, Don Nelson, Flip Saunders, and Gregg Popovich. The only coach who had a demonstrably negative impact on his players: the historically inept Tim Floyd. (For what it’s worth, Berri didn’t study Isiah Thomas. The NBA coaches study hasn’t been published yet; a version of it will be included in the 2009 book Stumbling on Wins, by Berri and Martin Schmidt.)
More interesting than the names on Berri’s list is his finding that the influence of even the best coaches was statistically very small and was distinguishable only from the worst-rated coaches, like Floyd. Even title-winning, Hall of Fame coaches like Pat Riley and Larry Brown were shown to have almost no impact on their teams. Players leaving Riley-led teams actually got better (except, it seems, for Antoine Walker).
And here is how McCarthy’s article ends:
Why is it that, in the NBA, inexperienced coaches can step in and succeed right away? (First-time coach Avery Johnson was named the NBA’s coach of the year in his second season on the job; newbie Doc Rivers won it after his first.) Berri’s contention is that an NBA coach’s record is determined almost entirely by the quality of his players. The claim makes sense: In comparison with football and baseball, NBA statistics vary little from year to year. The job of an NBA coach, then, may be less about coaxing better performances out of athletes than about getting their skills and personalities to fit together. By the time a player has moved through the basketball machine to the NBA, he’s a relatively finished product. Despite Mike D’Antoni’s best efforts, the plodding center Eddy Curry is doomed to be himself. “Think about it,” says Berri. “What is a coach going to say that will get Eddy Curry to rebound?”
Let me summarize: The majority of the coaches we looked at did not have a statistically significant impact on player performance. And some of these coaches are ranked among the all-time greats. Such findings suggest that the outcomes we observe for teams are mostly about the players, not the coaches. So teams that wish to improve should focus on the people in the uniforms, not the people wearing suits on the sidelines.
The Deck Chair Argument
When it comes to coverage of the NBA, one of the most valuable media resources we have is Henry Abbott. Every weekday Abbott tabulates virtually every story on the Association (and he adds quite a few also). And this story at Slate.com was no exception.
What a miserable article to read if you’re a coach. All those late nights of film study. All that competition for your job. All those tricks learned at conferences. All those books by the masters you have internalized.
And now there is evidence to support the notion you could be replaced by a deck chair.
Although I can see how people might think this is what our research is saying, I don’t think our results indicate that a “deck chair” could do as well as an actual NBA coach. To see why, consider the following story:
This past Saturday my family and I attended the Southern Utah University football game (against South Dakota). Sitting a few seats behind us was a man who had an annoying habit. Frequently he had the urge to yell advice down to the field of play. In the course of the game he had many “words of wisdom” for both the SUU coaches and the referees.
In listening to these words I concluded that it was unlikely that the coaches or referees were actually paying attention to this man. And I was pretty sure that this man knew that the people on the field did not care what he said. So why was he yelling? I think he was motivated by a desire to show people around him in the stands that he “knew” football. Unfortunately, his efforts to demonstrate his knowledge did not have his desired effect. The effect he actually had was to convince everyone around him (well, at least my wife and I) that he was an idiot.
Now let’s imagine that we had the power to grant this “idiot” his most fervent wish. Imagine that we could let him coach the team. In other words, rather than replace the coach with a deck chair, we replace the coach with an idiot from the stands.
What our study shows…. okay, our study doesn’t directly comment on this issue. Our sample only considers NBA coaches. We did not look at what would happen if a deck chair – or a person with the intellect of a deck chair – replaced an NBA coach.
What we did look at is the impact various NBA coaches had on player performance. And although we found a few who had a positive impact, many did not. This suggests that a few coaches might (and the word is might for the reasons cited in McCarthy’s article) be able to rise above the status of principal clerk. But for many coaches, we can’t make this distinction.
What does this mean? Certainly I suspect the coaches in the NBA know much more than the “idiots” – or deck chairs – in the stands. Our study, though, did not explore these differences. What we did try and do is explore the differences in NBA coaches. And our study found that in many cases, there were not any substantial differences. In sum, although one has to acquire substantial knowledge to be an NBA coach, there isn’t much one of these coaches is able to do to differentiate himself from his peers. Consequently, players perform in a similar fashion for most NBA coaches.
It’s important to note that this research is still on-going (and of course that can be said about all research). So there will be much more said on this story in our forthcoming book (but hopefully nothing more on this topic in this forum until the book is finished).
Speaking of the new book…we are not quite finished. Regular readers of this forum might have noticed that the posts – which used to appear almost daily – are now only showing up about three times a week. Specifically, I am trying to write a post for just Monday, Wednesday, and Friday.
That schedule should continue, at least for a few weeks. If it looks like we are not going to hit our deadline on the book, then the posts will start happening even less frequently. Hopefully this book will be finished soon, and then the posts will start to happen more often.
One last note on the posts… for the most part I am going to be focusing on events in the current season. So please look for stories similar to what I posted on Kevin Durant and the Oklahoma City Thunder. In other words, I will do my best to provide commentary on specific teams (and/or players) from the 2008-09 season. If there is a specific team (or player) you wish for me to examine, please let me know. In other words, I am more than happy to take requests (although it might take time to get the analysis posted).
The WoW Journal Comments Policy
Our research on the NBA was summarized HERE.
Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:
Finally, A Guide to Evaluating Models contains useful hints on how to interpret and evaluate statistical models.