TrueHoop posted a letter from a reader weighing in on the discussion of how to measure performance in the NBA. The title of the post – Let’s See the Proof – and the following statement, pretty much summarizes the argument.
…the only way to prove if Berri’s model is accurate is for him to use his model, at the beginning of the season, to predict which teams will end up with the most wins.
Although I am not sure that predicting a future season is the best approach, the essence of the argument is correct. As I have noted previously, part of the list of issues we consider in evaluating a model is how well the model explains what it purports to explain. Additionally we ask if the model allows us to forecast the future. In terms of the basketball model we offer in the Wages of Wins one has to ask how well it explains team wins. Furthermore, one has to wonder how well it allows us to explain future team and player performance.
The issue of forecasting team performance was addressed in a paper (A Re-Examination of Production Functions and Efficiency Estimates for the National Basketball Association) I have co-authored with Young Hoon Lee. This paper, which has been accepted for publication (although I am not sure when it will finally appear), explicitly addresses the proper specification and estimation of a production function in team sports. It is important to note that forecasting the future in the NBA was not the explicit purpose of the paper. Still, along the way to answering the question we did ask, we did provide some empirical results that suggest data generated by the NBA can be used to forecast the future.
Specifically, in addressing the issue of production functions we were able to show that between 65% and 75% of wins in the current season can be explained by the productivity of players in the previous season. The weakness of our study was that we did not take into account any factor that would cause a player to deviate from past productivity. In other words, we assumed that what you saw on a per-minute basis the last season is what you would see in the current season. We also did not have a model for rookies, which forced us to assume that your per-minute performance in the future would simply be the same as the per-minute performance we saw from rookies taken in a similar place in past NBA drafts.
Obviously what we used for both veteran and rookie performance could be improved upon. I would note that this research will continue in the future. The production function Young Hoon Lee and I developed does allow one to answer a number of questions that we think should be interesting to other economists.
So at this point all I can say is that our research has addressed some of the concerns noted by the comments posted at TrueHoop, but that research is still on-going. Certainly before we write the sequel to The Wages of Wins we will have more to say on this topic.
Before closing this post, I do want to make a comment on baseball. The statistics tracked to measure performance in baseball are often looked upon longingly by NBA fans. The thinking is that stats tracked on the diamond are “better” than what we see in basketball because baseball analysts do not need to worry as much about interaction between teammates in evaluating performance.
If we focus on the issue of forecasting, though, baseball stats are probably inferior to the numbers we see in basketball. As we note in The Wages of Wins, player performance in baseball is less consistent over time relative to what we see in basketball. Although we have increasingly sophisticated measures of performance for baseball players, it is still the case that before the season starts our ability to forecast the outcome in baseball appears to be somewhat weak.
Despite the inability to forecast the future in baseball, though, no one is advocating that we throw out all of baseball’s numbers. The numbers do provide some information. But expecting a crystal ball from any of these stats is probably unrealistic.
And I would argue, a crystal ball is not something anyone would really want. After all, why would anyone bother playing these games if I or anyone else could tell you the outcome before the games were played?