Evaluating The Coaching (The Coach is Wrong Redux)

“What the ancients called a clever fighter is one who not only wins, but excels in winning with ease.”
― Sun Tzu, The Art of War

A while back (a year ago to be exact), I wrote a lengthy post on the value of Coaching. This was not by any means my first jaunt down this path either. Some notable example are:

Prof. Berri’s  has also gone down this path before. To quote:

Readers of Stumbling on Wins would note that Bradbury’s results for baseball are quite similar to what we reported for the NBA. The study we review (which I co-authored with Mike Leeds, Eva Marikova Leeds, and Mike Mondello and published in the International Journal of Sport Finance) looked at 62 NBA coaches across thirty years of data. Across this sample, only 14 coaches were found to have a statistically significant and positive impact on player performance. So most NBA coaches – like most baseball managers — do not appear to make their players more productive.

Only a few coaches actually make their players individual production rise in a statistically significant fashion.

We, as NBA fans, may hear a lot of noise about what a difference a great coach can make. I can even list some of the more commonly repeated refrains :

  • He’s a leader.
  • He’s a motivator.
  • He inspires his team.
  • He makes his team better.

But as Dave notes, this is not – for the most part – generally true. The simple truth is that NBA coaches are in general evaluated improperly. For the most part, Players are who they are and coaches don’t generally affect that (with the aforementioned exceptions).

Reason#1 why Phil should be your first call if you have a job opening for a coach

Given this mathematical fact, how then do we proceed? Is there any actual value in the function of coaching – and if there is, how do we capture it? My previous and current research has suggested some answers. Today we focus on the allocation of playing time.

Coaches matter because they decide who plays

The graph that follows contains every player season for every team since 1978 for the National Basketball Association with greater that 100 Minutes Played. Each point represents a player playing for one team for one season and shows their minutes played per game and their Wins Produced per 48 minutes played. It looks like so:

In essence this is a graph of a player’s perceived value in the eyes of their coach (as represented by the minutes played per game) and their actual value (as represented by their actual productivity in WP48 using the new wins produced model). The thing that jumps out very quickly is that while there is correlation between perceived and real value (see the R2 = 27%) that only accounts for 27% of the variation we see.

This reinforces a point that we’ve been emphasizing recently. Playing the right guys is a non trivial skill. The data shows that there are real differences on a year to year, team to team basis in what actually gets players on the court. Teams and coaches simply do not play their best players; instead they play the players who they think are their best players.

That makes all the difference in the world.

Coaches do matter, but not in the ways that the media tries to sell us. It’s about who they put on the court. The surprising thing is that proper allocation of those minutes is actually a rare skill. Let’s see what is revealed when we look at all the data since the 1977-1978 Season:

And the rankings:

And the summary:

To me the conclusion is obvious. Success at minute allocation is not an accident. There are coaches (Phil Jackson and Jerry Sloan for example) and Teams (the Lakers, The Suns, the Jazz) who do an inherently better job at identifying what players help you win and getting them on the court. More telling is that there are organizations that never really get it (Wizards, Warriors ,Raptors) and they rightly have earned reputations as historically hapless.

Let’s take a closer look at the last five years:

Important points of note:

  • The Lakers, Suns, Celtics, Heat and Hawks have done the best job at getting the most out of their rosters.  These are all perennial playoff contenders so this is not really surprising (although Phoenix is about to crash hard).
  • The Twolves did the best last season and this bodes well for their dark horse hopes this year .
  • The bottom of the list is very surprising though. The Clippers, Pistons, Raptors or Warriors at 26 thru 29 are not really surprising but that the Daryl Morey led Rockets track as consistently among the worst franchises at properly using the talent available to them does.
  • Utah seems to have lost their way after loosing Jerry Sloan and with the upheaval in their front office after being a model franchise for decades at playing the right guys.
  • The Lakers have not even though they lost Phil. I will note that it took Chicago a few years to lose their way after Phil and that Mike Brown was a mixed bag in Cleveland.
  • The Pacers, Thunder and Magic all posted uncommonly good years for them last year and a regression to the mean may be coming.
  • The Wizards lost their mind a bit the last two years at 30 for the last two years under Flip Saunders. The coaching change to Randy Wittman ( #8 in 2000, #2 in 2001,#6 in 2002 in Cleveland, #9 in Minnesota in 2008) bodes well for their playoff hopes.

As for the conclusion, certain coaches have an ability to identify and play talent. Having one of those coaches (or developing one see Spolstra,Eric) is a key advantage to winning in the NBA.

Seems kinda obvious for a conclusion.

Oh, and Phil is two for two so far.

As for the next bit to tackle on coaching evaluation? Here’s some pieces to give you a few hints.


Comments are closed.