Hasani Davis is a longtime Wages of Wins fan and ABA referee (side note, check out this great interview he gave on the Wages of Wins podcast) Hasani has a fantastic story about data and “explaining” results.
In college my professor had research data where a team had collected data over the past 2 years on basketball referees. and found that the team in light colored jerseys got a higher amount of foul calls and the team with dark colors were penalized for fouls MORE.
Being as I was a young referee at the time I listened to the lecture and read the paperwork for about ten minutes before I cut in. For all the data they had collected saying how we view dark colors as EVIL and light colors as GOOD, they didn’t take into account that the HOME TEAM always has to wear light color jerseys and the away team darker (i just happened to have my rule-book with me at the time) and showed him and classmates.
Just because you find something with data we have to be sure as quants that it is relevant.
Hasani’s point is fantastic. We often collect data. We notice that often people fail to even verify if the data is useful. But beyond that, we often try and explain why the data says what it says. It we’re not careful, we can quickly come up with very compelling theories that may be missing the reality.