Data analytics, which is popular in business, is catching on in cities around the world. As I recently wrote, Memphis has reduced crimes such as robberies and rapes to their lowest levels in a quarter-century, thanks in part to IBM’s analytical software.
Why has this approach been so successful? Or put another way: Why did decision-makers, prior to data analytics, make such poor decisions? Part of that answer has to do with a kink in human reasoning, according Michael Lewis, author of “Moneyball,” which told the story of how the Oakland A’s turned to data analysis to find undervalued players passed over by baseball experts.
Lewis writes in the December issue of Vanity Fair that he did not know the answer to this question until he discovered the work of Nobel laureate Daniel Kahneman and his collaborator Amos Tversky. Both were psychologists and published work on judgment and decision-making.
“They had found that people, including experts, unwittingly use all sorts of irrelevant criteria in decision-making,” writes Lewis. This problem has to do with the way the brain accesses or remembers information. When making a decision, people tend to base it on examples that are easily remembered. But “reliable statistical evidence will outperform” that thinking, the article says.
Since this is a human problem, it affects not only baseball experts but also hedge fund managers, CEOs and even police officers.
In Memphis, police officers were able to get a better handle on crime patterns once they started using IBM’s data analytics software to figure out crime patterns and forecast where it was going to occur.
For example, Memphis police officers had long known that burglaries tended to spike in the month of March. But until the use of data analytics, nobody realized that the spike corresponded with Spring Break in Memphis-area schools. Once the police added more patrols during that week, the burglary rate dropped.
