Statistics and politics were the themes of a talk by openly gay statistician, blogger and author Nate Silver May 19 at the Spertus Institute for Jewish Learning and Leadership.
A New York Times best-selling author, Silver runs the website FiveThirtyEight.com which is featured on The New York Times website. Silver previously published his statistical analysis on the political blog the Daily Kos and prior to that developed a statistical method called PECOTA to analyze baseball statistics. He was named Out Magazine's "Person of the Year" in 2012 and in 2009 Time Magazine named Silver as one of "The World's 100 Most Influential People."
Spertus Institute CEO Hal Lewis and Chair of Spertus Institute's Board of Trustees Mark Mehlman provided words of welcome and spoke about the mission of the Spertus Institute. "We believe that a learning Jewish community is a vibrant Jewish community," said Lewis.
After an introduction by Steve Edwards, deputy programming director for the Institute of Politics at the University of Chicago (he works alongside political analyst David Axelrod who is the institute's director) and former host of WBEZ's news magazine Eight Forty-Eight, Silver spoke to over 400 people about "The Signal and the Noise: Why So Many Predictions Fail - But Some Don't" based on his book of the same name.
The focus of Silver's talk was why do we keep making mistakes with our predictions? Silver explained that the explosion of data in recent years has created problems since some of the data isn't really useful. Then Silver shared that with the invention of the printing press information was spread at a rapid pace and this created problems since differing ideas were shared across Europe. Silver noted that in the 100 years after the invention of the printing press there was an increase in the number of wars which led to that century becoming the bloodiest century across history until the 20th century.
An example of data problems, Silver explained, can be seen with flu statistics. Because of the way that Google measures data collection vs. the way that the Centers for Disease Control and Prevention (CDC) collects data there was a 4.3-percent difference in the way flu statistics were measured with the CDC showing less of a flu outbreak than Google.
Then Silver shared his data from the 2012 election and how he determined the winner of each state. The 538 method, Silver said, is to average the polls, count to 270 and account for the margin of error.
Silver asked when will big data produce big progress? Silver noted that there are three problems to contend with. The first problem is that big data can lead to big bias. "It helps if you have amnesia if you are a pundit" so you can claim that you are always correct, said Silver. The second problem is that people are desperately seeing a signal i.e. where is the unbiased correct information, Silver explained. The third problem, according to Silver, asks whether big data is a helpful feature or bug. Silver used GPS as an example because it is usually a helpful device but sometimes it goes wrong.
Another thing that Silver pointed out is the signal-to-noise ratio. Silver said that as more data (i.e., signals) is collected the incidents of inaccurate information or information based solely on opinions increases (i.e., noise).
Using population as an example, Silver said that when looking at the entire U.S. population 36 percent reside in blue states, 37 percent reside in swing states and 26 percent reside in red states. In looking at the Jewish population in particular, Silver noted that 68 percent reside in blue states, 26 percent reside in swing states and 6 percent reside in red states. Silver remarked that the only way that the Jewish population can have any influence in elections is to spread out to swing or red states.
Silver offered three suggestions to help produce better data results: think probabilistically meaning report all of the data for a particular situation; knowing where one is coming from when looking at specific data; and trial and error.
The values that are important to have when looking at data, according to Silver, are humility (which Silver said is the most important value), diversity, independence, respect, hard work, accountability and a love of learning.
During the Q&A, Silver was asked why people always want to know what will happen tomorrow, if there is a way to churn data to predict how much time we have left to live on earth, the ability to accurately assess primary races, using census data to predict results and medical data mining.
Silver was also asked the difference between the United Kingdom and the United States when it comes to death and he said people in the UK are skeptical about death whereas people in the United States have a curiosity about death. A question was posed about ways to improve assessing academic achievement and Silver said that college rankings are arbitrary. Someone asked if the United States is ready for Hillary 2016 and Silver said yes.
A reception and book signing took place following Silver's presentation and the Q&A session.
See fivethirtyeight.blogs.nytimes.com for more information.
