In a keynote address at the Enterprise 2.0 Conference in Boston, Andrew McAfee, Principal Research Scientist at MIT, explained how the rise of “Big Data” is not only changing the face of technology, but the face of human existence.

Big Data Dominates Silicon Valley

“There is one conversation in Silicon Valley, and it’s Big Data,” McAfee said. “Every startup and ambitious person in Silicon Valley is involved. Some people even say the world is one Big Data problem.”

While McAfee said the latter statement is “arrogant,” he did agree with the sentiment that with enough data, any problem can be addressed. “Google says, ‘We don’t have better algorithms than you, we have more data,’” he commented.

Science Fiction Becomes Reality

According to McAfee, social software is creating a new pool of data that can be fed into the Big Data “machine” to help “science fiction become reality" as computers continue to take on activities once considered the exclusive domain of humans.

“Computers keep getting better with remarkable speed,” said McAfee. He gave the example of the IBM Watson computer, which in November 2010 reached a level of accuracy and confidence in answering trial “Jeopardy” questions that McAfee described as equaling human “Jeopardy” champ Ken Jennings “with a head cold or hangover.” But when Watson faced Jennings for real during a three-night televised competition in February 2011, it beat him handily.

McAfee offered other examples of domains once thought to require the human touch that can now be performed as well or better by computers. These include writing financial reports and basic sports articles, analyzing photos of human tissue to detect and predict cancer, forecasting events including Supreme Court decisions and the reliability of suppliers, and even driving a car. In fact, McAfee said that comparisons of machine and human performance prove “we were never all that good.”

In an analysis of 136 studies comparing the performance of machines and humans in a wide variety of tasks, McAfee said humans won in eight of them, for a “batting average” of .059. “The likely explanation for those eight human victories is that the machines didn’t have enough data,” he said. “That won’t be a problem anymore.”

It's Time to Race with Machines

McAfee said it is understandable for people to find the prospect of computers taking over many human functions scary, but said the worst things we can do are to do nothing or to “smash the machines.” Instead, he advised people to “race with the machines” to obtain the greatest advantage from the power of computers while also retaining the innate advantages of the human brain.

“Competitions between machines and people in chess have not been interesting for the past 10-20 years,” he stated. “But there are ‘freestyle’ chess competitions where teams can combine computers and people to compete. Two amateurs won a competition using a predictive machine.”

In another example, McAfee said studies have shown that employees who actively participate in Enterprise 2.0 corporate social networks are more likely to avoid layoffs than employees who are not socially active in this manner.