The key to unlocking big data’s true value: human beings

The bigger data gets, the greater need for human interpretation.
The bigger data gets, the greater need for human interpretation.
Image: Kirn Vintage Stock/Corbis
We may earn a commission from links on this page.

Intel has served as underwriter for a series of Quartz roundtable discussions with leaders from the financial sector on the impact of big data on their businesses. This BULLETIN is inspired by those discussions.

A BNY Mellon strategy paper lays out “the transformational influence of big data on the 21st century global finance system” as the bank seeks to use rebuilt processes to “store, crunch, use, and deliver data” in new ways. Its CIO employs a staff of 13,000 technologists and is looking for a managing director of big data architecture. Other financial institutions are on similar missions to keep pace, at the very least.

As the financial sector and other industries are keenly aware of big data’s promise and search for optimal ways to unlock its value, a surprising insight emerges from the current dialog: the critical factor in gaining value from powerful new technologies for making business decisions based on troves of data may be found in a much older tool: human analysis.

Many sources, new possibilities

Big Data promises enormous value for the finance and investing spheres, as it does for other sectors. Big Data in finance even has the potential to help make a more stable world for everyone. Fertile areas for Big Data analysis range from capturing new data streams to finding better ways to look at existing data.

In the newer realm is social media, which through big data methods can allow businesses to hear the voice of consumers in ways that can be quantified. Nimble startups have introduced products for organizations that mine the social web. In an early move that implicitly legitimized the value of data from social networks for investors, securities regulator FINRA in 2010 gave guidance to investment firms on appropriate social media rules and disclosures.

However, some big data practitioners say, businesses still must filter that voice. The potential for conclusions to be affected by inaccurate, delayed, or misunderstood consumer data is real. Without expert human guidance and interpretation, the data may be little more than empty bits.

From news feeds to weather, using new tools to interpret more established data sets can also unlock business potential. Geospatial data can reveal occupancy rates in parking lots of big box retailers, helping businesses grasp consumer spending trends in advance of Commerce Department releases. With the expansion of the Internet of Things, geospatial data will become more valuable, delivering information from as many as 50 billion interconnected devices by 2020.

Still, in this example and others, the sheer scope of data can overwhelm the ability to fully analyze it. If a search isn’t narrowed in advance by human identification of the desired information, it will either skim too shallowly or reach only a small corner of a large field of data.

Don’t just ‘worship the black box’

Businesses can’t throw money at big data technology alone and expect miracles. As with any new technology, even transformative ones, the path to real value isn’t a cakewalk. Clearly, for optimal deployment of big data solutions, human experts with subject-matter expertise are needed to ensure the quality of the questions asked and hypotheses tested, and to use well-honed judgment to help filter out noise–not to mention applying sensitivity to social, legal, and policy issues.

The more information organizations reap, the more analysis they’ll require. Without the crucial collaboration between the mechanics of the tools and the judgment of humans, the data is just a bunch of useless terabytes and petabytes. That equation is simple enough. The calculations fall to a more complicated mix of bots and bipeds. The demands and rewards for the right balance will also be big.

This article is made possible by Intel; it is not written by and does not necessarily reflect the views of the Quartz editorial staff.