Man vs machine: Always consider context when using big data

Man vs machine: Always consider context when using big data
Gil Allouche is the Vice President of Marketing at Qubole. Gil began his marketing career as a product strategist at SAP while earning his MBA at Babson College and is a former software engineer.

Technology is a marvelous thing. There is more computing power in your pocket right now than was used to land men on the moon. Advanced computer and software technologies can predict natural disasters, stop crime and even drive driverless cars. Artificial intelligence is a powerful reality, and not just a comic book plotline. With all of these wonderful technological advances, there is an inescapable conclusion: technology coupled with raw, unbiased data offers superior solutions to human judgement.

Yes, the interpretation of data offers insight into overall trends and tendencies that can’t be accounted for by a person. However, pure data by itself can never be used without context. Here is why.

The Misuse of Big Data

Forbes recently released a compelling article exploring how big data deceived the U.S. in the Vietnam War. Even then people saw the tremendous potential in using numbers to make decisions. Numbers were superior to decisions made by humans because humans are biased and faulted in judgment. So, the U.S. war strategy played to the numbers. Success was measured by the amount of bombs dropped on the enemy, land area controlled by American forces and the ratio of enemy casualties to American casualties. According to those metrics, America was winning the war across the board. Strategies were conceptualized and executed accordingly. In a sense, the lives of thousands of soldiers rested with a spreadsheet.

An Incomplete Story

As the article suggests, the numbers couldn’t tell the entire story. Like the British two centuries before, the U.S. war department didn’t account for external variables that weren’t quantifiable. The war department ignored the guerrilla strategy of the Viet Cong and it ignored the waning support of the war on the home front. Ultimately, the last helicopter pulled out of Vietnam with bewildered military strategists scratching their heads and wondering what went wrong. The numbers after all had spoken for themselves.

Vietnam is a compelling example of where big data can go terribly wrong. The danger isn’t inherently in the data itself, but the use of the data without broader context. In Vietnam the data didn’t take into account factors as simple as culture and what motivated enemy forces. For your business, decisions should never tied to crunching the numbers.

The need for context doesn’t stop at just decision making. There is tremendous evidence that data left by itself cannot compensate for human social intelligence. A computer does not have the social know-how to fully understand all aspects of human communication.

Take the example of sarcasm. A word at its face derives meaning from what it is associated with. A chair is something you sit on and a book is something you read. However, things like tone of voice or facial expression also alter the meaning of words.

Even now programs are designed to read content and predict sentiment of the writer. Analytics programs use key words or phrases to determine if a certain passage has a happy, sad, or indifferent tone. But how effective can this really be? If I use the word ‘great’ to describe how I’m doing, what is my emotion? Analytics would say positive, but what if I’m rolling my eyes as I say it? The sentiment being expressed is actually negative, a difference that is impossible to determine without proper context (eyes rolling).

Tips to Giving Your Data Context

Without context data drives many respectable and intelligent institutions to make irrational decisions. To avoid this pitfall it is important to use holistic data management tools like Hadoop as a service to make fully informed decisions.

Below are three simple ways to give your company data context:

1) Look Past the Numbers

Don’t settle for the obvious. Are there any other implications? How would other situations change the data? This is difficult for companies in unexplored territory. Companies like Blockbuster hold too fast to the numbers and ignore obvious trends that emerge (online streaming). The result is extinction.

2) Get an Outside Perspective

Consultants exist for a reason. Business professionals get so embedded in a product or business that it is difficult for them to see anything outside of the company. Numbers may be unbiased, but outsiders can be too. Collaborate with outside professionals and get fresh insights into business data. The solutions that result will be dynamic and innovative.

3) Numbers Should Supplement Decision Making

Notice the word supplement. A supplement cannot replace, just like vitamins can’t replace eating. Use data to supplement business decisions without overpowering the decision making process. Data should be used in full partnership with broader understanding.

Technology has improved exponentially in recent decades as has the ability to gather information and interpret it in useful ways. Big data promises to change the world, and it already has to a large degree. Data-driven decisions without proper context often fall short of reality. Remember, like any tool big data should be applied correctly. 

 

https://www.iottechexpo.com/northamerica/wp-content/uploads/2018/09/all-events-dark-text.pngInterested in hearing industry leaders discuss subjects like this and sharing their use-cases? Attend the co-located IoT Tech Expo, Blockchain Expo, AI & Big Data Expo and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam and explore the future of enterprise technology.

View Comments
Leave a comment

Leave a Reply

Your email address will not be published.