Big Data - Means to an End

To say “Implement Big Data” is similar to saying “Implement the Internet”. Though the Internet is typically referred to as a thing or a place, it is actually a concept of interconnected machines. Whereas Big Data may be a term to mean high data volume, it is fast becoming a concept in reference to managing it. Therefore, it would be more sensible to say “Take advantage of Big Data” rather than “Implement Big Data”. And why does it matter? It matters because it is the difference between the “means” and the “end”.

So what?

Everyone would agree that Big Data offers many opportunities. The challenges in managing Big Data are also well established. The combination of technology to address these challenges has also been proven.

While the means in enabling Big Data is well understood, many still do not know what to do with it let alone justify for its investment. This could be due to the lack of vision and a valuable use case. Since the concept of Big Data is still relatively new, there are very few use cases that can be considered mainstream. Early adopters often have to build their own use cases ground up.

Which V comes first?

To build up a new use case a problem statement has to be established first. Problem statements can come in 3 forms: to innovate on old techniques to solve old problems; to create new techniques to solve new problems; or to look for simpler ways to solve complex problems.

You may start building problem statements with a vision and work backwards to figure out what data is required. Or you may do the reverse and start to look at what data is available and from that develop the value and vision.

What is the big idea?

Once the problem statement is well defined, the next step is to build up inspirational ideas to design the Big Data value and in turn describe the Big Data vision. Idea generation is best done collaboratively in groups. It is also a process of transforming observations into ideas that can be implemented. An example of this process is Design Thinking – a method that focusses on empathy.

Big Data to Big Value

Only when the vision is clear and when the value can be measured Big Data can be justified. Everything else would easily fall in place then – the data, the challenges, the technology, the target audience, the usage. Big data initiatives should rightfully be more about the justification than it is about the technology.