Big Data and now Dark Data are all the rage, but Ian Jindal – himself a rabid advocate of data acquisition, quality and usage – pauses to ask whether we are creating a well-stocked store cupboard of ingredients without the recipe for a tasty dinner. Where is the ‘Big Insight’ to match Big Data?
One of the hottest topics in the executive inbox at the moment is Big Data. After the Big Dig, Big Lunch, Big Society and maybe, as a segue, Big Brother, the term Big Data fits into that amorphous family of Big Stuff.Yes, 90% of the world’s data may have been created in the last 10 minutes, and yes we’ll need a good few zettabytes just to store our own iPhone videos, and yes we’ll link every system known to retail into a humungous bucket of stuff… leaving us with the question of “so what?”.
Don’t mistake this challenge for a luddite, data-unfriendly position. Data has become evermore fundamental to retail due to three factors: being able to capture more data points than before; being able to share, aggregate and correlate these data points; and being able to analyse, synthesise and report upon the data.
Data points can reveal further insight when processed, mined and correlated, but we need to have care about our collection model.An insurance company assessing a haulage company’s risk may benefit from knowing the driver histories, their medical background and perhaps their ability to deal with stress and key events imminent in their lives.This previously-unavailable “Dark Data” is – like the analogous ‘Dark Matter’ in cosmology – by far the larger part of data, but hidden away out of our grasp.The hunt to identify, acquire and use Dark Data is therefore ‘on’.
However, we also need to consider carefully the mental models we create about data sources and their relevance – these will affect the data we seek to collect. Consider your average bacteria living happily in the toilet bowl of life. Based upon observed phenomena in their short lifetime, their folk knowledge to link a blinding light to a catastrophic flood at least once per generation (light on, then flush).According to myth, some of the deluvian episodes are preceded by a terrible, burning plague that destroys the majority of the population (a Domestos moment).This information will pass into bacterial mythology, but there’s no point asking them about who composed the music playing on the radio in another room, or the artist’s cumulative sales in Japan that year. It’s not only beyond their comprehension, it’s of value neither to the bacteria, nor the radio station nor the Japanese music fan.
One could see data acquired at such a stretch as being “Anti-data”, where the cost of collection and management exceeds the discovery value.
Anti-data can also be formed as a result of incorrect assumptions.Are large numbers of clicks/visits a result of great engagement or a customer who’s lost? Is a long dwell time a compliment to our riveting content or a sign that the site is impenetrable and the customer confused? Either
are possible, but require additional data before we can know. One approach is to collect all data possible, another is to gain qualitative insights into the customer’s behaviour and then selectively seek correlating data points.
Savvy retailers – never known to waste a penny – will balance the cost of ‘everything’ with the cost of knowing nothing and look to find a profitable medium. Much as a small amount of anti-matter exists in our material world we can welcome some anti-data as a source of illogical leaps and flashes.We can also tolerate some anti-data in the mix without spoiling the directionally- correct value of indications.
As we look outside of our business’ owned data points and seek to appropriate, acquire and use other sources we need to develop insight approaches that are exploratory, revelatory, behavioural and personal. Walking in our customers’ shoes, understanding their motivations, qualitatively exploring their perceptions of our brand and proposition – these ‘softer’ approaches will give us a human contact with customers and allow us to direct our journey through the world of Dark Data to the areas of most benefit and relevance, avoiding catastrophic brushes with Anti-data along the way.
Meanwhile, there will be someone measuring whether the antibacterial efficacy of a detergent is linked to Beethoven violin sonatas on Radio 3.
