You just cannot be in business today without hearing the term "Big Data". Mind you, I blame the management consultants for producing yet another snazzy buzz phrase to tease their clients. To be clear, I am not against buzzwords just as long as they are relevant and convey the appropriate meaning. The "C-Suite", for example is pretty reasonable shorthand for the senior people in a company whose titles start with the letter "C" - Chairman, CEO, CFO, COO and so on. But this "Big Data" is leaving me somewhat cold by its ambiguity, and the real problem is that it has hit the C-Suite.
I heard a Chief Executive recently talking about the fact that he would in future have all the management information his company needed because they had embraced the idea of big data. He could save huge amounts of money because he would no longer need hordes of people like analysts in his back office, nor would he need to employ expensive management consultants. And because he had so much information he would be miles ahead of those companies that didn't understand big data. He was quite affronted when I suggested that big data might not be quite as much of a panacea as he was obviously thinking - and I spent the next 30 minutes discussing with him the merits - and otherwise, from my perspective! I hope I at least persuaded him that he might want to be asking quite a few more questions before he embraced it any further.
Don't get me wrong. I truly think that big data can be of enormous help to businesses - as long as it is understood and, even more importantly in my view, then used properly!
There are of course many people who know what big data is all about, (particularly at the working level) but for the less technically or business minded readers of my blog, let me just try an explanation.
Simply put, imagine the biggest reference library you have ever seen or can imagine, and multiply the size by at least ten times. Big data is like that. It is a collection of data so large that it becomes difficult to find what you are looking for by using traditional methods of database management. As a result, work that was previously being done by using analysts is now being handled through creating a series of algorithms that "interrogate" the data and spew out the information you need. Or you think you need. This, for the business community, could be used to spot business trends, seek out inefficiencies, and provide clues to current customer behaviour - leading perhaps to future trends (of course only as long as the conditions existing today remain unchanged!) Sounds good, doesn't it? And it is a big help.
So far, so good - but with the huge mass of information you are receiving who handles it, who "interrogates it", and who is then responsible for creating company strategy?
But there are a few basic things to remember here. Firstly, is all the information you think you need actually necessary to make some decisions, or are you just accumulating information that is of little real value? Secondly, who is creating the algorithms to interrogate the big data to provide the results? Until computers can think for themselves (and yes, I know it's coming), you will still need people to create the right algorithms. People who know the business, and therefore have the ability to ask the right "questions" Remember that old saying from Economics Nobel Prize Laureate, Ronald Coase (who by the way is 102 years old), who said "Torture the data, and it will confess to anything!" Quite!
From my point of view it is surely valid for consultants to be pushing big data as a concept to companies, but there is probably a better case to be made for those consultants who can successfully help in the creation of appropriate algorithms, in other words creating the right questions, and the subsequent interpretation of the answers. Heaven help many Boards of Directors if they are left to understand this stuff on their own. And yes - that includes me!
While we live in an increasingly technologically driven world, the users of that technology still only operate with human brains - which frankly have their limitations. So until we catch up, corporations need to realise that big data aside, much of the "little data" they have is still sufficiently powerful to enable them to do a better job for their clients, and therefore their businesses, than they are doing at present.
As one of my sons pointed out in a comment recently, how is it that a financial institution with whom he has had a relationship for 30 years still addresses him by mail as "Dear Valued Customer". They will have enough clues to realise that he is now married, since he has a joint account, should have an idea that he has children as they have accounts of their own, and with this should know - and be able to customise easily, suitable offers to him. We have the capability to do all this stuff today on "little data".
Other organisations do it - witness the slightly embarrassing story that came out of US retailer Target. (Do have a read - it's worth it, and it provides a lesson in what you can do today!)
So before we get hung up on big data, make a start by using more effectively the "little data" you already have.
But as for "Big Data"? - Sure, go for it and use it successfully but make darn sure those people in the Boardroom know what it's all about if they are being asked to commit millions of shareholder money in whatever currency into technology that might pay off - but might not if you don't use it properly!