It’s beginning to feel a lot like…

Last week I popped into a supermarket to pick up some groceries and turned into an aisle looking for bread only to be confronted by a wall of mince pies. More mince pies than any one could conceivably want and categories of fillings and pastry types I never even knew existed…

Jolly Christmas from the tarts, 1888 (ref: COPY 1/84 (375))

There was so much choice that, had I been in the mood, I’m not sure I’d have been able to decide. Where to start? Do I want puff pastry, short crust, lattice covered or a good old fashioned proper mince pie? Which brand? The store’s own brand might be cheaper, but what of the content? Would it be as good as the more expensive brands and would I, in all honesty, be able to taste the difference after the first box?

Then of course there’s other customer’s point of view that could be valuable. I could have stopped anyone walking past and asked them. Given it was 18:30 on a Friday the response rate and quality may well have been ‘varied’ to put it politely so I didn’t have that option.

What I needed was a way to compile the data about all the available pies (size, weight, contents etc) right there in the shop to help me classify the mince pies and determine if a ridiculously early Christmas treat would suit my current food mood.

This is the basis of an increasingly popular process called ‘Big Data Analysis’. In very simple terms this is a process of trying to classify, then make available, a large collection of a business’ information assets to find meaning and insights from them that might otherwise not have been possible. This is may be because the  information assets exist in many different locations or repositories. It may also be because their differing content (word process and spreadsheets for example) makes it hard for a user to easily identify a relationship between them.

Providing an enterprise wide solution for classifying information in a business has huge potential, especially as the tools are getting smarter. As with all things, however, they do come with a couple of caveats.

• The system requires training which takes time and effort on the part of the whole business
• The system requires reliable metadata so that it can provide an accurate understanding of all the information assets
• The business has to own and understand the process so that in future they can understand how and why the process was done at all

Done properly, big data analysis can provide provide appropriate content to users when and where they need it. But it isn’t a magic bullet and requires an investment in time, effort and money to make sure it’s done right and not end up with one big bucket marked ‘mince pies’.