Earlier this week I attended an excellent conference “Big Data: an Innovation Opportunity for Complex Services?” (www.cambridgeservicealliance.org/serviceweek/industry-conference.html).

Several important messages emerged.

In handling big data, it’s all about building the capacity to allow users to ‘have a dialogue with the data’.  The key to success is to allow queries with fast responses, so allowing users to refine their searches and their underlying hypotheses, to explore new directions driven by insights derived from the data itself.  Yes, there will remain the sophisticated analyses that do not fit this pattern of exploration, but the new world will be about interaction, not the creation of ‘an offline report’.  Hence the significance of the massive investments necessary to build the tools and the storage necessary to cope with big data. [1]

This bears on the debate about which comes first – the business model or the data?  Do you dream up a service and then acquire the data, or do you collect lots of data and figure out what to do with it?  In practice it seems to be both – iteratively and using prototypes to learn all the way along the line.

But there was considerable concern from many quarters about the emergent power of the combination of big data sets – of which loss of anonymity is only one dimension.

Matt McNeil provided an interesting description of Google’s layered approach to managing data and to having the tools to make the data manipulation scaleable and invisible.  It’s this that enables their developers to work at the right level of innovation while not getting bogged down in implementation detail.  And this is surely the basis of their capacity to offer web services in managing and working with big data (such as BigQuery).  But he also clarified that, while Google will collaborate with the providers of professional services, it’s an area they won’t enter themselves.  They’ll remain a tool provider.

Interesting to hear of wind turbine manufacturer Vestas’ use of data, coupled with big simulations to optimally site turbines and to predict their outputs.  It’s not about the cost of energy, it’s about the certainty of the investment business case when dealing with something as variable as the wind.  So they differentiate themselves by using their tools and data to offer this greater certainty to their customers.  A lovely example of service innovation, of business model innovation and of the use of big data.

But I was surprised that nobody brought up the ‘internet of things’.  Surely this will unleash a tsunami of data, and so of opportunities (and problems).  And as the internet of things develops we’ll see some of the most ubiquitous and engaging of services – most of which are not yet dreamed of.


[1] Google’s own research demonstrates that users conduct fewer searches if subject to delays of as little as 200ms, and that the effect lasts over a matter of weeks (admittedly, very small effects, but scaled over Google’s user base it could be significant. [http://googleresearch.blogspot.co.uk/2009/06/speed-matters.html].  So response rate matters.