What can Big Data learn from the Land of Oz?
Picture this… Dorothy, from the Wizard of Oz, fantasizing about her return to Kansas, desperately clicking the heels of her sparkly red shoes chanting “there’s no place like… SQL”!
OK, so Dorothy may not be how you might picture your typical DBAs, data analysts, and data warehouse specialists, but this desperate need for the familiar is not uncommon in many organisations today with their legacy infrastructures and heritage data teams. The men and women in these roles may have been in IT for two decades or more and, while they are more than comfortable with traditional relational databases, this new world of unstructured data and non-relational databases must seem as alien to them as the Land of Oz.
But, as the discussion at a recent breakfast briefing on the topic of big data - hosted by enterprise IT with guest speakers from Cloudera and Oracle - that I recently attended revealed, it doesn’t have to be that way. Just as Dorothy had help navigating in the wizard of Oz, so too do data specialist teams have help in simplifying the complexities of Big Data.
I talked to Devin Deen, the newly appointed director of the Analytics practice at enterprise IT about some of the key points that came out of the event.
Q: The wizard of Oz image you painted in your presentation, and comment about how ‘courageous organisations are getting smarter, faster by putting Big Data to work’, made me think about the film’s cowardly lion. What do you think is helping give some organisations courage?
New tools give courage like the elixir the wizard gives the lion to drink.
The hadoop world is primarily open source and doesn’t have as mature tools as the relational world and its somewhat more fragmented. What that means is that you have to stitch many components together to get data in and out. That said, increasingly there are organisations and tools coming on to the market that help with this, and while not quite as good as the elixir that the wizard gave the cowardly lion to drink, they go some way to giving organisations the courage to move forward more quickly and get faster value from their big data activities. For example, commercially supported distributions of Apache Hadoop like Cloudera do the stitching for us as well as providing management and administrative utilities to productionise Big Data deployments. Likewise, Oracle has a range of tools within its Big Data architecture and a number of proved use cases that enable companies to start small with what they need, and can also be scaled up and out in proportion to the rate that Big Data returns value to the organisation.
Q: Is it all about the tools?
No. It is as much about your mindset and your readiness to take advantage of big data as it is the tools.
Asim Khan, now an Enterprise Architect with Oracle NZ, gave a great example of this based on an experience when he was with Oracle’s America’s Cup syndicate and Americas Cup HQ as a performance analyst, Director of Information Systems and Director of IT, respectively. In the 2013 AC campaign, Asim led his technology team to crunch the data to help give Oracle’s team the edge they needed in what has been called the greatest come back in sports history.
The thing is, that while Team Oracle got into big data and doing analytics on the data collected from GPS enabled personal wearable devices on the crewman, cameras on the boat, cameras on chase boats and telemetry information from the boat itself, it wasn’t until the team pitch poled one of its AC72s on 12 October that it had time to understand what events contributed to the capsize. The forced pause in training in the water gave Team Oracle an opportunity to review the data they were collecting, discover the insights that it could provide to the team, and then bake-in data collection, reporting and analytics into making small modifications to “17” daily during racing and to eventually come behind and edge out Emirates Team NZ for the Americas Cup title in 2013.
I am not saying that you have to wait until you have been pitch poled to think, but you do have to approach big data rationally and really understand your readiness for it and from this, select the right use cases.
So like big data tools are to the business as courage is to the Lion, so is mindset to the organisation like the brain is to the Scarecrow.
Q: So don’t tell me you can bring in the tin man too!
Of course! What you really need and want is top down support. So for anyone embarking on a big data project, you need to make sure that you get the love and backing of a business sponsor before you spark the heartbeat in your project.
This point was driven home by a story told by one of the breakfast’s attendees and the winner of our “Big Data sob story” competition. They told the tale of taking insights gained from a Big Data experiment done by his team did in isolation to a business executive, whose support was key in gaining further funding and commitment to continue, but whose support hadn’t been sought previously. Regardless of the insight gained, and time and investment spent, the executive’s response was at best ho-hum. So the moral is definitely, don’t expect that just because you build something cool, business users will flock to use it.
Q: So, Devin tell me how can organisations start putting Big Data to work?
I’d start with a Big Data readiness assessment to help select the right use case(s) and definitely after having secured a supportive business champion. Then, I’d do a small proof of concept to validate business value in the project, followed by a pilot project using technology and business processes that you can scale up and out and make production ready.
Basically, don’t forget to call on Dorothy’s guides, the lion, the scarecrow and the tin man who helped her gain familiarity and simplicity in the uncertain and complex Land of Oz.