Questions Surround Hadoop's Staying Power
November 28, 2012
By Mae Kowalke
, TMCnet Contributor
Is Hadoop like Facebook (News - Alert) or Friendster?
“Will the ultimate realization of the powerful ideas it has introduced to the world be implemented in the form that Hadoop has created, or will a new way of making those ideas work emerge?” asked Dan Woods in a recent article for Forbes.
Data management framework Apache Hadoop has proven its viability, but it hasn’t yet shown that it will win against competing solutions such as Google (News - Alert) Big Query, Splunk, MPP SQL such as EMC Greenplum, Teradata Aster, Sybase IQ and NoSQL databases such as Couchbase and Basho Riak.
There are three open questions about Hadoop, according to Woods.
First, “Hadoop is neither a community-based open source project like Linux or Drupal nor a commercial open core company like Alfresco, or JBoss. Instead it is a strange hybrid that has some significant disadvantages.”
He described the ecosystem as a “three-headed open core” run by Cloudera, Hortonworks and MapR Technologies.
Can this model succeed?
“The problem is that the Hadoop community, even with the mature and excellent Apache process, is far from a smooth running process for governance, something which is hard to create in the best of conditions,” wrote Woods.
Second, is Hadoop evolving fast enough?
Earlier this year Facebook modified Hadoop because it just couldn’t handle the firm’s massive data sets, and Hadoop has many other areas where it needs to improve beyond just the area where Facebook tweaked it, MapReduce.
“Every open source project and every commercial product must evolve,” noted Woods. “But the need for Hadoop to evolve is perhaps more urgent because it was not created to meet the needs of the market it now finds itself in. Hadoop was created to help crunch massive web crawls into indexes to support searches.”
Third, is the funding mix right for Hadoop?
“Hadoop development is too small and needs to be supplemented by companies who are benefitting from Hadoop,” wrote Woods. “It is easy to see how the Three Headed Open Core model is diluting investment in Hadoop. The three companies are competing with each other, sometimes independently developing the same capabilities (such as management tools) and so on.”
All this is true to an extent, responds one head of the “three-headed open core,” MapR. But in its blog last week, MapR said these concerns should not be a concern for businesses. It doesn’t matter where Hadoop starts, it matters where Hadoop ends by the time it reaches the customer.
“The only decision customers have to make is what are their highest priorities in approaching a big data project,” wrote MapR. “MapR has chosen to bring what we believe is the most enterprise-ready version of Hadoop to market. This has taken significant work and investment in adding value to Hadoop where it matters most.”
Maybe Hadoop is not ready for every enterprise out of the box, but with partners such as MapR it is certainly a winner when it reaches the actual production servers.
Want to learn more about the latest in communications and technology? Then be sure to attend ITEXPO Miami 2013, Jan 29- Feb. 1 in Miami, Florida. Stay in touch with everything happening at ITEXPO (News - Alert). Follow us on Twitter.
Edited by Rachel Ramsey