Big data vendors Hortonworks, IBM, and Pivotal recently announced that their Hadoop based platform products will use the common Open Data Platform (ODP). They made the announcement at the recent HadoopSummit Europe Conference, of the open platform which includes Apache Hadoop 2.6 (HDFS, YARN, and MapReduce) and Apache Ambari software.
The ODP initiative is an industry effort focused on simplifying the adoption of Apache Hadoop for the enterprise, and enabling big data solutions. It will also provide a set of tools and methods for the group members to create and test their offerings based on the ODP Core platform.
Hortonworks Data Platform 2.2, IBM Open Platform 4.0 with Apache Hadoop, and Pivotal HD 3.0 are all based on the common ODP core. Other members of ODP include GE, Infosys, SAS, and Altiscale, Capgemini, CenturyLink, EMC, PLDT, Splunk, Teradata, Verizon, VMware and WANdisco.
Hortonworks Data Platform 2.2 with Apache Ambari 2.0 and the latest Hortonworks Sandbox are available now for download from their website.
Hortonworks also announced a series of cloud deployment, operations, security, data governance and user experience features for using Apache Hadoop in the enterprises. These features include the following:
- Automated cloud provisioning and elastic scaling
- Automated rolling upgrades
- Centralized management of health alerts and metrics using Ambari 2.0
- Visual query performance analysis tools: Ambari 2.0 introduces new user views for visually analyzing and tuning the performance of Hive and Tez applications.
- Apache Spark integration: Apache Spark 1.2.1 is now included with HDP v2.2, providing application developers a platform for deploying Spark applications on-premises or in the cloud.
- New Apache project proposal for data governance: The proposed project is called Apache Atlas that aims to provide data classification, centralized auditing, search and lineage capabilities paired with a security and policy engine.
In a related news, Hortonworks has recently signed a definitive agreement to acquire the open source provider of Hadoop deployment tools SequenceIQ. It can be used to launch on-demand Hadoop clusters in the cloud (including Microsoft Azure, Amazon Web Services, Google Cloud Platform and OpenStack) or to any environment that supports Docker containers. SequenceIQ optimizes the deployments using a policy-based autoscaling approach that expands and contracts clusters based on actual usage metrics.
After the acquisition transaction is complete, Hortonworks will incorporate SequenceIQ technology into HDP and transition the technology to Apache Software Foundation. If you are interested in learning more about the latest features of SequenceIQ products Cloudbreak and Periscope, Hortonworks will be hosting a webinar on May 19 to provide more details on the acquisition.