InfoQ Homepage Hadoop Content on InfoQ
-
Project Myriad: Mesos and YARN Working Together
An article by Jin Scott - A tale of two clusters: Mesos and YARN – describes hardware silos created by using different resource managers on different hardware clusters, most popular being Mesos and Yarn and introduces Myriad – a solution allowing to run a YARN cluster on Mesos.
-
Apache Hive 1.0 Released, HiveServer2 Becomes Main Engine, Stable API Defined
Apache Hive has released version 1.0 of their project on February 6th, 2015. Originally planned as version 0.14.1, the community voted to change the version numbering to 1.0.0 to reflect the amount of maturity the project has reached.
-
EMRFS Brings Consistency to Amazon S3
Amazon recently announced EMRFS, an implementation of HDFS that allows EMR clusters to use S3 with a stronger consistency model. When enabled, this new feature keeps track of operations performed on S3 and provides list consistency, delete consistency and read-after-write-consistency, for any cluster created with Amazon Machine Image (AMI) version 3.2.1 or greater.
-
Splice Machine Version 1.0 Supports Integration with Hadoop and Analytic Window Functions
Splice Machine version 1.0 supports analytic window functions and integration with Hadoop ecosystem. Splice Machine team recently released their Hadoop based RDBMS data management solution that can be used for transactional workloads on Hadoop.
-
LinkedIn Open Sources Cubert With an Eye To Big Data Analytics
LinkedIn recently open sourced Cubert, its High Performance Computation Engine for Complex Big Data Analytics. Cubert is a framework written for analysts and data scientists in mind.Developed completely in Java and expressed as a scripting language, Cubert is designed for complex joins and aggregations that frequently arise in the reporting world.
-
Gobblin, LinkedIn's Unified Data Ingestion Platform
At the 2014 QCon San Francisco conference, LinkedIn's Lin Qiao gave a talk on their Gobblin project (also summarized in a blog post) that is a unified data ingestion system for their internal and external data sources.
-
Stripe Open Sources Tools For Apache Hadoop
Stripe, the internet payments infrastructure company recently announced open sourcing a set of internally developed tools based on Apache Hadoop.Timberlake, Brushfire, Sequins and Herringbone all contribute to enriching the available tools for building an Apache Hadoop stack.
-
Spark Sets New Record in Sort Performance
Databricks has recently announced a new record in the Daytona GraySort contest using the Spark processing engine. The Daytona GraySort contest is a 3rd party benchmark measuring how fast a system can sort 100 Terabytes of data. Databricks posted a throughput of 4.27 TB/min over a cluster of 206 machines for their official run.
-
Hortonworks Data Platform Makes an Enterprise Push
Hortonworks Data Platform (HDP) version 2.2 with features based around Hadoop and YARN has better support for enterprise features such as security, compliance and so on as well.
-
Microsoft Expands Azure Machine Learning and Real Time Analytics Offering
Microsoft recently announced new machine learning capabilities for Microsoft Azure platform. Developers can also create their own web services and publish them to Azure Marketplace. Microsoft also announced availability of Apache Storm for Azure. Azure Stream Analytics, Data Factory and Event Hubs for Azure were all announced in the past few weeks by Microsoft. In this article we explore moreabout
-
Big Data Analytics: Using Hunk with Hadoop and Elastic MapReduce
Hunk is a relatively new product from Splunk for exploring and visualizing Hadoop and other NoSQL data stores. New in this release is support for Amazon’s Elastic MapReduce.
-
Apache Drill Included in MapR Latest Distribution Release
MapR recently announced including Apache Drill in its latest release of MapR distribution. Apache Drill is the open source version of Google’s Dremel. Dremel is the infrastructure on which BigQuery is based upon. Drill is offering a low latency SQL-on-Hadoop interface. While this puts it in the same space as several other technologies around Hadoop, Drill has some unique characteristics setting it
-
Hortonworks Announces Stinger.next Roadmap to Deliver Hadoop Scale SQL with Apache Hive
Following on from the Stinger initiative delivered in Apache Hive 0.13, Hortonworks has laid out the Stinger.next roadmap to provide fully ACID transactions, a sub-second query engine, and more complete SQL 2011 analytics support, all driving towards the goal of “enhancing the speed, scale and breadth of SQL support” in Hive.
-
Data Encryption in Apache Hadoop with Project Rhino - Q&A with Steven Ross
Cloudera recently released an update over Project Rhino and data at-rest encryption in Apache Hadoop. Project Rhino is an effort of Cloudera, Intel and Hadoop community to bring a comprehensive security framework for data protection. InfoQ recently talked to Steven Ross from Cloudera team to learn more about the project.
-
Cloudbreak, New Hadoop as a Service API, Enters Open Beta
Cloudbreak, a new open-source and cloud-agnostic Hadoop as a Service API, is now open for beta access to application developers and enterprises. SequenceIQ, Cloudbreak's maker, claims that its freely available product will make it easier to manage and monitor on-demand Hadoop clusters while also abstracting their provisioning.