/minibooks/modern-data-engineering/en/cover/98.Data Engineering-COVER-1631713323680.jpg)
In the “good old” days of software development for Internet applications, data engineering was all about getting data from a transactional data store to a data warehouse, whereby it enabled business analysts to create reports and gain insights for the larger organisation and leadership team.
In this second edition of the Modern Data Engineering eMag, we’ll explore the ways in which data engineering has changed in the last few years. Data engineering has now become key to the success of products and companies. And new requirements breed new solutions.
Data and data engineering are being used to power mission critical production applications, create powerful network effects, predict user behavior, and much more.
Free download
The InfoQ eMag - Modern Data Engineering: Pipeline, APIs, and Storage include:
- Building Latency Sensitive User Facing Analytics via Apache Pinot - At QCon, a virtual conference for senior software engineers and architects covering the trends, Chinmay Soman talked about how you can use Apache Pinot as part of your data pipelines for building rich, external, or site-facing analytics.
- Why a Serverless Data API Might Be Your Next Database - In this article, author Pieter Humphrey discussed database as a service (DBaaS) and serverless data API for cloud based data management.
- Indestructible Storage in the Cloud with Apache Bookkeeper - At Salesforce, we required a storage system that could work with two kinds of streams, one stream for write-ahead logs and one for data. But we have competing requirements from both of the streams. Being the pioneers in cloud computing, we also required our storage system to be cloud-aware as the requirements of availability and durability are ever more increasing.
- Piercing the Fog: Observability Tools from the Future - Visibility into those distributed systems and how they are performing is challenging. Despite all the observability tools available for site reliability, debugging remains incredibly difficult, and many SREs would agree that their debugging processes have only marginally improved. This article explores how observability for troubleshooting could be done from the user’s point of view.
InfoQ eMags are professionally designed, downloadable collections of popular InfoQ content - articles, interviews, presentations, and research - covering the latest software development technologies, trends, and topics.