InfoQ Homepage Data Management Content on InfoQ
-
/articles/infoq-software-trends-report-2022/en/smallimage/InfoQ-Trend-Report-logo-small-1672911593669.jpg)
InfoQ Software Trends Report: Major Trends in 2022 and What to Watch for in 2023
2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022, and what to watch for in 2023. This article is a summary of the 2022 software trends podcast.
-
/articles/oracle-to-mysql-aws-database-migration-service/en/smallimage/logo-1662492103321.jpg)
How to Migrate an Oracle Database to MySQL Using AWS Database Migration Service
Data migration efforts are typically taken up for database consolidation, cost considerations, or migrating on-prem databases to a cloud platform. In this article, author Deepak Vohra discusses the details of migrating a local database to MySQL database on the cloud, using AWS Database Migration Service.
-
/articles/business-system-integration/en/smallimage/business-systems-integration-is-about-to-get-a-whole-lot-easier-small-1660901081551.jpg)
Business Systems Integration is about to Get a Whole Lot Easier
A new breed of integration software is arising that syncs business data into a simplified data hub and then syncs that data to the destination system. The benefit of this integration pattern is that it reduces the number of manual transformations required (often to zero) and makes it easier to write manual transformations when you have to.
-
/articles/raft-engine-tikv-database/en/smallimage/logo-1652268636107.jpg)
Raft Engine: a Log-Structured Embedded Storage Engine for Multi-Raft Logs in TiKV
In this article, authors discuss the design and implementation of Raft Engine, a log-structured embedded storage engine introduced in TiDB distributed, NewSQL database version 5.4. They also discuss the performance benefits of the engine compared to the previous implementation based on RocksDB.
-
/articles/field-level-lineage-modern-data-systems/en/smallimage/logo-1646136031866.jpg)
Building End-to-End Field Level Lineage for Modern Data Systems
In this article, the authors discuss the data lineage as a critical component of data pipeline root cause and impact analysis workflow, and how automating lineage creation and abstracting metadata to field-level helps with the root cause analysis efforts.
-
/articles/data-application-process/en/smallimage/logo-1638797109490.jpg)
You’re Doing it Wrong: it’s Not about Data and Applications – It’s about Processes
Classic developer thinking tends to approach application design from a data-centric point of view. When the domain is process management, that often leads to excess complexity and work; it also (wrongly) over-reduces proactive processes to quick bursts of automation triggered by data changes. There’s a better way to do this: start with the process.
-
/articles/serverless-data-api/en/smallimage/logo-1619710774536.jpg)
Why a Serverless Data API Might Be Your Next Database
In this article, author Pieter Humphrey discussed database as a service (DBaaS) and serverless data API for cloud based data management.
-
/articles/future-data-engineering-riccomini/en/smallimage/the-future-of-data-engineering-logo-small-1613120173588.jpg)
The Future of Data Engineering
Chris Riccomini examines the current and future states of the art in data pipelines, data streaming, and data warehousing. He presents a six-stage evolution that data ecosystems follow, from a simple monolith to a complex data-microwarehouse architecture as the data engineers who manage them solve problems and clarify their roles as infrastructure engineers, rather than data stewards.
-
/articles/evolution-precomputation-technology-data-analytics/en/smallimage/the-evolutionof-precomputation-technology-small-1612883163091.jpg)
The Evolution of Precomputation Technology and its Role in Data Analytics
In this article, author Yang Li discusses the importance of precomputation techniques in databases, OLAP and data cubes, and some of the trends in using precomputation in big data analytics.
-
/articles/database-audit-system-kafka/en/smallimage/Building-a-SQL-Database-Audit-System-Using-Maxwell-logo-small-1609154420148.jpg)
Building an SQL Database Audit System Using Kafka, MongoDB and Maxwell's Daemon
In this article, the author discusses the importance of a database audit logging system outside of traditional built-in data replication, using technologies like Kafka, MongoDB, and Maxwell's Daemon.
-
/articles/data-leadership-book-review-interview/en/smallimage/Data-leadership-s-1595453339836.jpg)
Data Leadership Book Review and Interview
Data Leadership book, authored by Anthony Algmin, covers the data leadership topic and how data leaders should manage and govern the data management programs in their organizations. Data Leadership is how organizations choose to apply their energy and resources toward creating data capabilities to influence their business.
-
/articles/data-driven-privacy-architecture/en/smallimage/Privacy-Architecture-for-Data-Driven-Innovation-logo-1580376548756.jpg)
Privacy Architecture for Data-Driven Innovation
This article lays out how you build an internal data governance architecture early in the ingestion phase, which enables you to allocate risk to data and identify such data in your systems. You can then protect the data accordingly. The second half of this article lays out various techniques to share data in a privacy-conscious manner.