InfoQ Homepage Database Content on InfoQ
-
/articles/jit-analytics-architecture/en/smallimage/designing-jit-analytics-architecture-for-scale-and-reuse-small-1687790095857.jpg)
Designing the Jit Analytics Architecture for Scale and Reuse
As a SaaS provider, analytical data at Jit needs to be useful to both their customers and to internal stakeholders. AWS services including EventBridge, Kinesis Data Firehose, and Timestream handle data ingestion and UI platforms from Mixpanel and Segment provide data visualization.
-
/articles/analytical-data-management-duckdb/en/smallimage/in-process-analytical-data-management-with-DuckDB-small-1686236227723.jpg)
In-Process Analytical Data Management with DuckDB
DuckDB is an open-source OLAP database for analytical data management that operates as an in-process database, avoiding data transfer overhead. Leveraging vectorized query processing and Morsel-Driven parallelism, the database optimizes performances and multi-core utilization for analytical data processing.
-
/articles/debugging-beneath-trusted-abstraction/en/smallimage/debugging-outside-comfort-zone-diving-beneath-trusted-abstraction-small-1685961994992.jpg)
Debugging outside Your Comfort Zone: Diving beneath a Trusted Abstraction
This article takes a deep dive through a complex outage in the main database cluster of a payments company. We’ll focus on the aftermath of the incident - the process of understanding what went wrong, recreating the outage in a test cluster, and coming up with a way to stop it from happening again, and dive deep into the internals of Postgres, and learn about how it stores data on disk.
-
/articles/impact-machine-learning-climate/en/smallimage/minimising-the-impact-of-machine-learning-on-our-climate-small-1685091723283.jpg)
Minimising the Impact of Machine Learning on our Climate
This article introduces the field of green software engineering, showing the Green Software Foundation’s Software Carbon Intensity Specification, which is used to estimate the carbon footprint of software, and discusses ideas on how to make machine learning greener. It aims to give you the tools to take an active part in the climate solution.
-
/articles/dropbox-magic-pocket-exabyte-storage/en/smallimage/magic-box-dropbox-small-1682511216986.jpg)
Magic Pocket: Dropbox’s Exabyte-Scale Blob Storage System
A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud.
-
/articles/cassandra-kubernetes-microservices/en/smallimage/banking-on-thousands-of-microservices-small-1683022320724.jpg)
Banking on Thousands of Microservices
Lessons learned building a banking platform, starting from technological choices like using Cassandra and Kubernetes in the early days to maintain the speed of execution through platform engineering and developer experience. With some mistakes and incidents along the way.
-
/articles/knime-correspondence-analysis/en/smallimage/logo-1677000367450.jpg)
Understanding and Applying Correspondence Analysis
Customer segments, personality profiles, social classes, and age generations are examples of effective references to larger groups of people sharing similar characteristics. Correspondence analysis (CA) is a multivariate analysis technique that projects categorical data into a numeric feature space which captures most of the variability in the data by fewer dimensions.
-
/articles/data-protection-federal/en/smallimage/logo-1673874259012.jpg)
Data Protection Methods for Federal Organizations and beyond
The Federal Data Strategy describes a plan to “accelerate the use of data to deliver on mission, serve the public, and steward resources while protecting security, privacy, and confidentiality." This article covers what it is and how it can be applied to any organization.
-
/articles/infoq-software-trends-report-2022/en/smallimage/InfoQ-Trend-Report-logo-small-1672911593669.jpg)
InfoQ Software Trends Report: Major Trends in 2022 and What to Watch for in 2023
2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022, and what to watch for in 2023. This article is a summary of the 2022 software trends podcast.
-
/articles/secure-kafka-cluster-strimzi/en/smallimage/securing-a-kafka-cluster-in-kubernetes-using-strimzi-small-1671647644310.jpg)
Securing a Kafka Cluster in Kubernetes Using Strimzi
Deploying an Apache Kafka cluster to Kubernetes is easy if you use Strimzi, but that’s only the first step; you need to secure the communication between Kafka and the consumers and producers, provide RBAC to access topics, spread the secrets correctly to Kafka Connect components and all using a Kubernetes GitOps way.
-
/articles/dynamoDB-data-transformation-safety/en/smallimage/logo-1669045832728.jpg)
DynamoDB Data Transformation Safety: from Manual Toil to Automated and Open Source
Data transformation remains a continuous challenge in engineering and built upon manual toil. The open source utility Dynamo Data Transform was built to simplify and build safety and guardrails into data transformation for DynamoDB based systems––built upon a robust manual framework that was then automated and open sourced. This article discusses the challenges with Data Transformation.
-
/articles/anatomy-code-obfuscation/en/smallimage/logo-1667849003893.jpg)
Who Moved My Code? An Anatomy of Code Obfuscation
In this article, we introduce the topic of code obfuscation, with emphasis on string obfuscation. Obfuscation is an important practice to protect source code by making it unintelligible. Obfuscation is often mistaken with encryption, but they are different concepts. In the article we will present a number of techniques and approaches used to obfuscate data in a program.