BT

InfoQ Software Architects' Newsletter

A monthly overview of things you need to know as an architect or aspiring architect.

View an example

We protect your privacy.

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Logo - Back to homepage

News Articles Presentations Podcasts Guides

Topics

Development

Featured in Development

Building Inclusive Mini Golf: a Practical Guide to Accessible XR Development

Colby Morgan discusses practical strategies and technical examples for building accessible and inclusive XR experiences. Learn about their core design principles, including accessibility at the start, invisible features, simplicity, and layered depth, using Walkabout Mini Golf as a case study.

All in development

Architecture & Design

Featured in Architecture & Design

Comprehensive Approaches to Software Supply Chain Security

Mykhailo Brodskyi discusses critical software supply chain security risk categories relevant to FinTech and explains effective mitigation strategies. Drawing from real project case studies, he shares insights to protect systems from vulnerabilities and ensure platform security and resilience. Demo: third-party library risk mitigation through software bill of materials generation and analysis.

All in architecture-design

AI Infrastructure

Featured in AI, ML & Data Engineering

Responsible AI for FinTech

Lexy Kassan discusses the critical aspects of responsible AI, focusing on regulatory updates like the EU AI Act and its implications for FinTech. She explains the levels of responsible AI, from ethical principles to practical implementation, including AI governance. Explore FinTech's disruptive AI response, open-source use, workforce shifts, review boards & risk frameworks.

All in ai-ml-data-eng

Culture & Methods

Featured in Culture & Methods

Adam Sandman on Generative AI and the Future of Software Testing

In this podcast, Shane Hastie, Lead Editor for Culture & Methods, spoke to Adam Sandman about how generative AI is transforming software development and testing by automating mundane tasks, enabling faster prototyping, and collapsing traditional roles into broader generalist positions, while also highlighting challenges like increased defects and ethical concerns.

All in culture-methods

DevOps

Featured in DevOps

Checklist for Kubernetes in Production: Best Practices for SREs

This article provides SREs with a checklist for managing Kubernetes in production environments. It identifies common challenges including resource management, workload placement, high availability, health probes, storage, monitoring, and cost optimization. By implementing consistent GitOps automation across these areas, teams can significantly reduce complexity, and prevent downtime.

All in devops

Events

Helpful links

Choose your language

InfoQ Dev Summit Boston

Learn how senior software developers are solving the challenges you face. Register now with early bird tickets.

InfoQ Dev Summit Munich

Learn practical solutions to today's most pressing software challenges. Register now with early bird tickets.

QCon San Francisco

Explore insights, real-world best practices and solutions in software development & leadership. Register now.

QCon AI New York

Learn how leading engineering teams run AI in production-reliably, securely, and at scale. Register now.

InfoQ Homepage Data Pipelines Content on InfoQ

Presentations

RSS Feed

Newer Older

Architecture & Design

Streaming Reactive Systems & Data Pipes w. Squbs

Anil Gursel and Akara Sucharitakul focus on modeling and building software that considers all input and all output as stream of events, and introducing Squbs.

Akara Sucharitakul Anil Gursel
on May 04, 2018

Icon

39:48
Architecture & Design

Scaling Uber's Elasticsearch Clusters

Danny Yuan talks about how Uber scaled its Elasticsearch clusters as well as its ingestion pipelines for ingestions, queries, data storage, and operations by a three-person team.

Danny Yuan
on Apr 11, 2018

Icon

48:18
AI, ML & Data Engineering

Effective Data Pipelines: Data Mngmt from Chaos

Katharine Jarmul discusses implementation decisions for those looking for a practical recommendation on the "what" and "how" of data automation workflows.

Katharine Jarmul
on Mar 29, 2017

Icon

45:22
AI, ML & Data Engineering

Building Data Pipelines in Python

Marco Bonzanini discusses the process of building data pipelines and all the steps necessary to prepare data, focusing on data plumbing and going from prototype to production.

Marco Bonzanini
on Mar 28, 2017

Icon

48:49
AI, ML & Data Engineering

Cloud Native Streaming and Event-driven Microservices

Marius Bogoevici demonstrates how to create complex data processing pipelines that bridge the big data and enterprise integration together and how to orchestrate them with Spring Cloud Data Flow.

Marius Bogoevici
on Jan 14, 2017

Icon

01:10:46
AI, ML & Data Engineering

Spring and Big Data

Thomas Risberg discusses developing big data pipelines with Spring, focusing around the code needed and he also covers how to set up a test environment both locally and in the cloud.

Thomas Risberg
on Jan 08, 2017

Icon

55:24
AI, ML & Data Engineering

Data Microservices in the Cloud

Mark Pollack introduces Spring Cloud Data Flow enabling one to create pipelines for data ingestion, real-time analytics and data import/export, demoing apps that are deployed onto multiple runtimes.

Mark Pollack
on Jan 08, 2017

Icon

01:07:25
AI, ML & Data Engineering

Hydrator: Open Source, Code-Free Data Pipelines

Jonathan Gray introduces Hydrator, an open source framework and user interface for creating data lakes for building and managing data pipelines on Spark, MapReduce, Spark Streaming and Tigon.

Jonathan Gray
on Oct 23, 2016

Icon

41:39

Newer Presentations

Older Presentations

BT