InfoQ Homepage AI, ML & Data Engineering Content on InfoQ
-
/articles/ai-ml-data-engineering-trends-2025/en/smallimage/the-infoq-trends-report-500x500-logo-1758095546194.jpeg)
InfoQ AI, ML and Data Engineering Trends Report - 2025
This InfoQ Trends Report offers readers a comprehensive overview of emerging trends and technologies in the areas of AI, ML, and Data Engineering. This report summarizes the InfoQ editorial team’s and external guests' view on the current trends in AI and ML technologies and what to look out for in the next 12 months.
-
/articles/architecting-rag-pipeline/en/smallimage/architecting-rag-pipeline-thumbnail-1755593892021.jpg)
Effective Practices for Architecting a RAG Pipeline
Hybrid search, smart chunking, and domain-aware indexing are key to building effective RAG pipelines. Context window limits and prompt quality critically affect LLM response accuracy. This article provides lessons learned from setting up a RAG pipeline.
-
/articles/causal-reasoning-observability/en/smallimage/causal-reasoning-observability-thumbnail-1756372623637.jpg)
How Causal Reasoning Addresses the Limitations of LLMs in Observability
Large language models excel at converting observability telemetry into clear summaries but struggle with accurate root cause analysis in distributed systems. LLMs often hallucinate explanations and confuse symptoms with causes. This article suggests how causal reasoning models with Bayesian inference offer more reliable incident diagnosis.
-
/articles/mcp-connector-for-building-smarter-modular-ai-agents/en/smallimage/mcp-connector-for-building-smarter-modular-ai-agents-thumbnail-1756277862571.jpg)
MCP: the Universal Connector for Building Smarter, Modular AI Agents
In this article, the authors discuss Model Context Protocol (MCP), an open standard designed to connect AI agents with tools and data they need. They also talk about how MCP empowers agent development, and its adoption in leading open-source frameworks.
-
/articles/ai-infrastructure-aggregating-agentic-traffic/en/smallimage/ai-infrastructure-aggregating-agentic-traffic-thumbnail-1755682463373.jpg)
The Missing Layer in AI Infrastructure: Aggregating Agentic Traffic
In this article, author Eyal Solomon discusses AI Gateways, the outbound proxy servers that intercept and manage AI-agent-initiated traffic in real time to enforce policies and provide central management.
-
/articles/infusing-ai-java/en/smallimage/infusing-ai-java-thumbnail-1754979423763.jpg)
Infusing AI into Your Java applications
Equip yourself with the basic AI knowledge and skills you need to start building intelligent and responsive Enterprise Java applications. With the help of our simple chatbot application for booking interplanetary space trips, see how Java frameworks like LangChain4j with Quarkus make it easy and efficient to interact with LLMs and create satisfying applications for end-users.
-
/articles/reproducible-ml-iceberg/en/smallimage/reproducible-ml-iceberg-thumbnail-1753341474504.jpg)
Building Reproducible ML Systems with Apache Iceberg and SparkSQL: Open Source Foundations
Traditional data lakes are great for storing massive amounts of stuff, but they're terrible at the transactional guarantees and versioning that ML workloads desperately need. Apache Iceberg and SparkSQL bring database-like reliability to your data lake. Time travel, schema evolution, and ACID transactions help support reproducible machine learning experiments.
-
/articles/guide-curating-technical-conference-track/en/smallimage/thumbnail-logo-image-A-First-Timers-Guide-to-Curating-a-Technical-Conference-Track-1753687916291.jpg)
A First-Timer’s Guide to Curating a Technical Conference Track
One first-time track host shares the process, constraints, and takeaways from building a track from scratch at QCon London 2025.
-
/articles/optimizing-search-systems/en/smallimage/optimizing-search-systems-thumbnail-1752139120744.jpg)
Optimizing Search Systems: Balancing Speed, Relevance, and Scalability
Innovative software engineer focused on optimizing search performance in dynamic environments. This article highlights key strategies from our QCon San Francisco 2024 presentation, addressing challenges faced by platforms like Uber Eats in data indexing and retrieval. Our advancements ensure swift, relevant user experiences amidst ever-growing datasets.
-
/articles/agentic-ai-architecture-framework/en/smallimage/agentic-ai-architecture-framework-thumbnail-1752043295919.jpg)
Agentic AI Architecture Framework for Enterprises
To deploy agentic AI responsibly and effectively in the enterprise, organizations must progress through a three-tier architecture, Foundation tier, Workflow tier, and Autonomous tier where trust, governance, and transparency precede autonomy.
-
/articles/effective-practices-ai-chat-based-coding/en/smallimage/effective-practices-ai-chat-based-coding-thumbnail-1751458791140.jpg)
Effective Practices for Coding with a Chat-Based AI
In this article, we explore how AI agents are reshaping software development and the impact they have on a developer’s workflow. We introduce a practical approach to staying in control while working with these tools by adopting key best practices from the discipline of software architecture, including defining an implementation plan, splitting tasks, and so on.
-
/articles/docker-size-dive/en/smallimage/docker-size-dive-thumbnail-1751012053381.jpg)
Why Is My Docker Image So Big? A Deep Dive with ‘dive’ to Find the Bloat
AI images typically bloat from massive library installations and base OS components, with large Docker images slowing AI development and increasing costs. Chirag Agrawal demonstrates how to diagnose bloat using Docker's history and the interactive 'dive' tool to examine each layer in detail. The article shows how effective diagnosis leads to targeted optimizations.