Confluent, a subsidiary of IBM, has expanded its data streaming offerings with new functions aimed at supporting AI systems with real-time business context.
The company has made its Lightning Tables feature generally available on Amazon Web Services (AWS), finalising what it calls a five-pillar Data Streaming Platform (DSP).
This step adds query functionality to the existing capabilities to stream, connect, process and govern data under a unified approach for what Confluent terms the Streamhouse architecture.
The Streamhouse is intended to address a recurring challenge: as AI agents are introduced throughout enterprises, they must often make decisions based on fragmented, asynchronously updated data from a variety of systems.
This can result in decisions being made on outdated or conflicting information, potentially disrupting workflows.
The Streamhouse aims to resolve this by enabling the continuous capture, processing, governance and provision of current business information across systems, applications and analytics tools.
Confluent has illustrated the potential use case of an airline managing flight operations, where multiple systems can offer inconsistent views of aircraft locations, schedules and seat availability.
The Streamhouse model is designed to provide a constantly updated, shared view, allowing operational teams and AI decision-makers to respond in a timely and coordinated manner.
The architecture of the Streamhouse is built around three principles: it maintains continuously updated business information, it is designed for high availability and reliability, and it operates without requiring all data to be physically consolidated beforehand.
Rather than replacing existing data warehouses and lakehouses, it adds a real-time operational layer to support decision-making and actions.
At the centre of Confluent’s Data Streaming Platform is Apache Kafka, which streams business events.
The platform includes more than 150 managed and custom connectors for integrating enterprise data, batch execution features with Apache Flink, and tools for security and data governance such as schema and access policy controls.
The new query function offers several interfaces, including Confluent Tableflow for analytics, a Real-Time Context Engine for AI agents, and Lightning Tables for rapid SQL or API-based queries.
According to the company, Lightning Tables provides a read-only serving layer for data captured in Apache Kafka, enabling queries with reported latency between 10ms and 100ms and throughput exceeding 1,000 queries per second.
Features include multiple query interfaces, unified data governance, schema management and a machine learning feature store.
IBM Confluent and data general manager Shaun Clowes said: “The next era of enterprise AI will be won by the agents that know what’s happening in the business the moment they act. That’s what a Streamhouse can deliver.
“With Lightning Tables, our Data Streaming Platform gives each agent and application a live, governed view of the business. This means companies can act on fraud signals sooner, serve customers in the moment and run operations on what’s true now versus what was true yesterday.”
IBM completed the $11bn acquisition of Confluent earlier this year with the aim of enhancing real-time data streaming for enterprise AI applications.
