Browsing Tag

Data Streaming

37 posts
Read More
  • 507 views
  • 6 minute read

Free Ebook: Data Streaming Use Cases and Industry Success Stories Featuring Apache Kafka and Flink

Real-time data is no longer optional—it’s essential. Businesses across industries use data streaming to power insights, optimize operations, and drive innovation. After 7+ years at Confluent, I’ve seen firsthand how Apache Kafka and Flink transform organizations. That’s why I wrote The Ultimate Data Streaming Guide: Concepts, Use Cases, Industry Stories—a free eBook packed with insights, real-world examples, and best practices. Download your free copy now and start your data streaming journey!
Read More
0
0
0
SaaS vs PaaS Cloud Service for Data Streaming with Apache Kafka and Flink
Read More
  • 1.2K views
  • 6 minute read

Fully Managed (SaaS) vs. Partially Managed (PaaS) Cloud Services for Data Streaming with Kafka and Flink

The cloud revolution has reshaped how businesses deploy and manage data streaming with solutions like Apache Kafka and Flink. Distinctions between SaaS and PaaS models significantly impact scalability, cost, and operational complexity. Bring Your Own Cloud (BYOC) expands the options, giving businesses greater flexibility in cloud deployment. Misconceptions around terms like “serverless” highlight the need for deeper analysis to avoid marketing pitfalls. This blog explores deployment options, enabling informed decisions tailored to your data streaming needs.
Read More
0
0
0
Lakehouse and Data Streaming - Competitor or Complementary
Read More
  • 3.8K views
  • 12 minute read

How Microsoft Fabric Lakehouse Complements Data Streaming (Apache Kafka, Flink, et al.)

In today’s data-driven world, understanding data at rest versus data in motion is crucial for businesses. Data streaming frameworks like Apache Kafka and Apache Flink enable real-time data processing. Meanwhile, lakehouses like Snowflake, Databricks, and Microsoft Fabric excel in long-term data storage and detailed analysis, perfect for reports and AI training. This blog post explores how these technologies complement each other in enterprise architecture.
Read More
0
0
0
Apache Kafka Deployment Options - Serverless vs Self-Managed vs BYOC Bring Your Own Cloud
Read More
  • 2.1K views
  • 9 minute read

Deployment Options for Apache Kafka: Self-Managed, Fully-Managed / Serverless and BYOC (Bring Your Own Cloud)

BYOC (Bring Your Own Cloud) is an emerging deployment model for organizations looking to maintain greater control over their cloud environments. Unlike traditional SaaS models, BYOC allows businesses to host applications within their own VPCs to provide enhanced data privacy, security, and compliance. This approach leverages existing cloud infrastructure. It offers more flexibility for custom configurations, particularly for companies with stringent security needs. In the data streaming sector around Apache Kafka, BYOC is changing how platforms are deployed. Organizations get more control and adaptability for various use cases. But it is clearly NOT the right choice for everyone!
Read More
0
0
0
Unified Commerce with Data Streaming using Apache Kafka and Flink at the Edge and in the Cloud
Read More
  • 2.3K views
  • 8 minute read

Unified Commerce in Retail and eCommerce with Apache Kafka and Flink for Real-Time Customer 360

Delivering a seamless and personalized customer experience across all touchpoints is essential for staying competitive in today’s rapidly evolving retail and eCommerce landscape. Unified commerce integrates all sales channels and backend systems into a single platform to ensure real-time consistency in customer interactions, inventory management, and order fulfillment. This blog post explores how Apache Kafka and Flink can be pivotal in achieving real-time Customer 360 in the unified commerce ecosystem and how it differs from traditional omnichannel approaches.
Read More
0
0
0
Energy Trading with Apache Kafka and Flink at Uniper ReAlto Powerledger
Read More
  • 3.3K views
  • 9 minute read

Energy Trading with Apache Kafka and Flink

Energy trading and data streaming are connected because real-time data helps traders make better decisions in the fast-moving energy markets. This data includes things like price changes, supply and demand, smart IoT meters and sensors, and weather, which help traders react quickly and plan effectively. As a result, data streaming with Apache Kafka and Apache Flink makes the market clearer, speeds up information sharing, and improves forecasting and risk management. This blog post explores the use cases and architectures for scalable and reliable real-time energy trading, including real-world deployments from Uniper, re.alto and Powerledger.
Read More
0
0
0
RAG and Kafka Flink to Prevent Hallucinations in GenAI
Read More
  • 7.8K views
  • 5 minute read

Real-Time GenAI with RAG using Apache Kafka and Flink to Prevent Hallucinations

How do you prevent hallucinations from large language models (LLMs) in GenAI applications? LLMs need real-time, contextualized, and trustworthy data to generate the most reliable outputs. This blog post explains how RAG and a data streaming platform with Apache Kafka and Flink make that possible. A lightboard video shows how to build a context-specific real-time RAG architecture. Also, learn how the travel agency Expedia leverages data streaming with Generative AI using conversational chatbots to improve the customer experience and reduce the cost of service agents.
Read More
0
0
0
Data Lineage for Data Streaming with OpenLineage Apache Kafka and Flink
Read More
  • 12.4K views
  • 11 minute read

Open Standards for Data Lineage: OpenLineage for Batch AND Streaming

One of the greatest wishes of companies is end-to-end visibility in their operational and analytical workflows. Where does data come from? Where does it go? To whom am I giving access to? How can I track data quality issues? The capability to follow the data flow to answer these questions is called data lineage. This blog post explores market trends, efforts to provide an open standard with OpenLineage, and how data governance solutions from vendors such as IBM, Google, Confluent and Collibra help fulfil the enterprise-wide data governance needs of most companies, including data streaming technologies such as Apache Kafka and Flink.
Read More
0
0
0
The Past Present and Future of Stream Processing
Read More
  • 21.4K views
  • 12 minute read

The Past, Present and Future of Stream Processing

Stream processing has existed for decades. The adoption grows with open source frameworks like Apache Kafka and Flink in combination with fully managed cloud services. This blog post explores the past, present and future of stream processing, including the relation of machine learning and GenAI, streaming databases, and the integration between data streaming and data lakes with Apache Iceberg.
Read More
0
0
0
GenAI Demo with Kafka, Flink, LangChain and OpenAI
Read More
  • 9.7K views
  • 10 minute read

GenAI Demo with Kafka, Flink, LangChain and OpenAI

Generative AI (GenAI) enables automation and innovation across industries. This blog post explores a simple but powerful architecture and demo for the combination of Python, and LangChain with OpenAI LLM, Apache Kafka for event streaming and data integration, and Apache Flink for stream processing. The use case shows how data streaming and GenAI help to correlate data from Salesforce CRM, searching for lead information in public datasets like Google and LinkedIn, and recommending ice-breaker conversations for sales reps.
Read More
0
0
0