Browsing Category

Stream Processing

43 posts
Apache Flink - Overkill for Simple Stateless Stream Processing
Read More
  • 858 views
  • 8 minute read

Apache Flink: Overkill for Simple, Stateless Stream Processing and ETL?

Discover when Apache Flink is the right tool for your stream processing needs. Explore its role in stateful and stateless processing, the advantages of serverless Flink SaaS solutions like Confluent Cloud, and how it supports advanced analytics and real-time data integration together with Apache Kafka. Dive into the trade-offs, deployment options, and strategies for leveraging Flink effectively across cloud, on-premise, and edge environments, and when to use Kafka Streams or Single Message Transforms (SMT) within Kafka Connect for ETL instead of Flink.
Read More
0
0
0
Stateless and Stateful Stream Processing with Kafka Streams and Apache Flink
Read More
  • 1.4K views
  • 8 minute read

Stateless vs. Stateful Stream Processing with Kafka Streams and Apache Flink

The rise of stream processing has changed how we handle and act on data. While traditional databases, data lakes, and warehouses are effective for many batch-based use cases, they fall short in scenarios demanding low latency, scalability, and real-time decision-making. This post explores the key concepts of stateless and stateful stream processing, using Kafka Streams and Apache Flink as examples.
Read More
0
0
0
Data Streaming Trends for 2025 - Leading with Apache Kafka and Flink
Read More
  • 6.1K views
  • 18 minute read

Top Trends for Data Streaming with Apache Kafka and Flink in 2025

Apache Kafka and Apache Flink are leading open-source frameworks for data streaming that serve as the foundation for cloud services, enabling organizations to unlock the potential of real-time data. Over recent years, trends have shifted from batch-based data processing to real-time analytics, scalable cloud-native architectures, and improved data governance powered by these technologies. Looking ahead to 2025, the data streaming ecosystem is set to undergo even greater changes. Here are the top trends shaping the future of data streaming for businesses.
Read More
0
0
0
Dynamic Pricing with Data Streaming using Apache Kafka and Flink
Read More
  • 1.4K views
  • 9 minute read

A New Era in Dynamic Pricing: Real-Time Data Streaming with Apache Kafka and Flink

In the age of digitization, the concept of pricing is no longer fixed or manual. Instead, companies increasingly use dynamic pricing — a flexible model that adjusts prices based on real-time market changes to enable real-time responsiveness, giving companies the tools they need to respond instantly to demand, competitor prices, and customer behaviors. This blog post explores the fundamentals of dynamic pricing, its link to data streaming, and real-world examples across different industries such as retail, logistics, gaming and the energy sector.
Read More
0
0
0
The Past Present and Future of Stream Processing
Read More
  • 21.0K views
  • 12 minute read

The Past, Present and Future of Stream Processing

Stream processing has existed for decades. The adoption grows with open source frameworks like Apache Kafka and Flink in combination with fully managed cloud services. This blog post explores the past, present and future of stream processing, including the relation of machine learning and GenAI, streaming databases, and the integration between data streaming and data lakes with Apache Iceberg.
Read More
0
0
0
Read More
  • 7.9K views
  • 10 minute read

Top 5 Trends for Data Streaming with Kafka and Flink in 2024

Do you wonder about my predicted TOP 5 data streaming trends with Apache Kafka and Flink in 2024 to set data in motion? Discover new technology trends and best practices for event-driven architectures, including data sharing, data contracts, serverless stream processing, multi-cloud architectures, and GenAI.
Read More
0
0
1
Python Kafka Quix Streams and Flink for Open Source Stream Processing
Read More
  • 5.3K views
  • 12 minute read

Quix Streams – Stream Processing with Kafka and Python

Over 100,000 organizations use Apache Kafka for data streaming. However, there is a problem: The broad ecosystem lacks a mature client framework and managed cloud service for Python data engineers. Quix Streams is a new technology on the market trying to close this gap. This blog post discusses this Python library, its place in the Kafka ecosystem, and when to use it instead of Apache Flink or other Python- or SQL-based substitutes.
Read More
0
0
0
Data Streaming is not a Race it is a Journey
Read More
  • 4.3K views
  • 11 minute read

Data Streaming is not a Race, it is a Journey!

Data Streaming is not a race, it is a journey! Event-driven architectures and technologies like Apache Kafka or Apache Flink require a mind shift in architecting, developing, deploying, and monitoring applications. This blog post explores success stories from data streaming journeys across industries, including banking, retail, insurance, manufacturing, healthcare, energy & utilities, and software companies.
Read More
0
0
1
Apache Kafka and Apache Flink for Open Source and Cloud-native Data Streaming
Read More
  • 16.3K views
  • 12 minute read

Apache Kafka (including Kafka Streams) + Apache Flink = Match Made in Heaven

Apache Kafka and Apache Flink are increasingly joining forces to build innovative real-time stream processing applications. This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink.
Read More
1
0
0
Data Streaming Landscape 2023 with Apache Kafka Flink and much more
Read More
  • 8.1K views
  • 13 minute read

The Data Streaming Landscape 2023

Data streaming is a new software category to process data in motion. Apache Kafka is the de facto standard used by over 100,000 organizations. Plenty of vendors offer Kafka platforms and cloud services. Many complementary stream processing engines like Apache Flink and SaaS offerings have emerged. And competitive technologies like Pulsar and Redpanda try to get market share. This blog post explores the data streaming landscape of 2023 to summarize existing solutions and market trends.
Read More
1
0
0