Stateful Archives - Kai Waehner

1.1K views
8 minute read

Apache Flink: Overkill for Simple, Stateless Stream Processing and ETL?

ByKai Waehner
14. January 2025

Discover when Apache Flink is the right tool for your stream processing needs. Explore its role in stateful and stateless processing, the advantages of serverless Flink SaaS solutions like Confluent Cloud, and how it supports advanced analytics and real-time data integration together with Apache Kafka. Dive into the trade-offs, deployment options, and strategies for leveraging Flink effectively across cloud, on-premise, and edge environments, and when to use Kafka Streams or Single Message Transforms (SMT) within Kafka Connect for ETL instead of Flink.

Stateless and Stateful Stream Processing with Kafka Streams and Apache Flink

1.5K views
8 minute read

Stateless vs. Stateful Stream Processing with Kafka Streams and Apache Flink

ByKai Waehner
27. December 2024

The rise of stream processing has changed how we handle and act on data. While traditional databases, data lakes, and warehouses are effective for many batch-based use cases, they fall short in scenarios demanding low latency, scalability, and real-time decision-making. This post explores the key concepts of stateless and stateful stream processing, using Kafka Streams and Apache Flink as examples.