Kai Waehner

314 posts
builds cloud-native event streaming infrastructures for real-time data processing and analytics
Apache Kafka and Apache Flink for Open Source and Cloud-native Data Streaming
Read More
  • 14.6K views
  • 12 minute read

Apache Kafka (including Kafka Streams) + Apache Flink = Match Made in Heaven

Apache Kafka and Apache Flink are increasingly joining forces to build innovative real-time stream processing applications. This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink.
Read More
1
0
0
The Daily Life of a Field CTO
Read More
  • 7.5K views
  • 24 minute read

The Daily Life of a Field CTO

Field CTO is a job role that comes up more and more in software companies. There is no standard definition for this job role. As I regularly get the question about what a Field CTO does, this blog post summarizes a few general characteristics and then explores my daily life as a Field CTO. The post concludes with the answer to how you can also become a Field CTO at a software company in your career.
Read More
3
0
0
Apache Kafka for Data Consistency (and Real-Time Data Streaming)
Read More
  • 6.1K views
  • 6 minute read

Apache Kafka for Data Consistency (and Real-Time Data Streaming)

Real-time data beats slow data in almost all use cases. But as essential is data consistency across all systems, including non-real-time legacy systems and modern request-response APIs. Apache Kafka’s most underestimated feature is the storage component based on the append-only commit log. It enables loose coupling for domain-driven design with microservices and independent data products in a data mesh. This blog post explores how Kafka enables data consistency with a real-world case study from financial services.
Read More
10
0
0
Data Streaming Landscape 2023 with Apache Kafka Flink and much more
Read More
  • 7.0K views
  • 13 minute read

The Data Streaming Landscape 2023

Data streaming is a new software category to process data in motion. Apache Kafka is the de facto standard used by over 100,000 organizations. Plenty of vendors offer Kafka platforms and cloud services. Many complementary stream processing engines like Apache Flink and SaaS offerings have emerged. And competitive technologies like Pulsar and Redpanda try to get market share. This blog post explores the data streaming landscape of 2023 to summarize existing solutions and market trends.
Read More
1
0
0
Top Use Cases and Architectures for Data Streaming with Apache Kafka in 2023
Read More
  • 5.5K views
  • 7 minute read

Top 5 Data Streaming Trends for 2023

Data Streaming is one of the most relevant buzzwords in tech to build scalable real-time applications in the cloud and innovative business models. Do you wonder about my predicted TOP 5 data streaming trends in 2023 to set data in motion? Check out the following presentation and learn what role Apache Kafka plays. Learn about decentralized Data Mesh, cloud-native lakehouse, data sharing, improved user experience, and advanced data governance.
Read More
0
0
0
Apache Kafka vs Redpanda Comparison
Read More
  • 16.0K views
  • 18 minute read

When to choose Redpanda instead of Apache Kafka?

Data streaming emerged as a new software category. It complements traditional middleware, data warehouse, and data lakes. Apache Kafka became the de facto standard. New players enter the market because of Kafka’s success. One of those is Redpanda, a lightweight Kafka-compatible C++ implementation. This blog post explores the differences between Apache Kafka and Redpanda, when to choose which framework, and how the Kafka ecosystem, licensing, and community adoption impact a proper evaluation.
Read More
0
0
0
Decentralized Data Mesh with Data Streaming in Financial Services and Banking
Read More
  • 5.5K views
  • 5 minute read

Decentralized Data Mesh with Data Streaming in Financial Services

Digital transformation requires agility and fast time to market as critical factors for success in any enterprise. The decentralization with a data mesh separates applications and business units into independent domains. Data sharing in real-time with data streaming helps to provide information in the proper context to the correct application at the right time. This blog post explores a case study from the financial services sector where a data mesh was built across countries for loosely coupled data sharing but standardized enterprise-wide data governance.
Read More
0
0
0
Fraud Detection and Prevention with Apache Kafka KSQL Apache Flink
Read More
  • 7.4K views
  • 5 minute read

Fraud Detection with Apache Kafka, KSQL and Apache Flink

Fraud detection becomes increasingly challenging in a digital world across all industries. Real-time data processing with Apache Kafka became the de facto standard to correlate and prevent fraud continuously before it happens. This blog post explores case studies for fraud prevention from companies such as Paypal, Capital One, ING Bank, Grab, and Kakao Games that leverage stream processing technologies like Kafka Streams, KSQL, and Apache Flink.
Read More
3
0
0
Cloud Native SCADA Industrial IoT with Apache Kafka Data Streaming
Read More
  • 5.5K views
  • 6 minute read

A cloud-native SCADA System for Industrial IoT built with Apache Kafka

Industrial IoT and Industry 4.0 enable digitalization and innovation. SCADA control systems are a vital component of IT/OT modernization. The SCADA evolution started with monolithic applications and moved to networked and web-based platforms. This blog post explores building the 5th generation: A cloud-native SCADA infrastructure with Apache Kafka. A real-world case study explores the journey of a German system operator for electricity to show how such a journey to open and scalable real-time workloads and edge-to-cloud integration progressed.
Read More
0
0
0