Apache Kafka and MQTT are a perfect combination for many Industrial IoT use cases. This blog series covers the pros and cons of both technologies. Various use cases across industries, including connected vehicles, manufacturing, mobility services, and smart city are explored. The examples use different architectures, including lightweight edge scenarios, hybrid integrations, and serverless cloud solutions. This post is part three: Manufacturing, Industrial IoT, and Industry 4.0.
Apache Kafka + MQTT Blog Series
The first blog post explores the relation between MQTT and Apache Kafka. Afterward, the other four blog posts discuss various use cases, architectures, and reference deployments.
Subscribe to my newsletter to get updates immediately after the publication. Besides, I will also update the above list with direct links to this blog series’s posts as soon as published.
The following list shows different examples where Kafka is used as a strategic platform for various manufacturing use cases to implement Industry 4.0 initiatives:
I already covered this topic in detail recently. Hence, check out the blog post “Apache Kafka for Manufacturing and Industrial IoT“. That post includes a detailed slide deck and video recording.
Let’s specific examples for Kafka and MQTT in Manufacturing 4.0 in the following sections.
The following diagram shows the architecture of a smart factory. Both MQTT and Kafka infrastructure are deployed at the edge in the factory for security, latency, and cost reasons:
This example connects to modern PLCs and other gateways via MQTT and Sparkplug B. The benefit of using this technology is a lightweight communication protocol and open standard.
An OT middleware such as OSIsoft PI is required if legacy and proprietary protocols such as Modbus or Siemens S7 need to be integrated. Most plants today are brownfield. Hence both proprietary integration platforms and open MQTT or OPC-UA integration must communicate with Kafka from the OT side. “Apache Kafka as Data Historian” explores the integration options in more detail.
The IT components (such as SAP ERP) and the integration platforms (HiveMQ, Confluent) run in the factory. Obviously, many other architectures are possible if latency and security allow it. Check out “Building a Smart Factory with Apache Kafka and 5G Campus Networks” for a hybrid cloud architecture. Additionally, most smart factories are not completely independent from the central IT world running in a remote data center or public cloud. Various “architecture patterns for distributed, hybrid, edge and global Apache Kafka deployments” exist to replicate data bi-directionally between smart factories and data centers.
Manufacturing processes in the automotive industry cannot go down. Hence, Daimler built a Vehicle Diagnostic System (VDS) to efficiently share information between test devices on the factory floor and enterprise IT systems. The VDS fulfills some core functionality in the manufacturing process for E/E components, such as calibrating sensors controlled by an ECU, flashing new firmware, personalizing the key to the car, and testing to make sure each ECU works properly.
MQTT works in bad networks. Therefore, test devices behave properly even if the network connection is dropped and reconnected.
The system is rolled out to 24 factories around the world. 10,000 testing devices are connected. The devices generate 470 million messages/month.
The complete case study from HiveMQ explores the use case in more detail.
BAADER is a worldwide manufacturer of innovative machinery for the food processing industry. They run an IoT-based and data-driven food value chain on Confluent Cloud:
The Kafka-based infrastructure provides a single source of truth across the factories and regions across the food value chain. Business-critical operations are available 24/7 for tracking, calculations, alerts, etc.
The event streaming platform runs on Confluent Cloud. Hence, Baader can focus on building new innovative business applications. The serverless Kafka infrastructure provides mission-critical SLAs and consumption-based pricing for all required capabilities: Messaging, storage, data integration, and data processing.
MQTT provides connectivity to machines and GPS data from vehicles at the edge. Kafka Connect connectors integrate MQTT and other IT systems such as Elasticsearch, MongoDB, and AWS S3. ksqlDB processes the data in motion continuously. Stream processing or streaming analytics are other terms for this concept.
In conclusion, Apache Kafka and MQTT are a perfect combination for Manufacturing, Industrial IoT, and Industry 4.0.
Follow the blog series to learn about use cases such as connected vehicles, manufacturing, mobility services, and smart city. Every blog post also includes real-world deployments from companies across industries. It is key to understand the different architectural options to make the right choice for your project.
What are your experiences and plans in IoT projects? What use case and architecture did you implement? Let’s connect on LinkedIn and discuss it! Stay informed about new blog posts by subscribing to my newsletter.
Siemens Healthineers, a global leader in medical technology, delivers solutions that improve patient outcomes and…
Discover my journey to achieving Lufthansa HON Circle (Miles & More) status in 2025. Learn…
Data streaming is a new software category. It has grown from niche adoption to becoming…
Apache Kafka and Apache Flink are leading open-source frameworks for data streaming that serve as…
This blog delves into Cardinal Health’s journey, exploring how its event-driven architecture and data streaming…
In the age of digitization, the concept of pricing is no longer fixed or manual.…