Data Streaming with Apache Kafka at Daimler Truck for Industrial IoT and Cloud Use Cases

In the fast-changing automotive industry, Daimler Truck is recognized for its innovation and technological progress. As a global leader in the commercial vehicle sector, Daimler Truck is not only committed to producing high-quality trucks and buses, but is also at the forefront of pioneering sustainable and intelligent mobility solutions. This blog post delves into Daimler Truck’s role in automotive innovation, exploring how the company leverages data streaming powered by Apache Kafka together with Industrial IoT technologies to transform its manufacturing processes, enhance operational efficiency, and build a future-ready, scalable real-time data pipeline from the edge to the cloud.

Blog Series about Daimler Truck’s Data Streaming Use Cases

Inspired by a successful hybrid Kafka Day 2024 at Daimler Truck’s Gaggenau headquarters, where data streaming topics like open source, cloud, migration, and hybrid architecture were streamed globally, I’m launching a blog series to dive deeper into these transformative technologies.

Here are a few impressions from the Kafka Day:

This is part one of a three-part blog series exploring several innovative architectures and use cases for data streaming at Daimler Truck:

1. THIS POST: Data Streaming with Apache Kafka at Daimler Truck for Industrial IoT and Cloud Use Cases
2. COMING SOON: Real-Time Locating System (RTLS) and Digital Twin in Smart Factories with Data Streaming (Apache Kafka) at Daimler Truck
3. COMING SOON: Revolutionizing Fleet Management: Daimler Truck’s Real-Time Signal Tracking with Kafka Streams

Subscribe to my newsletter to stay in touch and receive new articles about all the things data streaming.

Driving the Future: Daimler Truck’s Role in Automotive Innovation

Daimler Truck is a global leader in the commercial vehicle industry, known for its innovation and commitment to quality. With over 40 production plants worldwide, the company produced over 500,000 vehicles in 2022, including renowned models like the Mercedes-Benz Actros and Freightliner Cascadia.

Daimler Truck’s strategy focuses on innovation and digitalization, aiming for carbon-neutral transportation by 2039 through investments in electric and hydrogen fuel cell technologies. The company is also pioneering autonomous driving technology to enhance road safety and efficiency.

The Digital Factory Layer (DFL): Strategic Hub for Data Collection, Processing and Insights

Digitalization is integral to Daimler Truck’s operations, with advanced telematics systems providing real-time data on vehicle health and driver behavior. This data-driven approach improves operational efficiency and supports proactive maintenance.

The Digital Factory Layer (DFL) platform is a strategic hub for data collection, processing, and real-time operational insights. DFL integrates a sophisticated ecosystem that includes various data platforms, including, Elastic, Kafka, and a standard Data Warehouse (DWH).

Daimler Truck is leveraging data streaming and Industrial IoT (IIoT) to transform its manufacturing processes. By integrating technologies like Apache Kafka and Apache Flink, the company aims to create smart factories with real-time monitoring and analytics.

In summary, Daimler Truck is dedicated to delivering high-quality, sustainable, and intelligent commercial vehicles, driven by a commitment to innovation and digitalization. From a business value perspective: 2min downtime = 1 truck less in production! Confluent provides the data streaming platform and support SLAs to solve these critical requirements to ensure a good KPI for the Overall Equipment Effectiveness (OEE).

Data Streaming with Apache Kafka and Flink for Industrial IoT at the Edge and Hybrid Cloud

In the realm of Industrial IoT (IIoT), data streaming has emerged as a critical component for real-time data processing and analytics. At Daimler Truck, Confluent creates a robust data streaming infrastructure. The data streaming platform serves as the central nervous system, enabling the seamless flow of data across various systems and applications. Kafka Streams (and Apache Flink in the future) provide powerful stream processing capabilities for continuous real-time analytics and decision-making.

Industrial IoT Use Cases at Daimler Truck

Daimler Truck has implemented several IIoT use cases to enhance operational efficiency and drive innovation. These use cases include:

• Monitoring and Alerting: Real-time monitoring of machine performance and alerting for any anomalies or deviations from expected behavior.
• Anomaly Detection: Identifying unusual patterns in data to prevent potential issues before they escalate.
• Data Quality and Correlation for Business Applications: Aggregating and curating data to feed various business applications and decision-making processes.
• Device Management and Visualization: Managing connected devices and vehicles and visualizing data through intuitive dashboards.

Use Case Examples in Production

Daimler Truck has successfully implemented several use cases, including:

• Virtual Edge Device: Providing new information based on original data, such as live shift-based part counters and machine efficiency.
• Change Over Detection: Using PLC data to detect changeovers and gather statistics for transparency.
• Tool Lifetime Information: Analyzing tool lifetime utilization to optimize tool changes.
• Cycle Time Analysis: Setting up statistics on cycle time for machines and parts using edge device data.

Concrete Example: Development of Cycle Time Analytics Using IIoT Data

Cycle time analytics involves measuring and analyzing the duration required to complete specific tasks or processes within a production cycle, using data collected from various sources, such as machine sensors, PLCs, and edge devices. This data provides insights into each stage of the production process.

The solution highlights bottlenecks and inefficiencies that can be addressed to optimize operations. By leveraging real-time data streaming from connected devices, cycle time analytics enables manufacturers to enhance productivity, reduce costs, and improve overall operational efficiency.

The development of cycle time analytics is a concrete example of how Daimler Truck leverages IIoT data. The goal is to build a comprehensive database for cycle time analytics, using streaming data to derive aggregated and meaningful insights. This initiative enables the company to optimize production processes and improve overall efficiency.

High-Level Architecture: Kafka as Central Nervous System at Daimler Truck

The high-level architecture at Daimler Truck shows Kafka as the central nervous system in its Digital Factory Layer (DFL) platform. This architecture supports the IIoT data pipeline, ensuring efficient data flow and processing across the organization.

A few general facts about the usage of the data streaming platform:

• DFL with IoT Stack deployed at 5 plants on-premise
• 110,000 messages/sec + 60MB/sec overall across 5 plants
• 1,400 machines and robots connected
• Approximately 900 Grafana Dashboards

Based on this flexible and scalable foundation, additional use cases and plants can be connected in the future without the need to re-invent the enterprise architecture.

Last Mile IT/OT Integration with MQTT and Other Connectors

The last mile integration involves connecting various data sources, such as PLCs, NC values, asset and machine values, and sensors, from the OT world via MQTT to the IT world. This integration is crucial for ensuring seamless data flow and communication between different systems. Other integration options, such as proprietary legacy protocols and databases, are also supported to accommodate diverse data sources.

Daimler Truck collects machine information via edge devices, focusing on several key parameters:

• Asset Values: Information and description of the data sent by machines.
• PLC Values: All information from the PLC/HMI, collected every 400ms.
• NC Values: All information from the NC, collected every 10ms.
• Tooling Values: Information regarding tool management systems, collected every 5000ms.
• Machine Errors: PLC and NC errors occurring in the machine, collected on an event basis.
• Energy Values: Energy consumption information of the machine, collected every 1000ms.

Industrial IoT Data Pipeline: Data Collection from the Edge to the Cloud for Data Processing with Kafka

Data collection from the edge to the cloud is a critical aspect of Daimler Truck’s IIoT strategy. By utilizing edge devices, data is collected directly from machines and transmitted to the cloud for further processing and analysis. This approach ensures that data is captured in real-time, providing valuable insights into machine performance and operational efficiency.

Data Products with Contracts for Data Sharing

Data products with Kafka Topics and Schemas built the foundation of Daimler Truck’s data processing strategy. This approach allows for data sharing with multiple decoupled consumers using different technologies and communication paradigms.

Scalable real-time data products with a data contract allow freedom of technology choice:

Daimler Truck uses various technologies for data processing:

• • Continuous Stream Processing: Using Kafka Streams or Apache Flink for real-time data processing.
  • Full Stack Applications: Developing applications that leverage the full potential of data streaming.
  • Real-Time Analytics: Using Trino for real-time analytics and insights.
  • Batch ETL for Business Intelligence: Employing Talend and Power BI for batch ETL processes and business intelligence.
  • Visualization: Visualizing long-term data with Grafana and using Jupyter notebooks for rapid prototyping by data engineers and scientists.

Data Streaming with Kafka as Strategic Hub for Industrial IoT and Cloud Use Cases

Daimler Truck’s strategic implementation of data streaming and IIoT technologies exemplifies the transformative power of real-time data processing. By leveraging Kafka and Flink in an event-driven architecture, the company has created a robust infrastructure that supports a wide range of use cases, driving operational efficiency and innovation. As the industry continues to develop, Daimler Truck remains at the forefront, setting new standards for adopting data-driven manufacturing and the creation of innovative business models in the commercial vehicle sector.

How do you leverage data streaming in your enterprise architecture for Industrial IoT? How do you connect to machines, equipment, and sensors in the IT/OT context? What is your data product strategy? Let’s connect on LinkedIn and discuss it! Stay informed about new blog posts by subscribing to my newsletter.