I did a webinar with Confluent‘s partner Expero about “Apache Kafka and Machine Learning for Real Time Supply Chain Optimization“. This is a great example for anybody in automation industry / Industrial IoT (IIoT) like automotive, manufacturing, logistics, etc.

We explain how a real time event streaming platform can integrate in real time with the legacy world and proprietary IIoT protocols (like Siemens S7, Modbus, Beckhoff ADS, OPC-UA, et al). You can process the data at scale and then ingest it into a modern database (like AWS S3, Snowflake or MongoDB) or analytic / machine  learning framework (like TensorFlow, PyTorch or Azure Machine Learning Service).

Here is the architecture we use to discuss and implement the supply chain optimization use case leveraging real time stream processing and machine learning:

We leverage various components from the Apache Kafka ecosystem. This includes:

• Kafka Connect as scalable and reliable integration framework
• Kafka Connect connectors like PLC4X – a great Apache framework to integrate with IIoT protocols
• KSQL for continuous processing (filter, transform, aggregate) of the sensor data
• Kafka Streams to deploy the trained analytic models to a real time streaming scoring engine

Use Case: Supply Chain Optimization using Real Time and Batch Processes

Automating multifaceted, complex workflows requires hybrid solutions including streaming analytics of IOT data and batch analytics. This includes machine learning solutions and real time visualization. Leaders in organizations who are responsible for global supply chain planning are responsible for working with and integrating with data from disparate sources around the world. Many of these data sources output information in real time. This assists planners in operationalizing plans and interacting with manufacturing output. IOT sensors on manufacturing equipment and inventory control systems feed real time processing pipelines to match actuals productions figures against planned schedules to calculate yield efficiency.

Using information from both real time systems and batch optimization, supply chain managers are able to economize operations and automate tedious inventory and manufacturing accounting processes. Sitting on top of all of these systems is a supply chain visualization tool. This enables users’ visibility over the global supply chain. If you are responsible for key data integration initiatives, join for a detailed walk through of a customer’s use of this system built using Confluent and Expero tools.

What will you learn?

• See different use cases in automation industry and Industrial IoT (IIoT) where an event streaming platform adds business value
• Understand different architecture options to leverage Apache Kafka and Confluent Platform in IoT scenarios in the cloud, on premise data centers and at the edge
• Learn how to leverage different analytics tools and machine learning frameworks in a flexible and scalable way
• How real time visualization ties together streaming and batch analytics for business users, interpreters, and analysts
• Understand how streaming and batch analytics optimize the supply chain planning workflow.
• Conceptualize the intersection between resource utilization and manufacturing assets with long term planning and supply chain optimization.

Industrial Internet of Things (IIoT) in Real Time at Scale with Apache Kafka

Here is the slide deck and video recording. Have fun watching it. Please let me know if you have any feedback or questions:

Slide Deck:

Video Recording: