A Real-Time Supply Chain Control Tower powered by Kafka

A modern supply chain requires just-in-time production, global logistics, and complex manufacturing processes. Intelligent control of the supply network is becoming increasingly demanding, not just because of Covid. At the same time, digitalization is generating exponentially growing data streams along the value chain. This blog post explores a solution that links the control with the data streams leveraging Apache Kafka to provide end-to-end visibility of your supply chain. All the digital information flows into a unified central nervous system enabling comprehensive control and timely response. The idea of the Supply Chain Control Tower becomes a reality: An integrated data cockpit with real-time access to all levels and systems of the supply chain.

What is a supply chain control tower?

A supply chain is a network of companies and people that are involved in the production and delivery of a product or service. Today, many supply chains are global and involve intra-logistics, widespread enterprise logistics, and B2B data sharing for end-to-end supply chains.

Supply chain management (SCM)

Supply Chain Management (SCM) involves planning and coordinating all the people, processes, and technology involved in creating value for a company. This includes cross-cutting processes, including purchasing/procurement, logistics, operations/manufacturing, and others. Automation, robustness, flexibility, real-time, and hybrid deployment (edge + cloud) are essential for future success and a pre-requirement for end-to-end visibility across the supply chain, regardless of industry.

The challenge with logistics and supply chain is that you have a lot of commercial off-the-shelf applications (ERP, WMS, TMS, DPS, CRM, SRM, etc.) in place that are highly specialized and highly advanced in their function.

Challenges of batch workloads across the supply chain

Batch workloads create many issues in a supply chain. The Covid pandemic showed this:

• Missing information: Intra-logistics within a distribution or fulfillment center, across buildings, and regions, and between business partners.
• Cost rising: Lower overall equipment efficiency (OEE). Lower availability. Increase cost for production and buyers (B2B and B2C).
• Customers churning: Bad customer experience and contract discussion as a consequence of failing with delivery guarantees and other SLAs.
• Revenue decreasing: Less production and/or sales means less revenue.

The specialized systems like an ERP, WMS, TMS, DPS, CRM, or SRM are often modernized and real-time or nearly real-time in their operation. However, the integrations between them are often still not real-time. For instance, batch waves in a WMS are being replaced with real-time order allocation processes, but the link between the WMS and the ERP is still batch.

Real-time end-to-end monitoring

A supply chain control tower provides end-to-end visibility and real-time monitoring across the supply chain:

The control tower helps answer questions such as

• What is happening now?
• Why is this happening?
• What might happen next?
• How can we perform better?
• What if…?

A supply chain control tower combines technology, processes, and people. Check out this tremendous seven-minute YouTube video to get a simple explanation of the supply chain control tower “for dummies”. Most importantly, the evolution of software enables real-time automation instead of just human visual monitoring.

A Kafka-native supply chain control tower

Apache Kafka is the de facto standard for data streaming. It enables real-time data integration and processing at any scale. This is a crucial requirement for building a supply chain control tower.

I blogged about Apache Kafka for Supply Chain Management (SCM) Optimization before. Check it out to learn about real-time applications for logistics, inventory management, track and trace, and other real-time deployments from BMW, Bosch, Baader, and more.

Let’s recap the added value of Kafka for improving the supply chain in one picture of business value across use cases:

If you look at the definitions of a supply chain control tower, well, that’s more or less the same 🙂 A supply chain control tower is only possible with real-time data correlation. That’s why Kafka is the perfect fit.

Global data mesh for real-time data sharing

Supply chains are global and rely on the collaboration between independent business units within enterprises, and B2B communication:

The software industry pitches a new paradigm and architecture pattern these days: The Data Mesh. It allows independent, decoupled domains using its own technology, API, and data structures. But the data sharing is standardized, compatible, real-time, and reliable. The heart of such a data mesh beats real-time with the power of Apache Kafka:

Kafka-native tools like MirrorMaker or Confluent Cluster Linking enable reliable real-time replication across data centers, clouds, regions, and even between independent companies.

Kafka as the data hub for a real-time supply chain control tower

A supply chain control tower coordinates demand, supply, trading, and logistics across various domains, technologies, APIs, and business processes.

The control tower is not just a Kafka cluster – but Kafka is underpinning business logic, integrations, and visualizations that aggregate all the data from the various sources and make it more valuable from end to end in real-time:

The data communication happens in real-time, near real-time, batch, or request-response. Whatever the source and sink applications support. However, the heart of the enterprise architecture is real-time and scalable. This enables future modernization and replacing legacy batch systems with modern real-time services.

Visualization AND automatic resolutions as core components of a modern control tower

The first goal of a supply chain control tower is end-to-end visibility in real-time. Creating real-time dashboards, alerts, and integration with 3rd party monitoring tools is a massive value.

However, the built pipeline enables several additional new use cases. Data integration and correlation across legacy and modern applications is the significant change for innovation regarding supply chain optimization: Instead of just visualizing in real-time what happens, new applications can take automated actions and decisions to solve problems automatically based on real-time information.

Real-world Kafka examples across the supply chain for visibility and automation

Modern supply chains rely on real-time data across suppliers, fulfillment centers, warehouses, stores, and consumers. Here are a few real-world examples across verticals using Apache Kafka as a real-time supply chain control tower:

• Retail: Walmart monitors information across the supply chain, including distribution centers, fulfillment centers, stores, mobile apps, and B2B interfaces. Examples for use cases are local and global inventory management and replenishment systems.
• Manufacturing: BMW ingests data from its smart factories into the cloud to provide access to the data for visibility and new automation applications. Baader built a real-time locating system (RTLS) to optimize end-to-end monitoring and advanced calculations like routing and estimated arrival in real time.
• Logistics: DHL, Swiss Post, Hermes, SBB, and many other companies in the logistics and transportation space built real-time track and trace platforms to optimize the visibility and efficiency of their business processes.
• Mobility Services: Almost any ride-hailing or food delivery app across the planet like Uber, Lyft, Grab, or Free Now (former MyTaxi) leverages Apache Kafka for real-time notifications to the front-end and data correlation in the backend to implement real-time driver-rider-matching, routing, payment, fraud detection, and many other use cases.
• Food: Albertsons, Instacart, Domino’s Pizza, Migros, and many other food-related enterprises improve business processes across the food value chain with real-time services.

No matter what industry you work in. Learn from other companies across verticals. Challenges to improving the supply chain are very similar everywhere.

Postmodern ERP / MES / TMS powered by Apache Kafka

While the supply chain control tower provides end-to-end visibility, each system has similar requirements. Enterprise resource planning (ERP) has existed for many years. It is often monolithic, complex, proprietary, batch, and not scalable. The same is true for MES, TMS, and many other platforms for logistics and supply chain management.

Postmodern ERP represents the next generation of ERP architectures. It is real-time, scalable, and open. A Postmodern ERP combines open source technologies and proprietary standard software. Many solutions are cloud-native or even offered as fully managed SaaS.

Like end-users, software vendors leverage data streaming with Apache Kafka to implement a Postmodern ERP, MES, CRM, SRM, or TMS:

Logistics and supply chain management require real-time data

Data streams grow exponentially along the value chain. End-to-end visibility of the supply chain is crucial for optimized business processes. Real-time order management and inventory management are just two examples.

Visualization and taking actions in real-time via automated processes either reduce cost and risk or increases the revenue and customer experience. A real-time supply chain control tower enables this kind of innovation. The foundation of such a strategic component needs to be real-time, scalable, and reliable. That’s why the data streaming platform Apache Kafka is the perfect fit for building a control tower.

Various success stories across industries prove the value of data streaming across the supply chain. Even software vendors of products for the supply chain like ERP, WMS, TMS, DPS, CRM, or SRM build their next-generation software on top of Apache Kafka.

How do you bring visibility into your supply chain? Did you already build a real-time control tower? What role plays data streaming in these scenarios? Let’s connect on LinkedIn and discuss it! Stay informed about new blog posts by subscribing to my newsletter.