This article explores use cases and architectures for processing data running on Apache Kafka in the Industrial Internet of Things (IIoT) across vertical sectors such as automotive, energy, steel manufacturing, oil and gas, cybersecurity, shipping, and logistics. Use cases include predictive maintenance, quality assurance, tracking and tracing, real-time location systems (RTLS), and asset tracking.
Why Kafka is a key component of the Industrial Internet of Things and manufacturing development
The Industrial Internet of Things (IIoT) is a jumble of monolithic and proprietary technologies accumulated over the past few decades. Modbus, Siemens S7, SCADA, and similar "concepts" have dominated the industry. Suppliers have locked companies in by deliberately building incompatible products with closed interfaces. These systems still run on outdated, unsupported operating systems like Windows XP and lack security.
Fortunately, this is a complete transformation. Apache Kafka and its ecosystem play a crucial role in the evolution of the IIoT. System integration and data processing have gained an open architecture with a scalable and reliable infrastructure.
I speak with clients in this industry globally every week. Different challenges, use cases, and innovative ideas all stem from this. I've discussed this topic extensively before.
Kafka and Industrial IoT related content
Check out my other related blog posts on IIoT and manufacturing. Learn about use cases and architectures for edge deployments (i.e., outside the data center), the relationship between Kafka and other IoT standards such as MQTT or OPC-UA, and how to build a modern, open, and scalable data historian.
I want to emphasize an article because it is important for discussions on workshops, PLCs, machines, robots, automobiles, and any other embedded systems: Kafka and other IT software are not real-time.
This article "simply" shares my latest introduction to this topic, including slide panels and on-demand video recordings. Before we get there, let's summarize the current Kafka scenarios in the Industrial Internet of Things with a concrete example.
The Industrial Internet of Things (IIoT) requires: ubiquitous, comprehensive, and cloud-based!
Let's look at a concrete example. The following diagram illustrates the use of event streaming in conjunction with other OT and IT technologies in the aviation industry:
This is an interesting example because it illustrates the challenges and requirements of many real-world IoT scenarios across vertical industries:
• Ubiquitous: The Industrial Internet of Things (IIoT) cannot be realized solely in the cloud. Since manufacturing produces tangible goods, the edge is unavoidable. For many use cases, integration between the (often disconnected) edge and the data center is essential.
• Complete: Industrial IoT is mission-critical. Within vertical regions, zero downtime, safety, and secure stability are paramount. The only realistic option is a robust, battle-tested enterprise-grade solution to realize IIoT use cases.
• Cloud Native: To remain competitive, enterprises need automated, scalable, decoupled, agile applications, and the flexibility of technologies and products. This applies not only to the cloud but also to the edge! However, not all use cases require a critical, scalable solution. For example, in a disconnected drone, a single agent for data processing and storage suffices.
One of Kafka's unique values is that you can use a single technology to achieve scalable, real-time messaging, storage and caching, continuous stateless and stateful data processing, and data integration with the OT and IT worlds. This is especially important at the edge, where hardware and network constraints are paramount. Deploying a single infrastructure, rather than gluing together the best types as you often do in the cloud, is much easier and more cost-effective.
Now that we've covered the basics, let's share the slides and video recordings to discuss all these points in more detail.
Slide table: Kafka 4.0 for Industrial IoT and manufacturing.
Apache Kafka provides an open, scalable, and flexible IIoT architecture.
The Industrial Internet of Things (IIoT) is a jumble of monolithic and proprietary technologies that has existed for decades. Fortunately, Apache Kafka is completely transforming many industrial environments. With an open architecture and a scalable, reliable infrastructure, it's changing how systems are integrated and how data will be handled in the future.
