A Kafka-Centric Communication Fabric for Near-Real-Time, Cloud-Replicated Closed-Loop Manufacturing Process Control
Smart manufacturing needs to move sensor data off the plant floor, react to it, and feed decisions back to actuators within bounded time. Programmable logic controllers (PLCs) handle fast, deterministic, safety-critical actuation, but they are not designed for the higher-level functions required by Industry 4.0, such as predictive maintenance, machine learning inference, and cross-facility analytics. These functions need a scalable, durable, and observable communication substrate. We present the communication architecture of a production system that provides this substrate and closes the loop back to the plant in near real time. The design is built from industry-standard components: Apache Kafka as the streaming backbone, OPC-UA for PLC connectivity, a relational time-series database for persistence, and JSON for serialization. The novelty is architectural. We show how these standards are integrated for closed-loop industrial control through four design choices: a single event stream per production line serves control, monitoring, machine learning, and durable recording, allowing one producer to serve many independent consumers; a protocol bridge converts polled OPC-UA traffic into publish/subscribe streams, aligns per-signal timestamps to a common time base to remove cross-signal jitter, and provides a symmetric actuation path; a transport technique carries sub-second process dynamics at a coarser publication cadence by packing timestamped samples into fixed-order arrays; and an edge-to-cloud replication scheme keeps the edge authoritative, so local control continues during wide-area network outages while cloud analytics operate on replicated data. We describe the loop latency budget, report measured broker transport latency, and discuss operational experience. The system provides soft, near-real-time behavior rather than hard real-time guarantees.