What Determinism Means
A deterministic pipeline produces the same output given the same input, regardless of system load, time of day, or concurrent activity. The latency from ingestion to routing is bounded and predictable.
This matters because industrial systems make decisions based on data timing. A quality inspection system that receives data 500ms late sees a different picture than one that receives it in 10ms. A safety system that depends on ordered data cannot tolerate reordering.
Where It Breaks
Non-deterministic behavior enters through:
Shared resources. When pipeline execution shares CPU, memory, or I/O with other processes, contention introduces variable latency. A garbage collection pause in a JVM-based middleware can spike latency from 5ms to 500ms.
Cloud-dependent timing. Pipelines that route through cloud services for transformation inherit internet latency variance. A 20ms operation becomes 20-2000ms depending on conditions.
Polling without guarantees. Fixed-interval polling with no delivery guarantee creates gaps. If a poll cycle takes longer than the interval, data points are skipped silently.
Bounded Latency
Bounded latency means: the maximum time from data acquisition to pipeline output is known and guaranteed. Sub-10ms for most industrial workloads. This requires:
- Event-driven execution (react to data, not timers)
- Isolated process execution per module
- Local-only processing (no network dependency in the critical path)
- Resource isolation to prevent contention
Design Implications
Deterministic pipelines require a fundamentally different architecture than general-purpose data integration. The execution model must be event-driven with process isolation. Transforms must be pure functions. And the entire pipeline must execute locally at the edge - not through cloud round-trips.