Streaming Inference Pipeline

What is a streaming inference pipeline?

‍A streaming inference pipeline is a streaming application that makes real-time, non-interactive predictions triggered by the arrival of an event and outputs predictions to some sink (such as the event bus).

‍When should I use a streaming inference pipeline?

‍If you have a non-interactive system that needs to make predictions in near real-time, then a streaming inference pipeline might be the best choice. For example, an IoT device might stream measurements to a Kafka topic, from where a streaming inference pipeline makes continual predictions about the IoT measurements and writes the results to different Kafka topic, from where an alerting component consumes events and takes actions if needed. In contrast, an online inference pipeline is a request/response architecture for interactive systems.