Distributed Messaging System for the IoT Edge

Recent advances in Deep Learning have made possible distributed multi-camera IoT vision analytics targeted at a variety of surveillance applications involving automated real-time analysis of events from multiple video perspectives. However, the latency sensitive nature of these applications necessitates computing at the Edge of the network, close to the cameras. The required Edge computing infrastructure is necessarily distributed, with Cloud like capabilities such as fault tolerance, scalability, multi application tenancy, and security, while functioning at the unique operating environment of the Edge. Characteristics of the Edge include, highly heterogeneous hardware platforms with limited computational resources, variable latency wireless networks, and minimal physical security. We postulate that a distributed publish-subscribe
messaging system with storage capabilities is the right abstraction layer needed for multi-camera vision Edge analytics.
We propose Mez - a publish-subscribe messaging system for latency sensitive multi-camera machine vision at the IoT Edge. Unlike existing messaging systems, Mez allows applications to specify latency, and application accuracy bounds. Mez implements a network latency controller that dynamically adjusts the video frame quality to satisfy latency, and application accuracy requirements. Additionally, the design of Mez utilizes application domain specific features to provide low latency operations.
In this dissertation, we show how approximate computation techniques can be used to design the latency controller in Mez. We also present the design of Mez by describing its API, data model and architecture. Additionally, Mez incorporates an in-memory log based storage that takes advantage of specific features of machine vision applications to implement low latency operations. We also discuss the fault tolerance capabilities of the Mez design.
Experimental evaluation on an IoT Edge testbed with a pedestrian detection machine vision application indicates that Mez is able to tolerate latency variations of up to 10x with a worst-case reduction of 4.2% in the application inference accuracy. Further we investigated two approximate computing based algorithms - a heuristic based
pruning algorithm and a Categorical boost machine learning model based algorithm, to make the Mez’s latency controller design scalable. Both algorithms were able to achieve video frame size reduction upto 71.3% while attaining an inference accuracy of 80.9% of that of the unmodified video frames.