Source: https://gem-corp.tech/tech-blogs/artificial-intelligence/cloud-computing-role-in-edge-ai/
The proliferation of high-definition video cameras, car sensors, and Industrial Internet of Things devices creates enormous amounts of video data. Processing that data through deep learning algorithms for computer vision tasks creates a conundrum for system architects: should inference be done on local edge devices where processing power and cooling capacity are limited, or should the raw high-definition video be streamed to powerful centralized data centers where the sheer volume of data increases latency and consumes bandwidth? The answer lies in collaboration. Modern video analysis systems implement a mix of edge and cloud processing to analyze video data. Understanding how those systems work requires an overview of how modern deep learning video analysis architectures work. Modern computer vision systems use deep learning neural networks organized in a directed acyclic graph architecture. Some of the early layers in neural networks tend to do more generic image transformations and extract general features. Later layers become more specialized and are organized in task-specific ways. Edge-cloud collaborative systems exploit the differences in objectives between the early generic layers and late specialized layers. Those systems partition the neural network in a way that the edge node executes the early shallow layers of the neural network and offloads the execution of the later deep layers to the central data center. By only transmitting the results of the early layers (intermediate features) from the edge node to the data center, collaborative systems reduce bandwidth requirements dramatically compared to transmitting the entire high-definition video data stream. The intermediate features after shallow early layers tend to be lower-resolution or compressed representations of the original video data. Transmitting compressed data is much more efficient in terms of bandwidth and battery power. Video analysis systems that use edge-cloud collaboration must be able to dynamically adjust the network partitioning based on available resources. Bandwidth between the edge node and the data center is a shared resource that frequently fluctuates in capacity due to interference. In addition, the processing power of the edge node itself is a shared resource that may become heavily utilized when analyzing video from multiple edge devices simultaneously. When the available uplink bandwidth capacity declines, collaborative systems must re-partition the neural network to shift some processing from the data center to the edge node. The reverse is also true: when higher than average uplink bandwidth capacity is available, systems can shift more processing to the edge node to reduce the amount of data that must be transmitted. Partitioning the neural network between edge and cloud introduces a new set of challenges. The amount of data that must be transmitted depends on the characteristics of the video being processed. If the video data has lower resolution or contains less information, it requires fewer resources to process. There is little benefit to using large amounts of bandwidth to transmit full-resolution video frames to the data center if the neural network layers running on the data center have already downsampled the images for their processing. Collaborative systems use adaptive controllers and serverless functions to make adjustments to the way video data is processed dynamically. When the edge node detects that a given video frame does not contain useful information, it can reduce the amount of data it transmits to the data center. On the other hand, if the edge node detects that a given frame has interesting content or motion, it can preserve the quality of its intermediate features to avoid losing information. Edge-cloud collaboration enables systems to reduce the amount of data that must be transmitted from the edge node to the data center while improving the latency of overall processing. By partitioning deep learning workloads between multiple devices, computer science systems reduce network bandwidth consumption and remove bottlenecks at the central data center.
Sources
https://wands.hk/papers/TOSN_24_1.pdf https://cse.hkust.edu.hk/~kaichen/papers/runespoor-infocom21.pdf