Table of Contents
What Is Edge AI? The Difference Between Edge AI and Cloud AI
Edge AI runs AI models and inference close to where data is generated, such as on smartphones, cameras, industrial equipment, vehicles, or edge servers, instead of sending all data to a remote cloud for processing.
A practical example is building an Edge AI platform with the ET2508 and an AI accelerator card. A user sends a request from a mobile device, and the local AI agent on the router analyzes real-time network conditions such as traffic patterns, latency, queue depth, and packet loss. Based on this analysis, the router can proactively provide insights to the user, such as: “One of your devices experienced 50 retransmissions in the last hour due to abnormal protocol handshake failures.“
Cloud AI, in contrast, runs AI training, inference, data processing, and model management on cloud infrastructure. Users access these capabilities over the network without deploying their own compute resources or AI platform locally. For example, when you open ChatGPT on your phone and submit a prompt, the request is sent to a cloud-hosted model, and the response is returned to your device. This is a typical Cloud AI scenario.
The difference becomes clear when comparing their workflows:
- Cloud: Device → Cloud → AI Processing → Response
- Edge: Device → Edge Node → Local Processing → Cloud (if needed)
The fundamental difference lies in where AI inference takes place. Edge AI performs inference locally, while Cloud AI performs inference in centralized cloud infrastructure. As a result, Edge AI is well suited for real-time applications, while Cloud AI is better for centralized computing resources and unified management.
Building an Edge AI platform requires more than AI models and accelerator cards. Edge deployments are usually distributed across many locations, involve a large number of nodes, and carry diverse traffic types. They must also support telemetry, model invocation, policy distribution, and cloud-edge collaboration at the same time.
In other words, the performance of an Edge AI platform depends on more than compute performance. It also depends on how reliably data can be transported, how quickly the network can react, and how easily the infrastructure can scale. This is where the network foundation becomes critical, and SONiC is one of the key technologies that enables these edge networking capabilities.
Why Open Networking Matters for Edge AI
To enable dynamic awareness and scheduling for high-performance Edge AI, a closed-loop workflow is required to translate AI inference decisions into network forwarding policies. From a networking perspective, the key lies in the decoupling and coordination between the control plane and the data plane.
- Control Plane – SONiC as the Policy Engine: As a leading open networking platform, SONiC acts as the orchestrator for compute-network integration. It uses standard protocols such as BGP and EVPN to build flexible network topologies. More importantly, it serves as the control plane that hosts policy logic from AI agents. After the AI model analyzes traffic characteristics and makes a decision, SONiC converts the decision into precise configurations, such as QoS and policing policies, enabling fine-grained traffic management.
- Data Plane – High-Performance Forwarding with DPDK and VPP: If SONiC is the orchestrator, DPDK and VPP are the high-speed execution engines. In demanding edge environments, VPP uses its user-space Vector Packet Processing (VPP) architecture together with DPDK’s high-performance packet I/O framework to handle data forwarding directly. It can respond to policies distributed by SONiC within microseconds, performing packet re-queuing, priority scheduling, and rate limiting to ensure that AI-driven decisions are enforced in real time on the physical network.
Built on SONiC, DPDK, and VPP, AsterNOS-VPP does not run AI workloads directly. Instead, it provides an open and programmable networking foundation for connecting, operating, and managing Edge AI infrastructure.
Hardware Acceleration in Edge AI Platform
The core challenge of Edge AI lies in the inherent limitations of edge hardware: restricted CPU resources, stringent power budgets, and the extreme demand for real-time inference. Traditionally, relying solely on a CPU to run AI models is insufficient. It leads to slow inference, high latency, and excessive power consumption, ultimately causing thermal throttling or severe performance bottlenecks.
To overcome these constraints, we must introduce AI Accelerators. By offloading heavy mathematical computations from the CPU to a dedicated inference engine, system performance leaps forward: TOPS (Tera Operations Per Second) capacity can reach up to 160 TOPS, inference latency is significantly reduced, and efficiency per watt is dramatically improved.
The M.2 Key: The “Compute Extender” for the ET2508
When building compact Edge AI platforms, the M.2 Key interface serves as the optimal solution for integrating AI compute power. The architecture logic is shown as below:
It is important to clarify that the M.2 interface itself is not the accelerator; rather, it is a highly standardized and compact physical interconnect that bridges the accelerator card to the system via the PCIe bus.
This architecture provides edge devices with four core advantages:
- Compact Form Factor: Designed for space-constrained environments, it integrates seamlessly into edge gateways or access devices without requiring cumbersome external boards.
- Easy Deployment: The plug-and-play nature of the interface drastically simplifies hardware assembly and field integration.
- Upgrade Flexibility: As business requirements evolve, operators can simply swap the AI module for a higher-performance specification without needing to replace the underlying network platform, effectively “future-proofing” the hardware.
- Low Power Consumption: Modern AI modules are specifically optimized for edge scenarios, ensuring high-performance computing while remaining well within the thermal and power envelopes of edge deployments.
The Synergy of AI Accelerator and SONiC: In our architecture, the AI accelerator and SONiC form a perfect complement of “compute” and “network.” The AI accelerator acts as the “computing brain,” efficiently processing inference models via the M.2 interface, while SONiC serves as the “network commander,” responsible for receiving these decision instructions in real-time and rapidly adjusting network policies through the underlying DPDK/VPP.
This combination thoroughly breaks the compute boundaries of traditional edge devices, allowing them to not only “compute fast” but also “connect stably,” truly enabling the agile deployment of intelligent inference at the network edge.
Edge AI Applications with Asterfusion ET Platform
Leveraging the deep synergy between the AsterNOS (enterprise-grade SONiC) control plane and the high-performance DPDK/VPP data plane, combined with the M.2 AI acceleration interface extended on our DPU, our platform provides robust “Compute-Network Integration” support for various edge scenarios—such as real-time network status monitoring and fault troubleshooting. Download to view.
As shown in the demonstration, users can instantly retrieve core network operating metrics through natural language interaction with an AI Agent. When encountering issues like network latency, there is no need to manually log into devices to check logs line by line. Instead, users can simply issue commands directly to the Agent:
- Real-time Traffic Insight: “Can you check the real-time traffic rate on switch [IP Address]?” — The Agent can directly penetrate to the DPU layer to retrieve monitoring data in real-time, providing accurate feedback on throughput and queue utilization.
- Millisecond-level Fault Localization: When a user reports, “We are experiencing some network latency,” the Agent quickly correlates current traffic load, policy execution status, and link health to immediately pinpoint where congestion is occurring or which policy is interfering with business operations.
By connecting SONiC’s flexible scheduling, VPP’s fast forwarding, and AI’s decision-making capabilities, the ET platform not only solves performance bottlenecks in individual nodes but also reshapes edge intelligence from isolated computing tasks into a new, self-aware, manageable, and automatically optimized form of infrastructure through a “compute-network synergy” model.
FAQ
1. Can AsterNOS-VPP Be Considered Edge AI Software?
Answer: AsterNOS is positioned as an AI-enabled network operating system, rather than a general-purpose AI training or inference framework such as PyTorch. In an Edge AI architecture, it serves as the control and orchestration plane. By providing an open and programmable networking foundation, it translates AI inference decisions generated by external AI accelerator modules into real-time network forwarding policies, such as QoS and traffic shaping.
2. Why Does Edge AI Need an Open Networking Stack Like SONiC?
Answer: Edge AI deployments are highly distributed and involve complex traffic patterns. Without a unified control framework, AI-driven decisions cannot be enforced on the physical network. SONiC provides a standardized control plane, including protocols such as BGP and EVPN, ensuring network manageability in distributed environments. Its QoS framework also enables efficient execution of policies generated by AI agents, turning computing intelligence into network performance.
3. What Are the Unique Advantages of Edge AI Over Cloud AI in Industrial Scenarios?
Answer: The primary advantages are real-time response and privacy/compliance. In industrial scenarios, such as PLC protocol diagnostics, network latency and data privacy requirements often make cloud-based processing impractical. Edge AI can use lightweight models to process real-time traffic locally and identify protocol anomalies, such as handshake retries. This avoids data exposure risks and enables fault detection within microseconds, without requiring a persistent Internet connection.
4. Why Is an AI Accelerator (M.2 Key) Necessary in Edge Devices?
Answer: Traditional edge devices rely on the CPU to handle both network forwarding and AI inference, which can create significant performance bottlenecks, including higher latency and power consumption. By introducing a dedicated accelerator through the M.2 interface, compute-intensive matrix operations can be offloaded from the CPU, enabling the separation of compute and networking functions and significantly improving real-time processing efficiency per watt (TOPS/Watt).
