The AI at the edge of the network rarely only means inference. Real world deployment typically involves high-speed input/output (I/O), signal conditioning, and real-time control loops, all of which are executed concurrently. These multifunctional workloads require tight coordination and high certainty, and designers have found it difficult to meet these requirements using mainstream AI hardware.
Two factors make this problem even more complex. Firstly, AI models are developing at an astonishing pace, prompting designers to adopt platforms that support rapid algorithm updates. Meanwhile, many edge systems have been in use on-site for up to ten years or longer, making it difficult to ensure long-term adaptability. Secondly, the path from well-trained models to system deployment and implementation is still fragmented. Data scientists use PyTorch and TensorFlow, while embedded teams use completely different toolchains, which creates friction during the handover process and slows down the production speed.
To address these challenges, platforms need to be able to combine high-throughput AI processing with deterministic behavior, flexible I/O, and long-term adaptability, all of which must be achieved within the typical power consumption range of limited edge deployment.
This article focuses on the application scenarios and related requirements that challenge designers to explore new edge AI architectures. Then, it introduced Altera's field programmable gate array (FPGA) devices and software tools that support edge AI, and demonstrated how to utilize them to meet the diverse performance and power requirements of these applications.
The evolution of edge AI requires architectural innovation
Edge systems are increasingly adopting diverse AI technologies, including classical machine learning (ML) for anomaly detection, convolutional neural networks (CNN) for perception, and converters for large language models (LLM). These computationally intensive algorithms often coexist with demanding non AI functions such as signal processing, network communication, and real-time control.
Autonomous systems are a good example. They typically need to capture data from multiple sensor modalities such as video, audio, radar, LiDAR, and motion/position feedback, preprocess these data streams with high throughput, analyze the results using complex AI, and then manage high-precision control loops, all of which require reliable determinacy.
There are many similar examples in industrial automation, medical imaging, defense, and telecommunications applications. A common challenge they face is that traditional architectures are difficult to adapt to constantly converging workloads.
Why FPGA is particularly suitable for edge AI
In contrast, these requirements are fully compatible with the functionality of FPGA. The core of FPGA is to provide configurable logic to perform operations in a truly parallel manner, with its timing behavior embedded at design time rather than fluctuating at runtime. This architecture can achieve low latency determinism, which is crucial for edge AI. Flexible logic can also utilize powerful I/O: FPGAs typically provide abundant high-speed I/O, which can be connected to various sensors and actuators to achieve tight coupling with AI processing.
FPGA also includes distributed internal memory, which enables data to be accessed by the logic that operates on it. This reduces the bottleneck that arises when multiple processing stages must compete to access the shared memory bus, which is a common limitation in processor based architectures.
Many FPGAs also integrate specialized digital signal processing (DSP) hardware. Compared to conventional structures, these enhanced circuits provide higher performance and better energy efficiency for signal processing workloads. Some FPGAs also integrate hard wired processor systems that can run standard software stacks (including Linux), enabling traditional software development for tasks such as networking, device management, and user interface.
In short, a single FPGA can integrate functions that may otherwise require separate I/O chips, AI accelerators, DSPs, and control plane processors. This can reduce the Bill of Materials (BOM), shrink the circuit board area, lower power consumption, while maintaining the low latency and certainty required for edge AI applications.
How to open up new possibilities with the addition of AI tensor blocks
Traditional FPGA DSP hardware is already very suitable for many edge workloads, but AI inference often relies on dense but low precision multiplication operations. To address this issue, Altera's Agilex 3 and Agilex 5 devices use enhanced DSPs with AI tensor blocks. This is specialized hardware for matrix matrix and vector matrix multiplication, which repeatedly appears in AI computation graphs.
The core of this method is scalar product and adder/accumulator engine (Figure 1). In tensor mode, the hard wired point engine uses 8-bit input and preloaded 8-bit weights to perform a 10 element dot product. In order to expand the dynamic range, the data path can also use a shared "common index" for block floating point scaling to cope with typical scenarios where AI inference typically requires high dynamic range but low accuracy.