Edge Computing Embraces AI-GPGPU-based System Performance

Author: Dan Mor, Director, Video & GPGPU Product Line

Today’s military systems are built with far more functionality in a much smaller footprint, typically referred to as optimized SWaP—size, weight, and power—while still needing to keep costs low. In addition, these systems function in extremely harsh environments and carry with them the need to operate reliably all the time, every time.

These rugged HPEC (high-performance embedded computer) systems not only support crucial, lifesaving, and security-focused applications, but must withstand extreme shock and vibration as well as severe and expansive temperature and humidity fluctuations, ranging from sub-zero to triple digits.  And they need to capture and process real-time data and graphics from several inputs simultaneously and manage it all from many I/O interfaces, providing what is known as “AI at the Edge.” (Figure 1) 

Data Challenges in Military Computing

Balancing all these requirements pose challenges, but with the right ruggedization and system development techniques, embedded designers are developing AI-based supercomputers used throughout several military applications, such as situation awareness systems, EW systems, and drones as well as a smart soldier and man-portable systems and augmented reality.

Especially noteworthy is the growing field of autonomous operations utilizing this powerful technology including UAVs (unmanned aerial vehicles) and unmanned ground vehicles (UGVs). These real-time response applications require this AI at the edge, where processing that is happening at the sensors is exponentially increasing computing requirements. (Figure 2)

But AI at the edge is not contained to only unmanned applications.  Security risks within the general in-the-field military computing applications make cloud computing a cause for concern.  Large data sets in other non-critical industries are typically sent to the cloud for computation and processing, but for applications handling sensitive, security-laden information, this less secure, cloud-based infrastructure may not be a viable solution.  This pushes the need for computing to the edge into a wide range of military applications.

The Power of Parallel Processing

As data needs continue to increase across the board, modern embedded systems are faced with some serious performance issues: continuing to only use a CPU as the main computing engine would eventually choke the system. Divesting the highly demanding data calculations to the GPU, while allowing the rest of the application to run on the CPU, helps balance system abilities and resources more effectively.

By combining a GPU with a CPU to accelerate applications, GPU-accelerated computing offloads some of the computation-intensive portions from the CPU to the GPU. Using a GPU with a parallel architecture instead of a CPU, which is serial, reduces development time and “squeezes” maximum performance per watt from the computation engine.

GPU accelerated computing runs compute-intensive portions on the GPU to accelerate the compute capabilities of a system, using less power and delivering higher performance over a CPU. Through this increased power-to-performance ratio, GPU-based systems can meet the exorbitant calculation demands these applications now require.

The specialized parallel computational speed and performance of GPGPUs enables system designers to bring exceptional power and performance into rugged, extended temperature small form factors (SFFs).   This has made AI and its subset of deep learning, a far more useful tool in military embedded systems.

AI is A DoD Priority

The United States Department of Defense (DoD) started an AI initiative that aims to bring AI and computer algorithms to the battlefield.

The following insights are outlined in the “DoD Unmanned Systems Integrated Roadmap 2017-2042”:

• Autonomy

“Advances in autonomy and robotics have the potential to revolutionize warfighting concepts as a significant force multiplier. Autonomy will greatly increase the efficiency and effectiveness of both manned and unmanned systems, providing a strategic advantage for DoD.”

• Human-Machine Collaboration

“If interoperability lays the foundation, then human-machine collaboration is the ultimate objective. Teaming between human forces and machines will enable revolutionary collaboration, where machines will be valued as critical teammates.”

Applications of AI in Today’s Military Systems

Think about this in the context of bringing AI to the defense drone industry. Drones are used for intelligence, surveillance, and reconnaissance and usually record many hours of footage every mission—video footage that takes a long time for human analysts to analyze. While human analysts process footage, the ground situation may change, and latency between analyzed footage and real-time battle conditions is presented.

Another example is eVTOLS, where pilotless prototypes are being rapidly developed, with many utilizing platforms that already exist, such as drones or unmanned helicopters, then integrating leading-edge technologies to achieve the needed functionality of these air transport vehicles.   The need for advanced features and increased computing functionality is being met by AI supercomputing.  (Figure 3)

Compact, High-powered Processing Upgrades Are Needed

The DoD roadmap also details the challenge of upgrading and maintaining antiquated systems, which often suffer from inferior capabilities as well as the need for manual data analysis. Removing these barriers and embracing a trusted AI GPGPU-based computing infrastructure will help alleviate some of the challenges associated with managing the volume, variety, and veracity of data within today’s military embedded systems

Using AI technology (deep learning process) would enable the processing of much more data within the same timeframe, which will bring situational awareness to “near real-time” status.  And with some AI-based SWaP-optimized supercomputers offering an ultra-compact footprint, roughly the size of a cell phone, unmanned systems can achieve incredibly high performance with remarkable levels of energy efficiency. They can operate longer during a mission, offer better reliability and provide real-time data inputs by processing and transmitting that data back to the main command center or even building upon existing vehicle platforms to extend the function of a single vehicle, effectively extending the amount of airspace one craft can cover.

Embedded AI Requirements for Military Applications

It has been established that the embedded AI defense market requires SWaP- and SWaP-C-optimized systems. This means that in addition to managing AI computing at the edge, next-generation AI system will also need to support:

• SFF Platforms (i.e. based on NVIDIA Jetson Xavier NX and Nano SoM) for “Image Classification, Image Location, Image Segmentation” processes in a growing number of in-the-field applications
• SWaP-optimized Systems – Low power consumption modules with compact footprints and exceptionally high processing capabilities
• NVIDIA CUDA – A common platform framework where the user can share already-trained deep learning models between devices
• Different Use Cases – Easily covered by simply retraining the model with different data sets
• Detection Precision – Constantly approved by fault detection, adding additional images to the data set and retraining the model
• Provisions for Next-generation Systems – As more powerful GPUs (embedded SoM) become available, better performance will equate to more complex and precise detection

Ensuring Reliable Edge Computing

The intuitive processing that AI provides has propelled many modern military systems into this new area of high-intensity computing using real-time data, enabling autonomous decision-making. AI in military systems is fueled by GPU accelerated computing, which uses parallel processing versus serial, to enable the handling of thousands of data points simultaneously.

Ruggedization of these HPEC systems means that computing power can be used in a wider number of remote and mobile locations. As processing demands increase, SFF (small form factor) systems using GPGPU (general-purpose graphics processing unit) technology and AI-based solutions are providing a path to next-generation embedded systems, poised to tackle the growing field of mobile, unmanned and autonomous vehicle technologies.