High Speed ISR Sensor Data Capture, Storage and Retrieval with the New HSR40

When originally introduced into service, UAVs such as the MQ-1 Predator would deploy airborne intelligence, surveillance, and reconnaissance (ISR) systems containing a handful of cameras and other sensors. These sensor systems generated several gigabytes of data throughout their mission, which was typically captured by a single Network Attached Storage (NAS) device. As UAV platforms have evolved, so have their ISR sensors. Modern ISR UAVs, such as the MQ-9 Reaper, may have hundreds of cameras and other sensors on board generating terabytes of data that must be captured, stored, and retrieved for later analysis. Earlier NAS devices are unable to record the new ISR data being generated; modern ISR systems cannot be slowed down or paused to let the old recorders catch up with the flow of data.


Previously, designers of high bandwidth data recorders would turn to the high-speed, low-latency Serial Front Panel Data Port (SFPDP) communications protocol, which could handle approximately 1.6 GB of data per second using multiple channels. The good news is that recent advances in processor architectures and solid-state drives (SSD) are helping recorder designers keep up with the expanding rivers of sensor data.

Now, thanks to the introduction of built-in 40 gigabit (40GbE) interfaces on digital signal processor (DSP) boards with powerful Intel Xeon-D processors, and the advent of a new class of superfast Non-Volatile Memory Express (NVMe) SSDs, it is possible to design digital data recorders that can absorb upwards of 6 GB per second of streaming data. This is 400% more bandwidth than the fastest SFPDP recorder could handle, creating new possibilities for designers of ISR systems.

To handle the increasing amounts of sensor data, Curtiss-Wright developed the High Speed Recorder – 40 GbE (HSR40) which utilizes modern technologies, namely the CHAMP-XD2 Intel® Xeon® D processor-based digital signal processor (DSP) module, to handle large amounts of data at high-speed and NVMe memory to store the data. Handling the incoming sensor data is a new class of OpenVPX DSP boards featuring dual Xeon-D processors and two built-in 40 GbE interfaces. (See example system diagram in Figure 1.) These modules can easily absorb the incoming data and redirect it to the persistent storage media, via PCIe Gen 3, to enable the most efficient movement of the data. In the HSR40, each 40 GbE interface is supported by 16 PCIe lanes capable of handling more than 6 GB per second of incoming data. The Xeon-D’s 12 cores run the recorder application under Linux, moving the data through the IP stack and over to the solid state storage. While the storage interfaces are all PCIe Gen 3, in this example x16 lanes are used.

In previous generations of recorders, the data was converted from PCIe to SATA; the new generation of recorders uses NVMe SSDs to eliminate that system bottleneck. NVMe memory is widely supported today, with laptops driving demand for increased performance. In our HSR40 example, using two sets of x16 PCIe lanes, the data is efficiently moved to two removable NVMe storage modules. Using the M.2 format, the storage module can house from 32 TB to 64 TB capacity today. While the SSD capacity can also be scaled down, most modern sensor applications push for more storage, not less.

This new class of 40 GbE-based data recorders delivers significant improvements compared to the former top-of-the-line Serial Front Panel Data Port (SFPDP)-based recorders. For example, if the SFPDP recorder had data on eight channels, each running at 200 MB per second, its total throughput would be 1.6 GB per second. To handle a 6.4 GB data stream would require 32 data channels, so four separate SFPDP recorder units would need to be deployed on the ISR platform, significantly adding to SWaP-C [size, weight, power, and cost].