+40Gbps Ultra-High-Speed Data Transfer and Processing
Time 10/29/14 03:30PM-04:15PM
One major challenge in high-performance computing is being able to move big data in and out of the data center. While high-performance servers and hardware are already deployable inside the data center, and WAN bandwidth can be provisioned beyond multi-Gbps, existing transport technology cannot fully utilize the end-to-end capacity, particularly over a wide area. To address this need, Aspera and Intel investigated ultra-high-speed (40 Gbps and beyond) data transfer solutions built on Aspera’s FASP transport technology and the Intel® Xeon® processor E5-2600 product family.
In Phase I of this investigation, the team achieved predictable 10 Gbps WAN transfer speeds on both bare metal and virtualized hardware platforms, over commodity Internet connections with hundreds of milliseconds of round-trip time and several percentage points of packet loss characteristic of typical global-distance WANs.
Phase II of the investigation focused on an experimental integration with Intel® Data Plane Development Kit (Intel® DPDK), which made it possible to directly control the network interface controller (NIC), thereby bypassing the kernel networking stack. This integration allowed Aspera to overcome the packet processing bottleneck for single-stream transfers, minimizing CPU, memory, and I/O bottlenecks, to achieve 40Gbps on the same commodity hardware.
The ability to achieve such high speeds—on both bare metal and virtualized hardware systems that are low-cost and off-the-shelf—could dramatically reduce data center hardware cost and footprint and the associated power and cooling costs and more easily scale up transfer capacity for large-scale data processing.
Speaker Michelle Munson Aspera
Primary track Advanced Networking/Joint Techs