OpenFlows based Adaptive Fabrics for Big Data Clusters Netcast Archive
Big Data clusters put a large array of distinct and varied workloads on Ethernet Interconnects. By using OpenFlows technology, we are looking into the usability of a network of switches that could change their topology as well as buffering and bandwidth settings on-the-fly, without any user interaction. By taking instructions from the running application, job-tracker, or even the database management itself, topologies can change based on what the most efficient cabling solution is at that moment, for that job. We are looking into having multiple topologies defined by their L2 forwarding tables on top of each depending on how many jobs are running on the same system at the same time. This technology could greatly improve the performance and latency figures of Ethernet and Big Data today and for the future. Imagine a network that custom sizes itself based on what the application needs.