The Intelligent Network Service Plane: Intent and Interaction (Working Toward a Smart Network and Smart Application Ecosystem)
Time 03/08/19 10:30AM-11:45AM
Network designs are evolving at a rapid pace toward programmatic control, driven in large part by the application of software to networking concepts and technologies, and the emerging concept of the network as a key subsystem in global scale systems, such as those serving major science collaborations that incorporate large scale distributed computing and storage subsystems. This software-network innovation cycle is important as it includes a vision and promise for greatly improved automated control, configuration and operation of such systems, in comparison to the labor-intensive network deployments of today. However, even the most optimistic projections of software adoption and deployment do not put network infrastructures on a path to realize what would be considered a truly smart or intelligent system from the application or user perspective, nor one capable of interfacing effectively with facilities supporting data analysis at sites located throughout the world.
We are now at the early stages of a networked systems technology and infrastructure inflection point. We use term network systems here to refer to the network infrastructure and the elements or subsystems that connect to and rely on the network. This transition is being driven by the maturation and convergence of several technologies whose development has been largely disjoint to date. Network designs are evolving at a rapid pace toward adopting automation and programmatic control. Artificial Intelligence/Machine Learning/Deep Learning technologies are evolving rapidly and are being applied to a wide variety of use cases. Big data systems and analytics technologies are readily available and being applied across many domains. These and supporting technologies will be integrated in unique ways to create truly autonomic, self-aware, smart networks. A smart networked ecosystem will enable the network to become an interactive component for use by similarly smart applications, security systems, and other domain specific workflow agents.
Today, domain science applications and workflow processes are forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. As a result, domain science applications frequently suffer poor performance, especially so in highly distributed environments. Indeed, the ability for a science application to interact and negotiate with network infrastructure of a science ecosystem, should be a hallmark of truly smart networks and smart applications. It seems clear that current static, non-interactive network infrastructures do not have a path forward to assist or accelerate domain science application innovations.
We therefore envision a new smart network and smart application ecosystem that will solve these issues, and enable future innovations across many Research and Education domain science communities.
In this session we focus on an Intelligent Network Service Plane which forms the boundary layer between the smart network and the smart application. This is where the application workflow agents can engage and obtain services from the smart networked ecosystem. In this context, the following key features of Intelligent Network Service Plane are emphasized in this session:
• Intent - The ability for an application to submit a service request in the form of a high-level statement of desired results or outcomes, as opposed to a specific set of network centric inputs.
• Interaction - The ability for an application workflow agent to engage in a "conversation" with the network as part of workflow planning. This conversation can include discovery of available services, asking "what is possible" or "what do you recommend" types of questions, engaging in iterative negotiations prior to actual service requests, or full-service life-cycle status and troubleshooting queries.
We believe that the Intelligent Network Service Plane will enable multiple new operational paradigms, including a fundamentally new concept of “consistent network operations”. This is where stable load balanced high throughput workflows cross optimally chosen network paths, up to preset high water marks to accommodate other traffic. This will be provided by automated interaction between the application workflow agents and the Intelligent Network Service Plane, in response to demands from the science programs’ principal data distribution and management systems. The result will be a “consistent outcome” and "deterministic" (or more deterministic) end-to-end system performance.
This session will include three presentations describing Intelligent Network Service Plane development status as well as the applications using this approach:
• The SENSE (SDN for End-to-end Networked Science at the Exascale) project developed Intelligent Network Service Plane will be presented, along with the associated underlying SDN and smart infrastructure. An example application workflow agent based on a Large Hadron Collider (LHC) Compact Muon Solenoid (CMS) use case will be demonstrated utilizing the Intent and Interaction feature sets to optimize workflow operations. A specific CMS data analysis use case aimed at agile distribution and caching of datasets using Named Data Networking, developed in the companion DOE/ASCR SDN NGenIA project and the NSF-funded SANDIE project also will be presented.
• The BigData Express high-performance data transfer system will be described and demonstrated along with its interaction with the Intelligent Network Service Plane to optimize operations and provide a more deterministic experience for its users.
• The ExaFEL project plans to stream multi-gigabits per second from the electron laser instrument in SLAC to supercomputing centers at different DOE labs. It’s approach on integrating with the Intelligent Network Service Plane, and use of intent, will be described to share how science experiments can benefit from this approach
The DOE Office of Science funded SENSE (sense.es.net) Project collaborative team is led by Energy Sciences Network (ESnet) and includes collaboration with Lawrence Berkeley National Laboratory (LBL), Fermilab (FNAL), Argonne National Laboratory (ANL), Caltech, and the University of Maryland/Mid-Atlantic Crossroads (UMD/MAX). This project is building smart network services to accelerate scientific discovery in the era of ‘big data’ driven by Exascale, cloud computing, machine learning and AI.
The DOE Office of Science funded BigData Express project (bigdataexpress.fnal.gov) collaborative team includes Fermilab and Oak Ridge National Laboratory. BigData Express is developing a schedulable, predictable, and high-performance data transfer service for DOE’s large-scale science computing facilities and their collaborators. The BigData Express team is collaborating with StarLight International/National Communications Exchange to deploy BigData Express at various research platforms. It is envisioned that we are working toward building a multi-domain and multi-tenant software defined infrastructure (SDI) for high-performance data transfer.
Session Flow (Total Time: 75 Minutes)
• SENSE (25 minutes)
The SENSE Architecture and the building of the Intelligent Network Service Plane will be described. Example use cases based on LHC/CMS use of the Intelligent Network Service Plane will be demonstrated
Inder Monga, ESnet (DOE Office of Science - Energy Sciences Network), email@example.com
Tom Lehman, University of Maryland - College Park, firstname.lastname@example.org
Harvey Newman, Caltech (California Institute of Technology), email@example.com
Xi Yang, University of Maryland - College Park, firstname.lastname@example.org
• Big Data Express (25 Minutes)
Topics Covered: Distributed Data Movement System and Deployment and its use of the Intelligent Network Service Plane will be described and demonstrated.
Wenji Wu, Fermi National Accelerator Laboratory (FNAL),
Philip DeMar, Fermi National Accelerator Laboratory (FNAL), email@example.com
Joe Mambretti, Northwestern University, firstname.lastname@example.org
Qiminu Liu, Fermi National Accelerator Laboratory (FNAL), email@example.com
• ExaFEL: Data Analytics at Exascale for Free Electron Lasers (15 minutes)
The Linear Coherent Light Source (LCLS) is revealing biological structures in unprecedented atomic detail, helping to model proteins that play a key role in many biological functions. The results could help in designing new life-saving drugs. This project analyzes in real time the throughput of data generated by the next generation free-electron lasers. Higher throughput allows the examination of the time-resolved behavior of dynamic reactions triggered by optical pulses, chemical mixing, or temperature jump. This talk will describe how SENSE is being used to accomplish that goal
Chin Guok, ESnet (DOE Office of Science - Energy Sciences Network), firstname.lastname@example.org
• Comments, Questions (10 Minutes)
Speaker Chin Guok ESnet (DOE Office of Science - Energy Sciences Network)
Speaker Wenji Wu Fermi National Accelerator Laboratory (FNAL)
Speaker Philip DeMar Fermi National Accelerator Laboratory (FNAL)
Speaker Joe Mambretti Northwestern University
Speaker Xi Yang MAX (Mid-Atlantic Crossroads)
Speaker Thomas Lehman University of Maryland - College Park
Speaker Harvey Newman Caltech (California Institute of Technology)
Speaker Indermohan Monga ESnet (DOE Office of Science - Energy Sciences Network)
Primary track Advanced Networking