Workflow Adaptation on Networked Cloud Infrastructures Using Intent and Performance Models
Time 10/16/17 10:20AM-11:10AM
We will present a dynamically adaptable networked cloud infrastructure for scientific workflows. The dynamic management of networked resources will be driven by an intent framework coupled with the Aspen performance models, and orchestrated by the Pegasus workflow management system. We plan to show active migration of workflow executions on dynamically provisioned slices across multiple GENI/ExoGENI racks interconnected by Internet2/ESnet by modifying virtual topologies on-the-fly. The migration will be triggered by an anomaly detection framework that will perform a callback to the intent system when performance anomalies are observed. The intent system will then explore alternate resources by consulting user choices, performance models and the workflow manager. We will showcase the above capabilities using an integrated framework developed in the DoE Panorama project, which plans, executes, monitors, and re-configures workflow executions on dynamic infrastructures. We will use a representative, data-intensive workflow from genome sciences as a driving use case.
Anirban Mandal, Paul Ruth, Ilya Baldin, Rafael Ferreira da Silva, Ewa Deelman, Vickie Lynch, Jeffrey Vetter, Chris Carothers, Mariam Kiran, Brian Tierney
Speaker Paul Ruth University of North Carolina - Chapel Hill
Speaker Anirban Mandal University of North Carolina - Chapel Hill
Primary track Applications for Research
Secondary tracks Advanced Networking