Session Abstract

Scientific research has been slower to adopt cloud technology than industry. However, new platforms are making it easier for scientists to leverage the flexibility and scalability offered by the cloud for exploratory research, particularly around big datasets. Participants from Clemson, Columbia, and University of Washington will, along with industry members Google and Omnibond, will explore some of the opportunities and challenges associated with cloud-based scientific research.

Part 1:

In this talk we will discuss various computational workflows that can be seamlessly routed to on premise or cloud computational resources. We will discuss the costs and benefits of cloud enabling HPC workflows.

Part 2:

Scientific reproducibility is a crucial part of maintaining an open, peer-reviewed academic research culture.Open source cloud technologies can With developments in the cloud, it provides tools to make scientific reproducibility of research results easy and affordable through technologies such Docker, Kubernetes, TensorFlow for machine learning, common user identity, Jupyter notebooks, Binder, and public, cloud-hosted data sets.

Here we’ll introduce some new platforms which combine such tools into complete environments for research, including Kubeflow-- a portable and scalable open-source machine learning stack built on Kubernetes that makes is easy to deploy machine learning pipelines on laptops, on-premises resources, and managed cloud services-- and Pangeo-- a Jupyter-based environment that incorporates open source tools like xarray, dask, and zarr to provide highly-scalable interactive analysis of large, complex datasets. Attendees will learn how the University of Washington and Columbia University are working with Google and other cloud providers to develop cloud architectures and tools to enhance the scientific process.