European trajectories to support climate data and climate data analytics: infrastructure and best practices
Time 07/15/14 04:20PM-04:40PM
From the European perspective an overview is given of the efforts to support climate data access and evaluation.
We concentrate on the approaches taken at BADC and DKRZ as well as IPSL and put them into the context of the emerging European infrastructure efforts supported by e.g. IS-ENES and EUDAT.
BADC, DKRZ as well as IPSL provide high-capacity data analysis facilities to end users
based on peta-byte range disk storage facilities. Data provisioning to end users is supported by the ESGF infrastructure as well as legacy systems which are partly integrated into ESGF (e.g. the DKRZ long term archive).
Data analysis is supported by growing compute clusters attached to the data storage facilities. Various pre-installed tools as well as virtualization infrastructures help end users to exploit the compute resources for their analysis tasks.
To expose data center near analysis functionality current developments are based on the OGC Web Processing Services (WPS) standard. Several European portals are under development making use of WPS services to support the climate impact community.
A first step to integrate European data centers into a trans-community infrastructure is taken within the EUDAT project. Data replication services as well as data storage and metadata services help communities to make use of the distributed storage facilities.
From a networking perspective the future application scenarios exploiting these resources and infrastructures pose unprecedented data exchange and data access challenges.
Based on the experiences with CMIP5 data access, replication, and archival we try to characterize the steps necessary at the individual centers to support the future distributed climate data usage scenarios to be supported by a flexible high performance networking infrastructure.