Internet2 Fall 2011 Member Meeting

Use Internet2 SiteID

Already have an Internet2 SiteID?
Sign in here.

Internet2 SiteID

Syndicate: A simple filesystem for scaleable data distribution

Time 10/03/11 03:00PM-04:00PM

Session Abstract

Syndicate is a global filesystem optimized to deliver large files (e.g,. scientific data sets, library collections, videos) to a scaleable number of clients distributed throughout the world. From the user's perspective, Syndicate offers an intuitive drag-and-drop interface (similar to Dropbox) through which data producers can make large files (and directories of files) available to a scalable set of clients, and data consumers can retrieve large files generated by one or more remote sites. Underlying this familiar interface is a scalable content delivery network, based on CoBlitz, a widely used CDN. Files written (published) to Syndicate are broken into chunks and distributed to a global set of high-performance caches. Files read from Syndicate are then pulled, in parallel, from these distributed caches, reassembled into their original form, and made available for local processing. Syndicate also maintains and distributes a file directory hierarchy, thereby supporting a full file system interface.

Syndicate (and the underlying CDN that it exploits) has application to both the scientific community that that has large data sets to distribute, and to community anchors that wish to both access and publish a wide assortment of content (including, but not limited to, scientific data sets) over modestly-provisioned access links.


Speaker Larry L. Peterson, Professor and Chair of Computer Science Princeton University

Speaker Jude Nelson Princeton University

Presentation Media

Secondary tracks United States Unified Community Anchor Network The Future: What's Next for the Net? Internet2 NET+ Services Industry Cyberinfrastructure

silver Sponsors

bronze Sponsors

supporter Sponsors