Syndicate: A simple filesystem for scaleable data distribution
Time 10/03/11 03:00PM-04:00PM
Session Abstract
Syndicate is a global filesystem optimized to deliver large files (e.g,. scientific data sets, library collections, videos) to a scaleable number of clients distributed throughout the world. From the user's perspective, Syndicate offers an intuitive drag-and-drop interface (similar to Dropbox) through which data producers can make large files (and directories of files) available to a scalable set of clients, and data consumers can retrieve large files generated by one or more remote sites. Underlying this familiar interface is a scalable content delivery network, based on CoBlitz, a widely used CDN. Files written (published) to Syndicate are broken into chunks and distributed to a global set of high-performance caches. Files read from Syndicate are then pulled, in parallel, from these distributed caches, reassembled into their original form, and made available for local processing. Syndicate also maintains and distributes a file directory hierarchy, thereby supporting a full file system interface.
Syndicate (and the underlying CDN that it exploits) has application to both the scientific community that that has large data sets to distribute, and to community anchors that wish to both access and publish a wide assortment of content (including, but not limited to, scientific data sets) over modestly-provisioned access links.
Speakers
Speaker Larry L. Peterson, Professor and Chair of Computer Science Princeton University
Speaker Jude Nelson Princeton University
Presentation Media

Speaker Larry L. Peterson, Professor and Chair of Computer Science Princeton University
Secondary tracks United States Unified Community Anchor Network The Future: What's Next for the Net? Internet2 NET+ Services Industry Cyberinfrastructure