Data intensive high energy physics analysis in a distributed cloud

Download
  1. Get@NRC: Data intensive high energy physics analysis in a distributed cloud (Opens in a new window)
DOIResolve DOI: http://doi.org/10.1088/1742-6596/341/1/012003
AuthorSearch for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for: ; Search for:
TypeArticle
Proceedings titleJournal of Physics: Conference Series
Conference25th High Performance Computing Symposium 2011, HPCS 2011, June 15-17 2011, Montreal, Quebec, Canada
ISSN1742-6588
Volume341
Issue1
Article number12003
SubjectBatch jobs; Calibration data; Cloud systems; Data intensive; High-throughput computing; Input datas; Running applications; System use; Virtual machines; Physics; High energy physics
AbstractWe show that distributed Infrastructure-as-a-Service (IaaS) compute clouds can be effectively used for the analysis of high energy physics data. We have designed a distributed cloud system that works with any application using large input data sets requiring a high throughput computing environment. The system uses IaaS-enabled science and commercial clusters in Canada and the United States. We describe the process in which a user prepares an analysis virtual machine (VM) and submits batch jobs to a central scheduler. The system boots the user-specific VM on one of the IaaS clouds, runs the jobs and returns the output to the user. The user application accesses a central database for calibration data during the execution of the application. Similarly, the data is located in a central location and streamed by the running application. The system can easily run one hundred simultaneous jobs in an efficient manner and should scale to many hundreds and possibly thousands of user jobs. © IOP Publishing Ltd.
Publication date
LanguageEnglish
AffiliationNational Research Council Canada
Peer reviewedYes
NPARC number21270119
Export citationExport as RIS
Report a correctionReport a correction
Record identifier62199ecd-5f15-4f2d-a459-42212e41bc9e
Record created2014-01-03
Record modified2016-05-09
Bookmark and share
  • Share this page with Facebook (Opens in a new window)
  • Share this page with Twitter (Opens in a new window)
  • Share this page with Google+ (Opens in a new window)
  • Share this page with Delicious (Opens in a new window)