The magic experiment

The MAGIC experiment

• The MAGIC experiment
• The data center
• Migration to Grid
• Tools for users

Scientific targets

• Scientific targets

• Cosmic Accelerators

• AGN, PWN, SNR, GRB …

• Fundamental Questions

• Dark Matter, Cosmic Rays, Quantum Gravity, Cosmology …

• The MAGIC collaboration:

• 21 institutes (mostly in Europe)
• ~ 200 members

• Telescope site in Canary Islands

• Observatorio Roque de los Muchachos

• MAGIC-I: operating since 2004

• MAGIC-II: in commissioning (2009)

Discovery of 10 new VHE g-ray sources

• Discovery of 10 new VHE g-ray sources

• New populations unveiled

• Detection of distant VHE g-rays

• Observation of GRB in prompt emission

• Test on Lorentz Invariance (QG effects)

• > 40 papers in high impact journals

• Including 4 in Science magazine

MAGIC VO started in 2004

• MAGIC VO started in 2004

• Initiative by H. Kornmayer et al.
• 1st A&A project with PLANCK

• Initial project: 2 main areas

• Monte Carlo production
• Data storage and reduction

• Operations:

• 2004-06: no manpower
• 2007-08: activity resumed
• MC: UCM, Dortmund, INSA
• Data Center: IFAE and PIC

The MAGIC data center is hosted at PIC

• The MAGIC data center is hosted at PIC

• Started in 2006 as a “classic” data center
• Joint project: IFAE, UAB, UCM & IAA
• Stores ~300 TB/year of raw data

• The MAGIC data center at PIC provides:

• Data transfer La Palma -> Barcelona
• Data storage
• Official data reduction
• Data access for the collaboration
• Resources and tools for users’ analysis (*)
• Publish data to the Virtual Observatory (*)
• (*) In preparation

The data center services are being migrated to Grid

• The data center services are being migrated to Grid

• While the data center is in production
• Still a centralized data center
• + Give access to computing for users’ private analysis

• Services to be published

• SE, using the LFC catalogue
• Project database, integrated in LFC as metadata
• PIC CEs and software distribution

• Requirements from the experiment

• Make sure no features are lost
• Data access without certificate
• Overall, a soft transition

Grid is not kind to new users

• Grid is not kind to new users

• CLI tools are complex and not very user-friendly
• Optimal data access and workflows need experience

• For specific VO users most of the complexity can be hidden

• Wrapper CLI tools with familiar look & feel
• Graphical interface to access most common features
• Set of workflow templates for parameter-based jobs
• CLI still available for power users

• Learning curve can be significantly reduced

• Better user support if known, mature tools are used

A Gateway is in development

• A Gateway is in development

• Web portal to Grid resources
• Built around workflow templates
• Specific development for MAGIC

• Full security

• Grid certificates, VOMS
• User profiles: user, admin, public

• Currently in design phase

• Define modules
• Platform: use existing tools?
• Reusability: similar VOs in PIC

Users will not wait for the Gateway

• Users will not wait for the Gateway

• Two short-term solutions to soften the transition

• Magicdoors

• Web based data access to local SE
• Based on httpdoors for dCache
• Password authentication: no certificate needed (requirement)

• Virtual-UI

• Personal UI in a virtual machine (VirtualBox)
• Shared Folders: Simplified data downloading
• Versatile and portable

The migration of the data to a Grid SE was effective in July

• The migration of the data to a Grid SE was effective in July

• Virtual UI and magicdoors are already in use
• While MAGIC Gateway is in design

• The migration is being soft enough

• Only small issues which are being solved
• Working close with IFAE magicians, ~ guinea pigs
• Otherwise not much feedback… No news, good news!

• Next challenge: opening the computing

The MAGIC data center is being migrated to the Grid

• The MAGIC data center is being migrated to the Grid

• All data is already published
• Rest of services will follow soon

• Tools are being developed for user-friendly use of Grid

• A specific Gateway for MAGIC
• Short term solutions while waiting for the Gateway

• The migration is ongoing and successful so far

• Thanks!