GridCoord DoW

Summary of international activities

The international activities in France are mainly done through the European projects, CoreGrid, EGEE, and DEISA. These stimulate European collaborations, and encourage international exchanges through travel funding. There are also several international collaborations going on:

• 
  Japan, through the NAREGI project of the NII (National Institute for Informatics), and the ITBL project of the JAERI;
  
• 
  Korea, which hosts the K*Grid project;
  
• 
  USA, for instance with Ian Foster's or Dennis Gannon's groups.
  

Annex 1: Summary of the first CNRS-STIC report on grid research and platforms (09/2002)

This text (editor Serge Fdida, 12^th December 2002) is a summary of the ideas exchanged between Christian Michau (CNRS), Yves Robert, Brigitte Plateau (CNRS) and Serge Fdida (CNRS-STIC). It describes the general position of the Informatics and Telecommunication CNRS Department (STIC Department) on this topic. This document is a translation of a report published in French under the title “Grille: Position générale sur les travaux de recherche et les plates-formes”.

Situation

It is recognized that computer science platforms, in the broad sense, have widely contributed to the improvement of science and engineering. They represent new objects to study, experiment, disseminate and eventually discover new research problems. It is therefore crucial to think about the creation of a new research platform, providing support to the scientific experiments, and maybe helping to study new contextual applications in the future.

Reflections on platforms

The concept of platforms may carry different forms, linked to the objectives aimed. We will identify two classes of different platforms that we need to differentiate in order to avoid to hinder the performances of each.

Applicative platforms and experimental systems (APF)

It is impossible to try to put together innovative applications on innovative platforms. Those are often dedicated to applications (Grid, e-Science, cooperative work, mobility, etc.). They must provide experimental services offering permanent support. We are here with a system close to a production system because the expectations of the end-users are in general incompatible with experimentation of informatics components and eventual modifications of infrastructure. In this context, this type of platform must have a team of administrators adapted to the use made by the users. The interest of this type of platform is that it can affect a wide range of applicative domain, as the users may come from different domains.

Platforms for Computer science research (PCSR)

These platforms are motivated by computer science research and experiments (distributed systems, parallel computing, networks, etc.). They do not offer a service assurance, in the sense that modifications may be brought to the infrastructure to study new concepts or algorithms. The whole point of PCSR is to prove the validity of concepts and to experiment them, in order to confront them with a synthetic environment. This proof will eventually help the transfer to industry. It is therefore essential to clearly identify which are the benefits of this platform (size, components, features, etc.), its type (local platforms federation, emulation, overlay, software components, etc.), and/or the integration of several levels (physical, network, middleware, applications). In a broader way, a PCSR must be seen as a large instrument of the STIC department enabling to put in common the contributions of different communities (distributed computing, networks, databases, distributed systems, security, etc.).

Network interconnection

The interconnection of geographically distributed elements of the platform is crucial. Its efficiency may favour or harm the execution of certain types of experiments which need for example an important bandwidth. The necessity of having a deterministic network of interconnections will have to be discussed. Emulation is an interesting alternative for certain types of applications. The existence of a deterministic infrastructure is essential in other cases.

Administration and engineering of the platform

Experience shows that the cost (human and software tools) of administration and usage are often more important than those invested for hardware. The human resources needed require a high level of qualification and are therefore harder to find. The choice of tools is essential because they simplify the access to the platform, (configuration, traffic and fault generation, etc.). A bad estimate of this factor may lead to dramatic failure.

The time factor

Finally, time is an essential factor in deploying a platform, its capacity of impact, and of success.

Recommendations

Based on the previous observations, we propose hereafter a few recommendations which define the general position of the CNRS STIC department on this theme.

STIC research programme

As those platforms play a critical role, we recommend beginning the support as soon as 2003, of a programme funding a platform for computer research (PCSR), widely distributed and heterogeneous (Grid). This project must be made in association with all the concerned and interested partners.

Implementation of the different platforms

We must make a difference between the hardware infrastructure, on which will be made the STIC experiment, (PCSR), and the infrastructure which is left to the users for operation (APF). For the latter, the funding of the CNRS is now in a grid perspective, in order to federate the computing resources. This will be started by the COMI by 2003. Within these production grids, we have:

• 
  Thematic grids organizing in networks laboratory equipment, which work on related domains
  
• 
  A more generalist grid featuring computing centres (evolution wished by the Ministry for its regional centres).
  

The gridification may be promoted by identifying applicative domains that would be announced and proposed to the researchers interested in their development and experimentation. An appraisal will be carried out with the different partners to guarantee exchange and permanent dissemination of information. The UREC will represent us in this coordination committee. The important effort that we should initialize and lead must therefore be on research Grids (PCSR), by separating the infrastructure, its funding and administration, of the research projects that will use it.

Interconnection network

Except for explicit strong constraints, we recommend that RENATER 3 connect the platforms.

Administration

The computing Grid (PCSR) will not only be the interconnection of hardware and the juxtaposition of system administration teams; above those must be added a structure of supervision and administration which would have the tasks:

• 
  Organize the grid;
  
• 
  Federate the local administration teams;
  
• 
  Set rules of usage;
  
• 
  Supervise the grid: usage and security;
  
• 
  Allocate support.
  

The CNRS has with the UREC a structure of engineering and services that must play a central role in the implementation of the orientations. Aware that there is a fast convergence of network and system with the evolution towards networked operating systems, the UREC has begun an extension of its initial competency scope and has an active part in the e-Toile project and the DataGrid project. In the same way that it has done for the RENATER network, it must ensure the engineering of grids and handle the interaction with STIC research teams of other domains. This mission covers:

• 
  A technical expertise mission;
  
• 
  A team federation;
  
• 
  An operational organization mission;
  
• 
  A mission of animation.
  

We propose that the UREC (possibly with other partners) be the entity that assures the administration and the supervision of the computing grid. The APF will be developed with their own administration (computing centres, universities, etc.). We advocate the creation of a coordination committee (APF, PCSR) within which the UREC will represent us for the administration aspect of the PCSR.

The time factor

We recommend that the project begins in 2003, to be well positioned on the critical path of research and computing tools. The deployment may be done in two stages, half in 2003, the other in 2004. Identifying the equipment hosting sites must start rapidly to check the motivation of researchers and the involvement of local actors. We will largely rely on the grid RTP and its actors. Other RTP may be associated with this appraisal. (networks, distributed systems, security).

European scale

We think that to maintain its visibility, the STIC should support in priority the European network of excellence CoreGrid, to which participate all the laboratories in the grid RTP. This does not constrain the possibility of the research teams or departments to be involved in other applicative or cross-disciplinary projects (for example, EGEE). The profit gained in the participation of our researchers and engineers in integrated applicative projects, or in operational infrastructure projects, must be weighed.

Annex 2: Summary of the first report on Grid5000 (07/2003)

This is a short summary of a 30 page report, called “Grid5000: Plateforme de recherche expérimentale en informatique” (Grid5000: experimental research platform in computer science), published in French for the “Direction de la Recherche” of the French Ministry of Research and New Technologies (editors Michel Cosnard, Serge Fdida, Olivier Poch, Yves Robert and Pierre Valiron, July 2003).

Summary of recommendations

The expert group has issued several recommendations:

• 
  Strategic objective. We recommend the creation by 2003 of a large research instrument, composed of the experimental platform Grid5000. This platform is essential to the national computer science community to attain its strategic objective in the domain of large-scale distributed computing, and scaling applications.
  
• 
  Creation of a GIS The structure proposed to administer the Grid5000 platform is a GIS (“Groupement d'Intérêt Scientifque” Scientific Interest Group, a specific legal status), gathering CNRS, INRIA, Universities, and may be CEA. Partner institutes are welcome to join the GIS from its creation time: regional communities and industry amongst others.
  
• 
  Durability of the financial commitment. Once the hardware is installed, the platform will not be limited to the interconnection of local clusters. A 3-year financial plan must be established to gather the needed resources to fund the people entrusted with the administration of the platform.
  
• 
  Synergy of efforts. On the national scale, within the ACI GRID, ACI MD, ACI Security National Co-operative Research Programmes, through the RNRT and RNTL National Co-operaative Research Networks, teams must be spotted and stimulated. The platform must also be widely open to the research community in its whole. The industry, the vendors (hardware and software), and the application users will have to be associated. Lastly, in the European context, the software and application development must be at the heart of the projects on the next FPRD (Framework programme for Research and Development) of the European Community.
  

Summary of the report

The report first provides a rationale to Grids, with examples, stressing the difference between production grids, used as number crunching tools, with quality of service requirements, and research grids, which aim at having a tool to run experiments on the concepts on grid computing. The latter are advocated as being flexible, with fast answer times, and openness. Essential to developers, they may also appear attractive to knowledgeable users. Follows a description of the Grid5000 project, first seeing the hardware deployed (up to 500 computing units per site, totalling a maximum of 5000, hence the name, all interconnected through the high-speed RENATER network). A view of the scientific programme is then given (give the computer scientists an experimental tool to do research on grids, and entice the applicative community to contribute to specific grid software development), as well as the administrative structure needed to supervise the grid, and the associated costs (23 engineers on three years, evaluated to 2.76 M€, and the hardware being evaluated to 10M€ - a total of 15M€ for the whole operational costs is deemed reasonable, even though some of these have already been paid for). A description of the context in Grids is also given, mentioning the TeraGrid project, as well as the DAS-2 project, and the Hungarian grid. The national landscape is also described, with the DEISA project, is lead by IDRIS (the French CNRS Supercomputing Centre), which is more a production platform project, the E-TOILE project, which came as a predecessor to Grid5000, and which is an experience to learn from. EGEE and CoreGrid, existing European projects, are also skimmed, and the report suggests using Grid5000 for these projects. Finally, the report gives the detailed scientific programme, split in three parts.

Grid community

The results in Grid research have to be made on solid ground, and not only based on simulations. Three themes are described: algorithms (adaptivity, monitoring, heterogeneity handling, modelling for scaling), middleware and components (communication between modules, load balancing, formal verification, distributed software component models, large numbers of machine cooperation), Peer-to-peer (virtualization allowing large system simulations and fault generation, of existing or new Peer-to-peer protocols).

Network community

New needs in the network community are arising, like spontaneous networks, sensor networks, overlay networks, ambient networks, etc. Grid problems often have overlaps with network-related problems: quality of service, multi-point, mobility, reconfiguration, etc. New needs are also emerging from the telecoms domain and provoke a transformation of the actual experimental networks towards new original platforms: clusters, overlay networks, federation of networks, etc. One can be easily convinced that many common points with grids exist. Furthermore, the Grid5000 platform could be seen as an Internet simulator, with nodes acting as client or servers, or as autonomous systems (routers with fault generation). The capabilities of dynamic reconfiguration of such an environment would be a real innovation.

Application communities

The availability and flexibility of such a grid are the first benefits for the application community, allowing to have considerable memory requirements, and jobs running for several days, without a prohibitive waiting list. The grid could be used for diverse scientific multi-parameter applications, such as in chemistry, astrophysics, particle physics, etc. Later on, one can imagine loosely coupled applications, which would use hierarchical domain splitting. We are also witnessing the emergence of a need of auto-adaptive applications, capable of adapting to irregular conditions and unspecified hardware topology. We also imagine that the grid could be split in an interactive sub-grid, and several batch sub-grids. Some work should also be done towards porting native applications to the grid, instead of scaling up already existing solutions. A call for proposals should be dressed up, with a yearly renewal, focusing the user feedback on of the workshops on the grid. Furthermore, the Grid50000 specifications seem to be compatible with cognitive models, with the global power corresponding to the raw information exchange capacity in a cortex fragment. Such an implementation would be a real breakthrough. But one must not forget that distributed data management is of prime importance, and disk space will be an important criteria for each cluster, as applications will require Terabytes of storage at least. Without efficient data storage, there will be very few ambitious openings to other scientific domains.