GridCoord d 1

Grid Initiatives and Research Directions

This chapter attempts to classify initiatives and research directions. It is worthwhile to mention that from the beginning the driving forces and motivations for pursuing Grid Research or for using Grid Technologies vary considerably from country to country. Adhering to the contributions in the Annex the analysis identifies the following main driving forces that shaped and led to Grid research in the GridCoord countries:

• 
  Computer Science related Grid Research
  
• 
  Parallel and Supercomputing (technology push)
  
• 
  Infrastructure / Research Networking (technology push)
  
• 
  e-Science (application pull)
  

The driving forces are further analysed depending on the country contributions and strengths, in the sense which driving force was dominant or still is. In some countries the driving forces are/were evolving and overlapping and fall, therefore in several categories.

Computer Science related Grid Research Projects

In some countries, the (“academic”) computer science community was quick to link their ongoing research activities with the fast spreading of the Grid concept. It is well known that the roots of Grid computing go back to classical distributed computing – a complex research area with many open questions and some known, but unsolvable problems. Research on distributed computing is clearly still in its infancy! This simple fact clearly points out the difficult state of Grid Computing today: In some cases, where fundamental problems cannot be solved to the satisfaction of computer science theoreticians, applied sciences have to find practical work-arounds for being able to deploy Grids without compromising important non-functional properties such as correctness, reliability, consistency, security, etc.

In the following we list some Member States’ efforts solely devoted to support and undertake research on Grid Computing and Distributed Systems..

France

From the beginning, Grids have been considered in France as a subject of research by the Academia, and the structure of public funding made it possible to maintain this line of action. The two main public research institutes in the IST field, CNRS and INRIA, had key roles in motivating their research groups in this direction and in setting up the necessary “human networking” between them through a number of co-operative research programmes. In France public research has taken the lead in the development of Grid technology and promoted a long-term vision of the scientific approach, which culminated in the national Grid’5000 large research instrument project. The main purpose of this platform is to serve as an experimental testbed for research in Grid computing. The originality of this project is that it is in some sense bottom-up: a number of groups decided to put in common their resources, whatever their origin, and to collaborate. Grid'5000 is a research effort developing a large-scale nation wide infrastructure for Grid research. 10 laboratories are involved, nation wide, in the objective of providing the community of Grid researchers a testbed allowing experiments in all the software layers between the network protocols up to the applications: applications, algorithms, runtime, middleware, operating systems, network protocols. Grid'5000 will allow Grid experiments France-wide in all these software layers.

Italy

Grid-like research started in the 80s-90s in the context of the CNR Special Programmes in Computer Science and Technologies with a special emphasis on Parallel Computing and High Performance Computing. In 1994 a special programme on Metacomputing for the solution of large-scale problems in engineering was proposed by CNR and a Metacomputing infrastructure was developed. The pioneering project in high-performance large-scale platforms in Italy has been the PQE2000 Project (1995-2000), led by CNR in collaboration with ENEA, INFN and Alenia Spazio. A large set of results was achieved in large-scale platform infrastructures (MIMD+SIMD complexes) and in software technology (high-performance programming environments). From this collaboration originated the industrial company Quadrics Supercomputers World Ltd (QSW).

In 2002-2004 two projects have been launched on “Grid Computing: enabling technologies and applications for e-Science” and on “High-performance Large-scale Distributed Platforms”. Important results have been achieved in software technology, programming tools and environments, knowledge-based and data-mining grid-aware tools, as well as in applications.

The results from the mentioned projects and those from PQE2000 have played a fundamental role in the definition of competences, research skills, tools and environments ultimately leading to the establishment the national research project Grid.it: the initiative that co-ordinates and tries to unify the various national efforts in Grid research. The project having a strong interdisciplinary character aims at defining, implementing and applying innovative solutions for network computing enabling platforms, oriented towards scalable Virtual Organizations and based on the Grid Computing paradigm. The research topics span from high performance photonic networks, innovative middleware services and high-performance programming environments to application testbeds.

The Netherlands

In the Netherlands research in distributed computing has a long tradition. The ASCI DAS project is a purely computer science project with an emphasis on research in parallel and distributed systems, parallel algorithms, operating systems and visualization among others. The Grid research being undertaken includes parallelization of algorithms, management of data and information (based on ontologies), text and data mining, dynamic workflow automation, interactive computational steering, problem solving environments, high performance and high throughput computing.

Sweden

There is a strong Grid research community in Sweden, but there is so far no comprehensive national funding programme specifically for Grid, and the attitudes of the various funding bodies tend to be optimistic but cautious and the responsibility is delegated to existing organizations with funding in the HPC and general computing area.

Forming a main research line, several research projects in Sweden are dedicated to make Grid technology more mature and user friendly. Examples include a Grid Resource Brokering tool and a project to combine the power of the Grid with a modern high-level modelling language by developing a framework for simulation on computational grids, GridModelica

UK

UK has CS project also.

The other countries

In the other surveyed countries there was research in computer science ongoing, but it was not reported having had a leading effect in to the development or uptake of Grid technologies.

Parallel and Supercomputing related Grid Projects

In several European countries the parallel and supercomputing communities understood early the potential of Grid computing to improve high performance and high throughput computing to better serve their users. To solve data intensive applications the community had developed parallel I/O techniques and distributed file systems, precursors of Data Grids. Several traditional scalar and vector supercomputers have recently been replaced by computing systems based on large clusters of more commodity oriented Unix processors. These still use sophisticated interconnect systems rather than conventional networks. At the same time commodity PC/Linux based systems have become more and more powerful. Large scientific and engineering applications have been ported out from the mainframes to distributed cluster systems. As WAN performance and cost come close to LAN level, it becomes quite natural to imagine to expand a cluster outside the computer centre room and implement a virtual computer centre distributed over the world.

This technology push (reduced hardware prices, more powerful hardware etc.) positively contributes to the vision that large-scale distributed supercomputers can be implemented. Still today many people only think about the computational side of Grids, the so-called virtual supercomputer built from aggregating CPU capacity from multiple systems.

In following we examine the grid-like activities triggered by parallel and supercomputing roots in the GridCoord countries.

Austria

The majority of Grid Computing activities were and are based on the efforts of the Austrian Centre for Parallel Computation (ACPC). Since Grid computing is often considered as the logical evolution of parallel and distributed computing, the groups combined in the ACPC represented the logical basis for a Grid computing initiative in Austria. As the national awareness about the importance and necessity of Grid computing had been continually increased over the last few years, people from ACPC started to assemble the AUSTRIAN GRID consortium.

France

The current Grid trend in France stems from two seminal projects, VTHD++ and e-Toile.

VTHD++ is a test platform for new generation internet services. It lasted 28 months, and closed in May 2004. Conducted by an industrial prime contractor (France Telecom) it had as objectives: upgrade the previous VTHD platform, operation of MPLS traffic engineering tools, development of a broadband firewall and of applications, and active network testing. It was financed by the RNRT.

e-Toile was the first major French high performance data transfer grid project, including industrial partners (CEA , EDF, IBCP), during the years 2002-2004. Its objectives were: research in distributed computing, usability of Grid technology by end-users applications and test networking approaches. This project was financed by the RNTL.

Germany

In the late 1990s a number of small, decentralized Grid projects have been conducted, mostly with the goal to provide user-friendly access to remote HPC systems and clusters. Several names like Metacomputingmetacomputing, hypercomputing, distributed supercomputing etc. have been coined, each of them denoting a slightly other flavour of what is called today Grid Computing. The emphasis was mainly on the coordinated use of HPC. Driven by the need for a unified user access to the supercomputers in Germany the Unicore project was initiated. The resulting software was successfully deployed and is now used nation-wide in many supercomputer centres and also abroad (mainly in Japan).

On a regional scale, some state governments supported projects aiming at the utilization of distributed computing equipment. The “Northrhine-Westphalian Metacomputer” is one such project. Here, the emphasis was put on linking university computer services to allow for a better utilization of local resources. The state of Baden-Württemberg has recently established a centre of competence in high performance computing which haswith a focus on Grid computing.

Italy

The pioneering project in high-performance large-scale platforms in Italy has been the PQE2000 Project (1995-2000), led by CNR in collaboration with ENEA, INFN and Alenia Spazio. A large set of results was achieved in large-scale platform infrastructures (MIMD+SIMD complexes installed at CNR in Naples) and in software technology (high-performance programming environments). From this collaboration originated the industrial company Quadrics Supercomputers World Ltd (QSW). Metacomputing was a step towards Grids (see research section).

The Netherlands

The Dutch graduate school of advanced computing and imaging (ASCI) took the initiative to realise a distributed computer infrastructure for research into parallel and distributed systems called Distributed ASCI Supercomputer (DAS), the third generation of which was recently awarded funding. Research groups from Delft University of Technology, Vrije Universiteit, University of Amsterdam, Leiden University, University Utrecht, University of Twente, University of Groningen and Eindhoven Technical University participate in ASCI.

Sweden

Grid systems related research started in the 1990s, emphasis was on co-ordinated use of high performance computing systems and clusters. One of the early sites was PDC at KTH where their annual conference of 1999 had the title “Simulation and Visualization on the Grid”, proceedings published by Springer Verlag. The same year a proposal was drawn up for the creation of a Swedish HPC-metacenter. In the beginning of 2002 the Swedish Research Council decided to establish such a metacenter, SNIC, Swedish National Infrastructure for Computing.

UK

In UK HPC is handled separately.

Infrastructure / Research Networking

The rapid development of networking technology has allowed for easier coupling of distributed applications. The relentless increasing of network bandwidth has slowly eroded the boundary between the server and the network, turning a collection of interlinked, discrete resources into a fabric consisting of physical resources, such as servers, storage and network components, as well as logical resources—for example, operating systems, database servers, application servers, and other platform or infrastructure software.

The improvements of network technologies will boost Grid development even more in the next few years taking into account that reduced increased network latencies bandwidth promotes the popularity of using distributed, Grid- like applications.

This technology push (increased bandwidth, reduced latency, quality-of-service-contracts, etc.) positively contributes to the vision that large scale Grids can be implemented.

France

The French networking system is based on collective research co-operation programs of the CNRS and of the INRIA, and federate essentially all the academic French groups active in Grids, holding conferences (RenPar) and national schools. There are also important cross-border collaborations. Industry also takes part in research through the National Networks RNRT and RNTL.

The current main French infrastructure is Grid’5000, which is available to all public researchers, and which is a nation-wide research grid on which all levels support reconfiguration, from network protocols to the application layers. There are currently more than 750 PCs connected on nine sites, the aim being to have 5000 on ten sites in the coming years.

Germany

The German Research Network (DFN Verein e.V.) supported since the mid nineties several projects for the advancement of network infrastructure and protocols (broadband research network, Gigabit network, etc.). This also includes the adaptation of some non-trivial demonstrator applications to run in such a distributed environment. Additionally, a grid competence centre in Karlsruhe (GridKA) was established with financial support from the German Science Ministry.

Hungary

There is a national Grid infrastructure-oriented initiative to provide a nation-wide Grid infrastructure for the Hungarian higher educational institutions. The project called ClusterGrid project aims at integrating the Intel processor based PC labs of the Hungarian higher educational institutions into a single, large, countrywide interconnected set of clusters. The PCs are provided by participating Hungarian institutes, such as universities, polytechnics or public libraries. The central infrastructure and the coordination are provided by NIIF/HUNGARNETT. The applied technologies are based on Condor and SGE for task distribution and VPN for security. Currently more than 1400 PCs of 29 clusters are connected from 20 higher educational institutions.

The Hungarian Grid Competence Centre (MGKK) aims at creating a knowledge centre in Grid technology where the available critical mass enables the intensive support of establishing and operating nation-wide Grid infrastructures for the benefit of the whole Hungarian academic community. MGKK provides consultancy in Grid computing also for the business and government sectors in order to accelerate the employment of the leading edge Grid technology in companies and governmental institutions.

Italy

The activities started in February 2000 with the approval of the INFN Grid project aimed at the development of the middleware and the Italian Computational Infrastructure for the CERN Large Hadron Collider Experiments, and the R&D programme on Grid technology of the Institute for Advanced Interdisciplinary Study of the University of Lecce supported by the Ministry of Research and Education (MIUR) funds. IT Industries like Datamat SpA and Nice SRL established since then an early and fruitful collaboration with INFN in Grid middleware development. Several partners from Universities, CNR and other research centres participated in the INFN initiatives through applications on e-science fields, such as bioinformatics, geophysics, and astronomy.

The Netherlands

Networking and networking research have always been high on the Dutch research agenda. The national networking organization SURF has recently obtained funding for an innovative lambda networking infrastructure, Surfnet6, this being the 6^th generation networking infrastructure in the Netherlands. Experience has shown that while grid research makes demands on networking technology driving research in that area, the provision of advanced networking infrastructure can in turn drive grid research by enabling new possibilities.

Poland

The main initiative through which the Grids are being developed is the PIONIER initiative. Full name of the initiative is “PIONIER – Polish Optical Internet, Advanced Applications, Services and Technologies for Information Society”.

The most prominent Grid project spanning almost whole Poland is the Clusterix project (CLUSTERIX = National CLUSTER of LInuX Systems). This project contains the concept of building a core infrastructure for clustering local PC clusters, allocated and distributed among universities.

Spain

Complementarily to this network, the IRISGrid initiative aims at providing interested parties with best practice advices on the use of Internet resources and on security issues (i.e., in general, building a good infrastructure to deploy Grid-based applications) and also at achieving a higher scientific and technological coordination. IRISGrid was born within RedIRIS, the Spanish national organization which manages the academic Internet network. IRISGrid is a meeting point of many institutions (listed below) and it also maintains contacts with the Thematic Network for Middleware in order.

Sweden

NorduGrid is a Nordic Grid Research and Development collaboration initiated in 2001, aiming at development, maintenance and support of the ARC (Advance Resource Connector) grid middleware.

UK

UK has a national Grid Infrastructure supporting a broad portfolio of e-Science projects.

e-Science and Applications

Applications have evolved from once simple, often monolithic, computer-or server-centric binaries to services that are disaggregated and then distributed across the network. In parallel, there is an application pull: several sciences depend on the collection and the analysis of very large amounts of data. Particle physics, Astronomy, Earth System ObservationSciences, Biology are typical examples. These sciences have developed very sophisticated instruments which are only available in a few centres but which produce large amounts of data which must be distributed to world-wide scientific communities. Originally, the user community was restricted to have access only at certain sites but the technology push has contributed to the distribution of the user community, and thus access "without limits" has become a strong requirement of such users.

France

Grid public research funding in France has initially been focused on building and experimenting with grids, through the ACI GRID Program <http://www.recherche.gouv.fr/recherche/aci/grid.htm>. This program was launched as early as March 2001 by the Ministry of Research. It was led by Michel Cosnard, which was later succeeded by Thierry Priol. Very soon, this program explicitly included projects about applications of Grids to support other sciences: physics, biology, applied maths, etc. but also linguistic databases, medical data management, etc. The annual budget was 2-3 MEuros (remember that this does not include salaries).

After a couple of years, this program split into two programs: the GRID'5000 Program, which aimed at building a real-scale, experimental grid across the country; and the ACI "Masses of data" <http://acimd.labri.fr/> Program, which explicitly targeted the applications of grids (and more generally, distributed computing techniques) to the management of very large amounts of data.

This latter program gradually moved to a real e-Science program. It selected two dozens of projects per year, with a special emphasis on projects aiming at using grids in a wide spectrum of scientific contexts: data-mining, art museum archive management, dynamic data analysis, geography, high-energy physic, geology, biology, satellite image processing, astrophysics, French linguistic, etc. In all cases, the focus of the research was in using the most advanced grid techniques to support the development of another field.

A special mention should be made for the astrophysics community, led by Françoise Genova from the "Center of Astronomical Data of Strasbourg" <http://cdsweb.u-strasbg.fr/> (CDS, after the French wording). This research group played an instrumental role in federating many first-rank astronomy research groups around the world so as to build the "International Virtual Observatory" (IOVA) <http://www.ivoa.net/>. The ACI programs strongly supported this initiative and contributed to foster contacts between the French part of the IOVA (VO-France <http://www.france-vo.org/>) and the researchers in Grids and related topics. Amazingly, this blend was quite successful, and many advanced Grid techniques developed in France at that time could be experimented at a real-scale through this alliance. The links which have been secured through these two communities are still living today. This program is now closed, but a similar program <http://mdmsa.labri.fr/> has just been launched by the new French ANR. This new program explicitly targets the management of huge masses of data in modeling, simulation and applications.

The description above was only concerned with programs directly supported by the Ministry of Research. One should also mention the very strong involvement of para-public institutions, such as the "Atomic Energy Agency" <http://www.cea.fr/gb/index.asp> (CEA, after the French wording). CEA is a French government-funded technological research organization. A prominent player in the European Research Area, it is involved in setting up collaborative projects with many partners around the world, including the well-known CERN. CEA has been instrumental in developing applications of grids to large-scale numerical simulation in nuclear physics, but also in a large spectrum of domains such as climate modeling, biology, etc. With the recent creation of the French National Research Agency ANR <http://www.gip-anr.fr/>, CEA is now leading a specific program about Intensive computing and grids (CIGC <http://www.gip-anr.fr/appels/2005/cigc.htm>). For its first year, this program was provided with a budget of 6 MEuros to fund a dozen of 3-year projects. Most of them are concerned with numerical simulation in physics or (bio-) chemistry, but there are also projects concerned with intensive financial simulation, etc.

Germany

In 2003/2004, German Grid researchers organized into national D-Grid consortium. The consortium established a steering committee, which developed a strategy and a research framework for establishing e-Science in Germany. The BMBF adopted the research framework and compiled a first call for proposals.

As a result from the call for proposals, the German Ministry for Education and Research (BMBF) The resulting D-Grid projects are structured as follows: There will befunds one horizontally organized “integration project” and three to five vertically organized “community projects”. The community projects will beare seen as the driving forces. The integration project will take up the results (and software) of the community projects and will consolidate and integrate them into a national Grid environment which will become the basis for several overlay Grids for the different science communities. The first five community projects will cover the domains climate and earth sciences, medicine and bioinformatics, high-energy physics, and astrophysics, computational sciences and engineering, and humanities and social sciences. An additional supplementary call for proposals covers the area of information and knowledge management.

The Netherlands

The Scientific and Technology Centre Watergraafsmeer (WTCW) in Amsterdam, comprising amongst others the University of Amsterdam (Institute for Informatics as well as Biology), the National Institute of Nuclear and High Energy physics (NIKHEF), the latter with connections to CERN, and the Dutch Computer Centre SARA, took the initiative to start a number of application-oriented Grid initiatives in the Netherlands for which the second generation so-called Virtual Laboratory for e-Science (VL-e) was recently funded. In this project, networking, distributed computing and Grid researchers collaborate, together with applied scientists from both governmental and industrial research.

The VL-e project concentrates on application and application-driven computer and computational science research. The philosophy is to develop e-Science generic methodology along the total technology chain where the applications are the drivers of the research. It makes a distinction between a rapid-prototyping environment where new research ideas can be quickly evaluated and a proof-of-concept environment where application cases can be developed and that should be the basis for a later roll-out of Grid technology.

Spain

While a certain lag with respect to other European countries is acknowledged, Spanish funding bodies recognize that Grid computing can be important for the future development of several scientific and technical areas. There is, therefore, a commitment to give priority to Grid-related research at national and regional level by empowering this e-Science initiative. A committee to study the impact of e-Science, in which Grid technologies are seen as a core part, was set up and endowed with the task of designing a roadmap thereof. This task has been recently finished and its results presented to the scientific community.

As a result, the recently presented White Book on e-Science in Spain details a series of areas in which Grid computing can be of invaluable help, supporting this list with detailed reasons for their selection.

The white book lists a series of already existing resources which should be shared among researchers in order to increase productivity, and makes the point that a well-developed Grid infrastructure is key to take advantage of these resources in a transparent, homogeneous manner. It also highlights that the driving force of such a joint effort comes from applications, whose users will greatly benefit from the added computing power and transparent access

UK

The overall goal of the e-Science Core Programme is to identify the key generic middleware requirements arising from e-Science projects. In collaboration with scientists, computer scientists and industry, the programme has a mandate to develop a framework that will promote the emergence of robust, industrial strength Grid middleware that will not only underpin individual application areas but also be of relevance to industry and commerce. There is an Open Source/Open Standards requirement on all middleware developed within this programme.