Evaluation 1 Evaluation panel and evaluation procedures

9. 
   EVALUATION
   

9.1 Evaluation panel and evaluation procedures

The evaluation panel consisted of:

Dr. Eric FLEURY

INRIA / INSA de Lyon, France

Tel: [phone],

Fax: [phone]

E-mail: [e-mail]

Prof. Jozef LUBACZ (TC TIST Rapporteur)

Warsaw University of Technology, Poland

Tel: [phone]

Fax: [phone]

E-mail: [e-mail]

The evaluation was based on all documents made available on the web pages of the COST 279 Action. The panel also used the CD issued for the action’s Final Seminar which contains a full documentation: publications and Temporary Documents. The panel members participated in the Final Seminar and followed all talks given by the action’s participants and also by distinguished invited academic and industrial experts. The evaluators had several discussions with the MC Chairman and other MC members.

9.2 Results versus objectives

In general, it can be stated that the action achieved the main objectives defined in the MoU, i.e. developed techniques for the analysis, design and control of advanced multiservice networks supporting mobility, multimedia and interworking. Principal results concern queueing models, traffic measurement, characterization and modelling in the context of core networks (mostly IP based networks and optical networks) and wireless network (mostly cellular), and also peer-to-peer services and applications. The details of obtained results are well documented in publications in peer-reviewed literature and conference proceedings, a comprehensive book and also in numerous internal documents.

9.3 Outcome and achievements

The main achievements of the action consist in the development of analytical techniques, algorithms, computer tools and traffic measurement methods, pertaining to:

(1) network performance, especially with respect to quality of service QoS,

(2) network protocols,

(3) network topologies and architectures,

(4) traffic modelling for peer-to-peer applications,

(5) economic aspects such as pricing principles and network cost estimation.

The results obtained allow for better understanding of network and traffic behaviour and also support network/services analysis, planning, design and optimisation.

All major results of the action, which were first published in journals, conference proceedings (over 150) and temporary documents (over 170), are summarised in a comprehensive Mid-Term Report, and finally, in a well structured, solid and complete book published by Springer.

An important achievement of the action is the organizing of two successful Summer Schools targeted for PhD students and young scientists. Considering the objectives of the action, also the Mid-Term and Final Seminar were very successful.
9.4 Impact of the Action

The results of the action will have a very positive influence on the development of future generation networks and services, especially with respect to methodological aspects. The COST 279 scientific community, which inherited and developed the achievements of previous COST actions (201, 214, 224, 242 and 257), is, according to the evaluators, an excellent example of benefits of international scientific co-operation within the COST framework. We believe that the achievements of COST 279 are outstanding and that thanks to the action the European research community may be still considered as the leader in the area of fundamental research pertaining to network modeling, design methodology and performance evaluation.

9.5 European added-value

The COST framework was essential in achieving the results obtained by COST 279. No other current European research framework provides comparable conditions for long-term, fundamental research.

The results of COST 257 had positive impact on several IST research projects within the VIth FP. It is also important to note that the networking established within the action played a major role in the establishment of the EuroNGI Network of Excellence (“Design and Engineering of the Next Generation Internet”).

9.6 Coordination and management

The coordination and management of COST 279 required special effort due to the very large number of participants and rapid changes in the object of research – the rapidly evolving networks and services. In these circumstances the action risked becoming amorphous in terms of its results. The MC of the action successfully avoided this risk by means of grouping particular research tasks into homogeneous and integrated issues, and by adapting the issues to changing circumstances. The inherent flexibility of the COST research framework and mode of operation was a significant aid in this respect.

9.7 Dissemination of results

The dissemination of the COST 279 results may be considered as intensive and effective (comp. section 9.3). The evaluators believe that the book published by Springer, which summarizes the main results of the action, will become an important reference point for the world-wide scientific community involved in research concerning modeling, performance evaluation and design of future generation communication networks and services.

Nevertheless, considering that an important part of the research pertained to the IP networks technology, the MC could have considered a wider presentation of some of the results of the action to the IETF circles, e.g. by means of RFCs.

A sorted list of publication could have been added to the action’s web page in order to emphasis all results of the action. It would be also beneficial if the programs and course programs of the two Summer Schools organized by the action were made available on the web page of COST 279 and the main COST web page.

9.8 Strengths and weaknesses

COST 279 succeeded to create a large and effective community of scientific cooperation in the area of fundamental research pertaining to network design and performance evaluation. As already mentioned, the importance and impact of this community is recognized world-wide. Many results of the action become a methodological reference point for application-oriented research projects carried on in Europe and elsewhere. Nevertheless it seems that more effort should be put into presenting the results of the action in a manner easier to comprehend by non-specialists in the area, especially the qualitative conclusions of the results. This would increase the impact of the action on application-oriented projects.

As already mentioned, more effort could be made to establish a comprehensive list of the action’s publications, sorted by type of reference (journals/major conferences/workshops and smaller conferences, etc.). Also the very important Summer Schools organized by the action could have been more widely announced.

In the final report more emphasis could be put on the effectiveness of cooperation of the action partners, e.g. by indicating the number of documents/publications that were an effect of collaboration of researchers from different countries/institutions.

Unfortunately, in the course of the action, the number of participants from telecom operators and manufacturers was declining for reasons which were independent of the MC. This however resulted in a rather minor impact of the action in the field of standardization

It is somewhat surprising that the budget for STSMs was completely used – only three STSMs were funded. The MC could have been more proactive in utilizing this important COST tool.

In spite of these weaknesses, the evaluators consider COST 279 as very successful.

9.9 Recommendations

The current development of communication technologies and services is extremely rapid both in quantitative as well as in qualitative terms. The need for sound, scientifically based methodology for networks and services modeling, performance analysis and planning is apparent. Considering this and the success of COST 279 it is recommended that the research effort undertaken by the action be continued. If the research in the field of COST 279 is continued within a follow-up action, it is recommended that it is closely linked to two COST actions recently launched: COST 293 (GRAAL - Graphs and Algorithms in Communication Networks) and COST 295 (DYNAMO - Dynamic Communication Networks). It is also recommended that the possible follow-up action has well defined links and complementarities with FP NoEs.