A technical Note on Web Evaluation Questionnaire & its analysis

1.0 Introduction:
Over a period, ISRO has established the real time data acquisition and data processing facility which acquires the data from ISRO science missions. This facility called as Meteorological and Oceanographic Satellite Data Archival Centre (MOSDAC) is housed at Space Applications Centre (Bopal campus), Ahmedabad. Objective of MOSDAC is to help the scientific community in the fields of meteorology and oceanography for their research work. The data Centre acquires processes and archives the meteorological/oceanographic data-products from ISRO science missions on MOSDAC. These data are then disseminated through web based service www.mosdac.gov.in to the users.
The key features of the MOSDAC website is:

1. 
   To disseminate Satellite and in-situ data under a single umbrella.
   
2. 
   Allow metadata search through various mechanisms.
   
3. 
   User friendly graphical interface.
   
4. 
   Facilities like Bulletin Board, Tools & Services etc.
   

Presently, the Kalpana-1 (K1), INSAT-3A (3A), Automatic Weather Stations (AWS) and Argo Met Stations (AMS), along with weather information are archived on MOSDAC.

This is purely an academic activity and hence the data is free but this data should be used only for research purpose. The commercial use of data is strictly not permitted without prior permission. The users are however requested to acknowledge MOSDAC, if the data is used in their research work.
It is envisaged that as more and more satellites get launched, the scope of MOSDAC will be expanded manifolds.
A website has enormous potential to reach the user community. Thus it has tremendous power to increase the usability of the data and outreach people. With the manifold rise in the users of the World Wide Web and also with the availability of many Websites serving similar data and information, Website visitors tend to go away to sites that are more user-friendly and efficient. Present day Website visitor is a very busy person and wants quick answers without any hassles. It is really an up-hill task to attract more users and keep them hooked to the Website. The Web team at Bopal, SAC campus is constantly striving hard to improve upon the Web design and augment the tools for utility of the data it archives. An attempt is made here to evaluate the MOSDAC Website through a questionnaire to the users and receive the feedback from them. This questionnaire appears in the ‘Bulletin Board’ services and the feedback is sought from the users. This document gives the details of the questionnaire and the feed-back that is received from users.

2. 
   Design of the questionnaire:
   

The questionnaire has been designed to evaluate major three aspects: (i) the web design (ii) the usability of MOSDAC data and (iii) the augmentation of processing facilities i.e. the tools that are provided to the users, apart from gathering general information about the user, who is responding to the questionnaire. The questionnaire consists of about twenty questions. Most of the questions are with multiple choices where user has to generally click the option of his choice. The questionnaire has been given as Annexure-I.

Out of the twenty questions:

1. 
   About 13 questions (i.e. 3, 4,5,6,7,13,14,15,16,17,18,19,20) are pertaining to the Web design.
   
2. 
   Six questions (i.e. 4, 6, 7, 8, 11, and 12) pertain to usability of MOSDAC data.
   
3. 
   Six questions (i.e. 8, 9, 10, 13, 19, 20) pertain to augmentation of MOSDAC processing facility.
   

As can be seen from the above details, it can be very well understood that certain question are common and they touch more than one aspect. For example question no. 4, ‘How often do you visit the MOSDAC site?’ covers both the aspects i.e. Web design and usability of the MOSDAC data; as a user will turn to this Website if he feels the Website design is doable as well as the data he finds here is of some use. The same logic applies to the questions that span across the domains.

Thus the layout of the questionnaire is sufficient to give an idea about the responder i.e. his interest in the data and his maturity and proficiency in using the MOSDAC data. User’s responses to the above should give the designers a fair idea of how the person who is responding to the Website evaluation questionnaire feels about the Website.

2. 
   Feedback on the questionnaire:
   

The questionnaire has been hosted on the Bulletin Board services on www.mosdac.gov.in as shown in the Fig. 1. As the user clicks on ‘WEBSITE EVALUATION QUESTIONNAIRE’ hyperlink, the questionnaire is thrown open to him for interaction as shown in Fig. 2. The user has to fill-in the questionnaire. His feed-back is stored directly in the database i.e. Oracle 10g. This database can be accessed and analysed by the MOSDAC team.

Fig. 1: Website hosted on internet www.mosdac.gov.in

Fig. 2: Web Evaluation Questionnaire thrown open to the respondent for interaction.
There is no restriction on the number of times a user can fill-in the questionnaire. However, during the analysis, the records with similar names and e-mail ids are checked. The records are examined for entries for the same individual during the specified time. In case, the record is filled incompletely, this is checked by the MOSDAC team. The record is meaningfully updated and the latest complete information is used for the analysis of the questionnaire.
A Web based automated procedure has been developed for viewing and analyzing the user’s feedback. The feedback could be viewed by the MOSDAC team by activating the automated procedure. Fig. 3 shows the home page of the ‘MOSDAC Evaluation Questionnaire Report’ Software package.

Fig. 3: Home page of ‘MOSDAC Web Evaluation Questionnaire Report’ Software package

On selecting the order of listing one can view the feedback to the entire questionnaire by all the users till date.

Fig. 4: Feedback list (sorted name-wise).
Other options available for viewing are; date-wise, e-mail wise, organization-wise and Negative inputs. The options date-wise or e-mail-wise or organization-wise are well understood in the sense the listing of the feedback will be displayed as per the order selected. A variation to this is the negative inputs. An option given as poor or very poor, disagree or strongly disagree etc. are qualified as negative remarks. Such options are listed in negative inputs list. This option has been intentionally put to closely examine the inputs. These are the feedback that needs to be closely studied. A careful study of unfavorable reports from the users will help MOSDAC team to improve upon the MOSDAC Website.

2. 
   Analysis of the Feedback
   

The Web Evaluation questionnaire was hosted on the MOSDAC Website since February 2011. Till date about 52 users have very kindly extended their valuable support by filling the questionnaire.

It has been observed that members/employees of prestigious organizations like BARC, CSIR, GSDMA etc. have enriched MOSDAC inputs by participating in giving the feed back. Table – 1 gives the break-up of organizations with no of members filling the questionnaire.
4.1 User response Question wise Analysis
The Web Evaluation Questionnaire has been carefully studied. In order to better analyse the questionnaire, statements have been formed and the analysis of the responses have been given here against each statement. Table – 2 gives the analysis of individual responses to various questions.
Fig. 5 to 36 give histograms for each of the question asked to the users in the form of statements.


Fig. 5: Question: What kind of connectivity do you have for Internet?


Fig. 6: Question: What kind of connectivity do you have for Downloading?



Fig. 7: Question: Do you visit metadata/data/images/forecast/information/all?

Fig 8: Question: Did the menu items on the home page give clarity about MOSDAC website?


Fig. 9: Question: How often do you visit the MOSDAC site?

Fig. 10: Question: Do you visit the MOSDAC site during a particular season?


Fig. 11: Question: Once you log onto MOSDAC website, what is the time you spend working on this website?

Fig. 12: Question: Approximately how many pages do you browse during a visit?


Fig. 13: Question: Do you suggest some more processing information should be hosted on MOSDAC website?


Fig. 14: Question: Do you have any general purpose algorithms related to meteorological/oceanographic data that you use routinely?


Fig. 15: Question: Are you ready to share the same with others?



Fig. 16: Question: Do you archive the data downloaded from MOSDAC locally and share it within your institution/organization?


Fig. 17: Question: Do you keep a record of how many users are accessing this data?


Fig. 18: Question: Have you published any research papers/articles using MOSDAC data?


Fig. 19: Question: Would you like to share this with other users?



Fig. 20: Question: Do you use the interaction forum facility on MOSDAC site?


Fig. 21: Question: Would you like to have any interaction forum for scientific discussions?

Fig. 22: Question: Will you recommend this website to others?

Fig. 23: Question: How did you first hear about this site?


Fig. 24: Question: Please grade our website on overall content.



Fig. 25: Question: Please grade our website on ease of navigation.


Fig. 26: Question: Please grade our website on overall look.


Fig. 27: Question: Do you agree that the information of this website is accurate and appropriate for the intended audience?


Fig. 28: Question: Do you agree that the information of this website is timely and up-to-date?



Fig. 29: Question: Do you agree that the user can see where he/she is on the site?


Fig. 30: Question: Do you agree that the style is consistent throughout the site?



Fig. 31: Question: Do you agree that the links are current and working?


Fig. 32: Question: Do you agree that the visuals enhance the message/purpose of the site?



Fig. 33: Question: Do you agree that the site is interesting?


Fig. 34: Question: Were you able to find suitable answers to your questions while browsing the website?



Fig. 35: Question: Did you experience any problems downloading files?
Responses from users/respondents fall under the following three categories:

1. 
   Favorable or positive responses – Where users have filled the questionnaire with excellent/good/average options or strongly agree/agree options.
   
2. 
   Unfavorable or negative responses – Where users have filled the questionnaire with bad/very bad or strongly disagree/disagree or the respondent has mentioned that he faced problem/s in carrying out a particular operation/utility.
   
3. 
   Neutral - Where a user has filled the questionnaire with undecided/did not try the utility or option.
   

It is not mandatory to answer every question that has been put up in the questionnaire.

We have specifically studied every record where even one question has fallen into the negative/unfavorable category as these are the feedback that will really help us in improving the Web Design and the utility of MOSDAC data. Table – 3 gives the analysis of negative inputs.

4.2 Feedback Interpretation summary
About fifty-two users have taken the trouble of filling the questionnaire. The following are the inferences derived from the feedback:

1. 
   Most of the respondents have good facility of internet and downloading. 76% have access to internet for surfing and download. 2 out of 51 resonant have lease line facility.
   
2. 
   50% of the users visit data on the MOSDAC site, 36% visit images, 25% visit forecast.
   
3. 
   92% feel that the menu items on the homepage give clarity about the MOSDAC Website and most of the people look to be generally satisfied with the Web design aspects. However some respondents (13/50) feel that suitable answers were not found to their questions while browsing.
   
4. 
   20% of the respondents visit the Website everyday and 20% pay an event based visit.
   
5. 
   July to September i.e. the monsoon months of India – people are interested to view data on seasonal basis.
   
6. 
   65% of the respondents spend about one hour and 27% spend about one to two hours on the MOSDAC interaction once they login.
   
7. 
   72% of the respondents want some more processing information hosted on the MOSDAC Website. 12 users are ready to share the general purpose algorithm.
   
8. 
   Three users have published research papers using the MOSDAC data.
   
9. 
   74% Users say they would recommend this Website to others however there are some users (3 users – 5%) feel that this site is not worth recommending.
   
10. 
    12/49 i.e. 25% faced some problem while downloading the data.
    

Suggestions from users have been given as Annexure – II

2. 
   Cumulative Analysis:
   

The questions in the questionnaire and the responses thereto give us idea of the various aspects pertaining to the Web site. But there is a need to assess the overall performance of the Website; hence it is felt that cumulative analysis be provided.

1. 
   Extract of Feedback from users
   

As mentioned above there are three aspects of the Website evaluation viz. (i) the web design (ii) the usability of MOSDAC data and (iii) the augmentation of processing facilities. The cumulative analysis takes into consideration the number of inputs received from all the questions from all the users. Hence, the count in cumulative analysis is the number of inputs and not the number of respondent. Table – 4, Table – 5 and Table – 6 give the criteria of the cumulative analysis of Web Design, Data Usability and Augmentation of MOSDAC processing facility respectively. Fig. 36 to Fig. 38 give the histograms of cumulative analysis.

Fig. 36: Analysis of all pertaining to Web Design (all questions – all respondents)



Fig. 37: Analysis of all pertaining to Data Usability (all questions – all respondents)






Fig. 38: Analysis of all pertaining to Augmentation of MOSDAC processing facility

5.2 Inference from cumulative Analysis
As far as Cumulative analysis is concerned, it could be inferred that:

1. 
   About 88% inputs have given a satisfactory grading to the Web Designing with 379 out of 957 inputs as very good, 308 as good and 159 as average. Only 7% of inputs have graded Web Design as bad or not up to the mark.
   
2. 
   About 89% of inputs have qualified that the data usability is of good quality with 218 inputs out of 420 inputs graded as very-good, 66 as good and 93 as average. Only 21 inputs (5%) have graded data usability as bad.
   
3. 
   98 inputs Out of 151 i.e. (64%) have opined that Augmentation of MOSDAC processing facility in terms of tools is required whereas 44 inputs (29%) express that augmentation is not required.
   

2. 
   Conclusion:
   

When a portal is being designed it is but natural that some people would find it useful and interesting where as some people may not like it. But on the whole it has to be appealing to a majority of the population. We as MOSDAC team feel that still there is a scope of lot of improvement but the portal is appealing and useful to a majority of the users.

This was the first attempt to know user’s feelings about MOSDAC portal and we have succeeded in building up our image in users’ mind. However, the effort for continuous improvement of the website in terms of contents, look & feel and adding more and more tools/facilities along with increasing the efficiency of delivering data/information as per user demand have to be made. The questionnaire will be regularly floated to users during January to March every year and the evaluation report will be brought out by April based on users feedback responses. This will help in understanding the users demand and improve upon the MOSDAC site accordingly as a continuous activity.


Tables

Table – 1 : Response from Organisations/Institutes
Sr. No.	ORGANIZATION	Number of Respondents
1	Advanced Centre for Atmospheric Sciences Lab., Sri Venkateswara University, Tirupati, Andhra Pradesh	1
2	AMPL, Hyderabad	1
3	Ashok Farms, Dhule Maharashtra	1
4	Astra Microwave Products Limited, Hyderabad	1
5	AWS, Aurangabad	1
6	BARC, Karwar, KArntaka	1
7	CSIR-CMMACS, Bangalore	1
8	Dept of physics, vidyanager, Khambhat, Gujarat	1
9	DHM, Kathmandu, Nepal	1
10	Electronic tech, Kochi, Kerala	1
11	Guwahati university, Guwahati, Assam	1
12	GSDMA,Gandhinagar,Gujarat	1
13	Gujarat University, Ahmedabad	2
14	HEXATRIK Electronics, Hyderabad, A.P.	1
15	Hindu Business Line, Thiruvananthapuram, Kerala	1
16	IGCAR, KALPAKKAM, Tamil Nadu	1
17	IIRS(ISRO), Dehradun, Uttrakhand	1
18	IISc, Bangalore, Karnataka	2
19	IIT KHARAGPUR	1
20	IMD, Pune, Maharashtra	2
21	Indian Council of Agricultural Research, Goalpara, Assam	1
22	ISRO HQs, Atmospheric Science Programme, ISRO Head Quarters, Bangalore	1
23	J VIS, Panchkula, Haryana	1
24	Karnataka state Sericulture Research and Development Institute, Bangalore, Karnataka	1
25	Kavi, Ahmedabad, Gujarat	1
26	Krishi Vigyan Kendra, Salem, Tamil Nadu	1
27	KSNDMC, Bangalore, Karnataka	1
28	MM University, Ambala, Haryana	1
29	Monash University, Melbourne, Australia	1
30	National Institute of Oceanography, Goa	3
31	NOAA CREST at CCNY, New York	1
32	NPOL, Cochin, Kerala	1
33	NRSC, Hyderabad, A.P.	4
34	POWERGRID, Solan, Himachal Pradesh	1
35	Research Institute for Humanity and Nature, Kyoto, Japan	1
36	SCMS Educational Institution, Cochin, Kerala	1
37	SDSC SHAR, Sriharikota, Andhra Pradesh	1
38	Space Physics Laboartory, Trivandrum, Kerala	1
39	Space Science Laboratory Dept of Physics,Barkatullah University ,Bhopal	1
40	Student, Pratapgarh, Uttar Pradesh	1
41	Dept. of Atmospheric & Space Sciences, University of Pune,University of Pune	1
42	Water Resources, Deulgaon Raja, Maharashtra	1
43	Yogi Vemana University, KADAPA, Andhra Pradesh	1