Theoretical Framework

6. ICT Tools for assessment

(this chapter will be finished before 1/3/07 and will form the basis of an article)
In this chapter we will give criteria for ICT tools for assessment. Based on these criteria an ICT tool will be chosen for the first prototype. This is our method:

1. 
   First we compiled a large list of criteria, based on our theoretical framework and own experiences;
   
2. 
   From this list we extracted some minimal requirements and categories of requirements;The categories were determined by collecting all possible criteria from the theoretical background, grouping them and reformulating a category;
   
3. 
   A large list if tools was compiled, based on experiences from earlier projects, the Special Interest Group Mathematics ‘toetsstandaarden’ working group {Bedaux, 2007 #160}, KLOO research {Jonker, #22}, the FI math wiki on digital assessment and math software (http://www.fi.uu.nl/wiki/index.php/Categorie:Ict), and google searches. As there are hundreds of math tools an initial selection was based on the tool having at least some characteristics of tools for assessment. Based on these requirements we:
   
4. 
   First selected tools that met the minimal requirements. Tools that didn’t, were described but not considered any further. For this we used a template.
   
5. 
   Then the remaining tools were graded on the other categories by:
   
   1. 
      Browsing the web on more information and usage on the tool;
      
   2. 
      Using the tool with already existing content. We aim at using quadratic equations as this tends to be subject that is catered for almost always;
      
   3. 
      Authoring our own content from chapter 5. Here it is possible that not all the finesses of a tool become apparent. A minimal requirement is that authoring can be used. This means for most tools that they have to be installed. For every installed tool I keep a log of screenshots.
      
6. 
   This resulted in a separate descriptions of the tools and a matrix, giving an overview of strong and weak points of several tools.
   

Minimal requirements

• 
  Webbased: we find it essential that using the tool can be anytime, anyplace, anywhere, using just a web browser.
  
• 
  Ease of authoring, configurable: it should be possible to add own content.
  
• 
  Actively developed: it is important that the tool is supported and has some sort of continuity.
  
• 
  Minimal math support: formulas should be displayed correctly and support basic mathematical operations.
  
• 
  Storage of progress: it should be possible to store results, so students can come back later, if necessary.
  

Categories

Content: algebra

• 
  Multistep exercises, process feedback
  
• 
  Sound representations and operations (!!!): mathematical and cognitive fidelity
  
• 
  Possibility of using more than one representation;
  Multiple representations can help better understanding of concepts.
  
• 
  Randomization
  
• 
  Wysiwyg formulas and input editor
  
• 
  Integrated CAS
  
• 
  Level of competencies.
  
• 
  Gradual difficulty curve. Coherence and balance.
  

Technical: tool

• 
  Results and answers should be stored and accessible for both student and teacher.
  
• 
  Sharability and use of standards (questions and units)
  
• 
  Easy authoring for teachers (because teachers are often “neglected learners” (Sangwin & Grove, 2006))
  
• 
  Scoring as a game, for motivation.
  
• 
  Other tools available
  
• 
  Performance
  
• 
  Possibility of using multimedia
  
• 
  Ease of use
  
• 
  Customizability by teacher
  
• 
  Contexts.
  
• 
  Prerequisites server and client-side
  
• 
  Stability
  
• 
  Languages
  

Assessment

• 
  It should provide feedback at the right moment
  
• 
  Several "modes" ranging from diagnostics to exam. “Zoom” level. Several training modes (from practice to exam).
  
• 
  Enables a qualitative analysis of student work to reap the benefits of formative evaluation;
  
• 
  on a meta-level scoring should also be implemented to allow for quantitative data;
  
• 
  A student should be able to correct an answer. Possibility to correct an answer
  
• 
  The combination of elements from both summative and formative assessment should enable a teacher to adapt better to student's learning.
  
• 
  Go from feedback only to weak formative assessment or even moderate. This has a lot to do with the state of the art. (level of feedback)
  
• 
  The 11 conditions under which assessment might support student learning and improve chances of success.
  
• 
  Seven principles of good feedback practice.
  
• 
  Open questions providing an expressive environment
  
• 
  Halfopen/closed questions. Open/closed questions
  
• 
  Functional feedback on the answers
  Formative assessment
  
• 
  Feedback on the process`
  
• 
  Global feedback: mastery level within curriculum
  
• 
  Possibility of providing hints
  
• 
  One answer versus multiple steps
  
• 
  Storage of answers
  
• 
  Storage of the process
  
• 
  Question management
  
• 
  Adaptivity
  

General

• 
  Licenses
  
• 
  User base for content availability
  
• 
  Continuity
  

The descriptions of the tools that were considered but did not meet the minimal requirements can be found in appendix C. Descriptions of the tools that did meet the minimal requirements can be found in appendix D. A comparative matrix assessing the strong and weak points of these tools can be found in appendix E. In the long term we aim to publish these assessments in a wiki type environment so they can be kept up to date.
Based on these results we choose tool X because:

• 
  Argument 1
  
• 
  Argument 2
  
• 
  Argument 3
  

Now we will use tool X to model the questions from chapter 5.