Theoretical Framework

7. Designing the prototype and instruction

So based on a rationale and a problem statement, a conceptual framework was formulated, leading to a motivated choice in ICT tool and content, plus design principles. In this chapter we describe what prototype resulted from all of this. This prototype will be the prototype used in the actual research cycles.

I want to make sure that transfer from the tool towards pen and pencil takes place (Kieran & Drijvers, 2006). This is why I design an instructional sequence with both tool use and pen/paper tests. This sequence has some similarities with the Hypothetical Assessment Trajectory in the CATCH project.
For our prototype this means

• 
  Formulate an orchestration for tool use
  
• 
  A visual approach that facilitates transfer from computer to pen-and-paper
  

A prototype of the test can be found on

http://www.fi.uu.nl/dwo/voho
Login name: vohodemo

Password: omedohov

In appendix F some screenshots can be found of the authoring process.
The first implementation has no randomization. The second implementation will have more questions making use of randomization.
Authoring capabilities are available on request.
Part III Methodology
(This part is under construction for the research plan. Deadline: 1/4/08)
See research plan: hypothetical assessment trajectory

Self-assessment

It is tempting to quote here a postulate by Wiggins (1993): “An authentic education makes self-assessment central.”
In 1998, Black and Wiliam were surprised to see how little attention in the research literature had been given to task characteristics and the effectiveness of feedback. They concluded that feedback appears to be less successful in “heavily-cued” situations (e.g., those found in computer-based instruction and programmed learning sequences) and relatively more successful in situations that involve “higher-order” thinking (e.g., unstructured test comprehension exercises).
“Let us start with the Professional Standards for School Mathematics (NCTM, 1991). These standards envision teachers’ responsibilities in four key areas:

• 
  Setting goals and selecting or creating mathematical tasks to help students achieve these goals.
  
• 
  Stimulating and managing classroom discourse so that both the students and the teacher are clearer about what is being learned.
  
• 
  Creating a classroom environment to support teaching and learning mathematics.
  
• 
  Analyzing student learning, the mathematical tasks, and the environment in order to make ongoing instructional decisions.”
  

The consideration of (a) the learning goals, (b) the learning activities, and (c) the thinking and learning in which the students might engage is called the hypothetical learning trajectory (Simon, 1995).

Our basic assumptions will be the following: there is a clearly defined curriculum for the whole year—including bypasses and scenic roads—and the time unit of coherent teaching within a cluster of related concepts is about a month. So that means that a teacher has learning trajectories with at least three “zoom” levels. The global level is the curriculum, the middle level is the next four weeks, and the micro level is the next hour(s). These levels will also have consequences for assessment: end-of-the-year assessment, end-of-the-unit assessment, and ongoing formative assessment.
Hypothetical Assessment Trajectory.

Some of the ideas we describe have been suggested by Dekker and Querelle (1998).

Before

Entry test. A short, written entry test consisting of open-ended questions.
During

During. While in the trajectory, there are several issues that are of importance to teachers and students alike. One is the occurrence of misconceptions of core ideas and concepts

• 
  Short quizzes, sometimes consisting in part of one or more problems taken directly from student materials.
  
• 
  Homework as an assessment format (if handled as described in our earlier section on homework).
  
• 
  Self-assessment—preferable when working in small groups. Potential important difficulties will be dealt with in whole-class discussion.
  

This ongoing and continuous process of formative assessment, coupled with the teachers’ so-called intuitive feel for students’ progress, completes the picture of the learning trajectory that the teacher builds.
After. At the end of a unit, a longer chapter, or the treatment of a cluster of connected concepts, the teacher wants to assess whether the students have reached the goals of the learning trajectory. This test has both formative and summative aspects depending of the place of this part of the curriculum in the whole curriculum.

Appendix A

This appendix describes the chosen questions in more detail, answering the following questions.

1. 
   Why is this an interesting question?
   
2. 
   What skill or behavior is assessed here?
   
3. 
   What answers do we expect?
   
4. 
   What could be obstacles in answering this question?
   
5. 
   What feedback could be given for this question?
   
6. 
   What tools could be used to model this question?
   

1	This question addresses whether a student has ‘gestalt’ quality: does her or she recognize similar parts of an equation. Behavior #6: flexible manipulation skills. When a student recognizes similar parts he or she would:  or  (Note: students could easily forget that  becoming 0 yields two answers.)  or   or  or  Students could also be attracted by the brackets and tempted to lose them. Perhaps the fact that this is a lot of work will keep them from doing so. Perhaps an easier but similar question like  would be dealt with this way. Kop and Drijvers (Kop & Drijvers, in press) mention some causes: firstly the fact that students see formulas as recipes (process) instead of coherent objects. Secondly visual characteristics of questions play a role (‘visually salient’). In question one it is tempting to lose the brackets. In question 2 it is tempting to square the roots, as to lose them. In both cases this gives a lot of work, but brings us no closer to the solution. Thirdly a lack of flexibility in manipulating expressions. Fourthly a lack of meaning. Perhaps using a model helps understanding what is asked. Lastly there often is a lack of practice in solving such questions. With this last point we explicitly contend that insight also comes from practicing. Or to quote Freudenthal: “Advocates of insightful learning are often accused of being soft on training. Rather than against training, my objection to drill is that it endangers retention of insight. There is, however a way of training — including memorisation — where every little step adds something to the treasure of insight: training integrated with insightful learning.” (Freudenthal, 1991) Perhaps giving the possibility of substitution, or at least making visual or symbolic groupings could help a student. Pointing out that a student should look for corresponding terms. Here giving the possibility to plot functions and thus to see what solutions an equation has, could be helpful. A tool should also enable students to algebraically solve an equation, preferably in ‘What you See Is What You Get’ form, to help transfer from the tool to ‘pen and paper’.
2	 solved for v This question addresses whether a student has ‘gestalt’ quality: does her or she recognize similar parts of an equation, what characteristics are ‘visually salient’ Behavior #6: flexible manipulation skills. This question has to do with –as Arcavi puts it- ‘gestalt’ and ‘circularity’. Wenger (Wenger, 1987) : “If you can see your way past the morass of symbols and observe the equation #1 (  which is required to be solved for v) is linear in v, the problem is essentially solved: an equation of the form av=b+cv, has a solution of the form v=b/(a-c), if a≠c, no matter how complicated the expressions a, b and c may be. Yet students consistently have great difficulty with such problems. They will often perform legal transformations of the equations, but with the result that the equations become harder to deal with; they may go “round in circles” and after three of four manipulations recreate an equation that they had already derived…Note that in these examples the students sometimes perform the manipulations correctly…” Here the roots are important: they attract attention, but actually trying to take squares on both sides would be a mistake. The chance that students are not able to continue or come ‘back where they were’ (circularity) is large. Also, the fact that there are two variables, contrary to question 1, poses a problem. Perhaps giving the possibility of substitution, or at least making visual or symbolic groupings could help a student. Pointing out that a student should look for corresponding terms, but also what ‘solved for v’ means. Here giving the possibility to plot functions and thus to see what solutions an equation has, could be helpful. A tool should also enable students to algebraically solve an equation, preferably in ‘What you See Is What You Get’ form, to help transfer from the tool to ‘pen and paper’. Two variables should be supported.
3	This question addresses whether a student has ‘gestalt’ quality: does her or she recognize similar parts of an equation, what characteristics are ‘visually salient’ Behavior #6: flexible manipulation skills. This question again has to do with –as Arcavi puts it- ‘gestalt’ and ‘circularity’. This equation can be solved ‘the easy way’ of ‘the hard way’. Noticing that Here the square and the brackets are important: they attract attention, but actually rewriting the left side of the equation would be a mistake. The chance that students are not able to continue or come ‘back where they were’ (circularity) is large. This mistake could be even more tempting to make when is used. Perhaps giving the possibility of substitution, or at least making visual or symbolic groupings could help a student. Here giving the possibility to plot functions and thus to see what solutions an equation has, could be helpful. A tool should also enable students to algebraically solve an equation, preferably in ‘What you See Is What You Get’ form, to help transfer from the tool to ‘pen and paper’.
4	Find the pattern and proof the result always is 2 (source: Martin Kindt) This question addresses whether a student understands that using certain symbols could help solving a problem. Behavior #5: the choice of symbols. In this question Arcavi would call some premonitory feeling for an optimal choice of symbols an important part of symbol sense. Here we expect students to make a choice of symbols and then rewrite the expression. So there are two steps. It could also be interesting to see whether the choice of symbols differs, but of course both giving 2 as answer. For example: versus Choosing symbols wisely also could make the next step easier or more difficult. Of course, not choosing ones symbols wisely here –or not even having a clue as to how to choose symbols- poses a problem in this question. A suggestion for choosing a variable good be given, for example “remember that subsequent numbers could be modeled by n, n+1, n+2”. A tool should be able to ask for an expression, determine whether it is correct, and enable the student to rewrite the expressions. So the solution process is very important.
5	Of a function f(x) we know:  How much is  ? (Note: the original question is a multiple choice question) This question addresses whether a student has ‘gestalt’ quality: does her or she recognize similar parts of an equation. Behavior #6: flexible manipulation skills. Students would have to recognize that a function is not given, and hopefully be triggered that something else plays a role, in this case transformations. If this is recognized it should not be too hard to see that the function was shifted two to the right and the integral as well. Here the difficulty of the question lies in the fact that the function f is unknown. Students will probably wonder how the integral can be calculated without actually knowing the function. A more specific question arises when we choose a certain function, or first start with two question with given functions, and then the general statement. Hinting on the fact that this is a general statement - it holds for all function- is important. Therefore any chosen function could provide insight. Also the process of transformation of a function is an important concept. Here giving the possibility to plot functions could be helpful.