Master's Dissertation First Full Draft

4.7. Instruments

A range of instruments were used to conduct this experiment, including electronic hardware (devices), text passages and related test questions and a demographic questionnaire, among other instruments.

Devices

The devices which were used in this study are: touchscreen tablet computers, pens and A4 paper with printed text. In cases where participants did not have a tablet of their own, one was provided by the researchers. At three of the six sites, most participants were in possession of tablet computers which they made use of on a daily basis as part of their academic activities, many of whom had done so for three years. This implies a high degree of device and app familiarity both in general and more specifically for use in the classroom setting. Although allowing participants to make use of their own devices to participate in the experiment would result in a lack of standardisation, particularly in terms of screen size and performance, this disadvantage is substantially outweighed by the potential improvements in ecological validity which could be obtained. Participants were therefore allowed to make use of their own devices to complete the assigned experimental tasks. The models of tablet used by these individual participants varied significantly and screen sizes of these tablets varied from 7” to 10” measured diagonally.

Demographic Questionnaire

Each participant completed a demographic questionnaire consisting of three parts. The first part contains items which established participant age, grade, gender and school subjects taken at grade 10 – 12 level. The second part focused on the participant’s language experience and proficiency. This section is a shortened, adapted version of the Language Experience and Proficiency Questionnaire (LEAP-Q), a well-established measure of language proficiency developed by Marian, Blumenfeld and Kaushanskaya (2007). South Africa is a highly multi-lingual society and English is only spoken as a home language by a small proportion of South Africans (Statistics South Africa, 2012). The primary focus of this section, therefore, was to gain a measure of insight into the participant’s experience and proficiency with the English language. The final section of the demographic questionnaire contains two questions which addressed participant access to electronic devices (specifically laptop, desktop PC, tablet, smartphone and e-reader) and frequency of use. The demographic questionnaire is included as Appendix A.

Reading Material

The four texts used for this experiment were adapted from materials used by Mueller and Oppenheimer (2014). Prior to conducting these research experiments, a pilot study was conducted using one of four texts from Mueller and Oppenheimer (2014) as well as an extended factual article taken from British Broadcasting Corporation (BBC) News service of similar length (BBC News, 2015b). Based on the results of this pilot study, the four texts used by Mueller and Oppenheimer (2014) were chosen for use in the full-scale experiment and were adapted to make the texts more appropriate for South African grade 11 and grade 12 learners. These adaptations were implemented by the research team incorporating the principal researcher and research assistants, all of whom are postgraduate psychology students. Each text is approximately 1150-1200 words long and is centred on either bread, bats, respiration or vaccines. All four texts are included as Appendices D, E, F, and G.

Multiple Choice Tests

Approximately one week after reading/studying these texts, participants completed a 10-item multiple choice test based on the text they had read. All four tests were adapted from materials used by Mueller and Oppenheimer (2014). Each test consisted of five levels of or types of question and each test contained two questions belonging to each of the five question types/categories. Table 5 below lists each question type and provides an example question for illustration. Participants were given four possible answers to each question and were required to select the correct option (i.e. standard multiple-choice answer format). The questions used were taken from Mueller and Oppenheimer (2014), while the research team created the four possible answers for each question. All four tests used in the full-scale experiment are included as Appendices L, M, N, and O.

Table 5. Examples of question types used

Question type	Example of question type
Factual	What compound will form when calcium and chlorine react with one another?
Conceptual	If a person’s larynx is not working properly, what will the mostly likely result be?
Inferential	“Sometimes bats die while they are sleeping. What will happen if a bat dies while it is hanging upside down?” (Mueller & Oppenheimer, 2014, p. 6)
Application	“Psychologists have investigated a phenomenon known as “attitude inoculation,’ which works on the same principle as [medical] vaccination and involves exposing people to weak arguments [which are] against a viewpoint [that] they hold. What would this theory predict would happen if the person was later exposed to a strong argument against their viewpoint?” (Mueller & Oppenheimer, 2014, p. 6)
Seductive Detail (i.e. interesting but irrelevant information)	What were the names of the two sheep which were cloned at the University of Istanbul in Turkey in November 2007?