Master's Dissertation First Full Draft

4.11. Data analysis

Each valid participant response involved a completed 10 item test score. As described in Chapter 4.7., four different text passages (and accompanying texts) were utilized in this experiment. In order to account for perceived and actual differences in difficulty across these four text passages, all test scores were standardised to z-scores, with each of the four text passage groupings being normalized separately. These standardisations were conducted across the whole sample, rather than separately for smaller groupings/sites.

Research Question 1

Reading-only condition: a non-parametric Mann-Whitney U test is used to analyse between-subjects (one-week delay) data, while a parametric paired-samples t-test is used to analyse within-subjects data. Independent variable (IV): device condition; dependent variable (DV): test scores

Research Question 2

Reading and note-taking condition: a non-parametric Mann-Whitney U test is used to analyse between-subjects (one-week delay) data, while a parametric paired-samples t-test is used to analyse within-subjects data. IV: device condition; DV: test scores

Research Question 3

For both the between-subjects (one-week delay) and within-subjects conditions, the results obtained for research questions 1 and 2 are compared and contrasted.

Research Question 4

Data from the between-subjects (two-to-three-day delay) condition are compared to data from the between-subjects (one-week delay) condition.

Supplementary Analysis

A supplementary analysis is conducted to evaluate the impact self-report English proficiency on test results. A second supplementary analysis is conducted to evaluate whether any performance differences exist for gender.

Chapter 5

Results

5.1. More detailed participant demographic information

More detailed participant demographic information is presented below, focusing on English language background and self-report proficiency (as the language in which the experiment was conducted), and is split into three sections – between-subjects (one-week delay), within-subjects, and between-subjects (two-to-three-day delay). The ‘English Home Language’ and ‘First Additional Language’ in each table refers to the level at which participants study English as an academic subject. The majority of South African secondary school learners do not speak English as their home language, although a significant proportion of these learners still study English at home language level (Statistics South Africa, 2012), necessitating additional items to build a better picture of language background and self-report proficiency.

1. 
   Between-subjects (one-week delay)
   

Table 6. Between-subjects (one-week delay) demographic information

	Reading only		Note-taking
	Paper	Tablet	Paper	Tablet
Total number (N)	33	12	59	26
Average age (years)	18.2 [25] ^*	18.4 [7]^*	17.0 [49]^*	17.7 [21] ^*
Number of male participants	12^1(39%)	4^2 (44%)	23^3 (42%)	9 (35%)
Number of female participants	19^1 (61%)	5^2 (56%)	32^3 (58%)	17 (65%)
English Home Language (subject)	21 (64%)	7 (58%)	57 (97%)	14 (54%)
English First Additional Language (subject)	12 (36%)	5 (42%)	2 (3%)	12 (46%)
Average estimated percentage of time participant is exposed to the English language (%)	40.9% [24] ^*	36.4% [7] ^*	49.7% [47] ^*	37.3% [19] ^*
English most dominant language? (% of participants)	64% [25] ^*	57.1% [7] ^*	71.4% [49] ^*	42.1% [19] ^*
English as first language of acquisition (%)	28.0% [25] ^*	0% [7] ^*	28.6% [49] ^*	5.3% [19] ^*

^* Figure in brackets shows N used to calculate. Variable N’s resulted from missing demographic data

^{1, 2, 3} 2, 3 & 4 participants did not answer, respectively

The average age of participants ranges from 17 to 18.4 years of age, while the gender distribution across cells is skewed in favour of females, with female to male ratios of approximately 1.6:1 for the paper reading-only condition; 1.25:1 for the tablet reading-only condition; 1.4 for the paper note-taking condition; and 1.9:1 for the tablet note-taking condition. A higher proportion of participants report taking English at home language level than at the level of first additional language, with slightly more taking English home language for the tablet reading-only (58%) and note-taking (54%) conditions and substantially more for the paper reading-only (64%) and note-taking (97%) conditions.

No more than 29% of respondents in any cell listed English as their first language of acquisition and for no condition did the participants (on average) estimate that they were exposed to the English language more than 50% of the time. Despite this, however, a much larger proportion of students listed English as their most dominant language. Although this may seem somewhat strange at first, when one considers the ever-increasing prevalence and predominance of the English language in various aspects of modern life, particularly among younger generations combined with the prevalence of learning English at the level of home language already displayed in this sample, this very high proportion of participants listing English as being their most dominant language seems not only plausible, but very possible that these statistics are fairly close to being representative of national demographics for this age group.

2. 
   Within-subjects
   

Table 7. Within-subjects demographic information

	Reading only	Note-taking
Total number (N)	27	42
Average age (years)	16.45 ^1	16.75 ^2
Number of male participants	8 (30%)	24 (57%)
Number of female participants	19 (70%)	18 (43%)
English Home Language (subject)	27 (100%)	40 (95%)
English First Additional Language (subject)	0 (0%)	2 (5%)
Average estimated percentage of time participant is exposed to the English language (%)	53.3%	65.5%
English most dominant language? (% of participants)	80% ^1	80.5% ^2
English as first language of acquisition (%)	15% ^1	41.5% ^2

1. 
   7 participants did not answer – calculated using remaining 20
   
2. 
   1 participant did not answer – calculated using remaining 41
   

The average age of participants in each sample group (reading only and note-taking) is within the expected 16-18-year age range. All of the participants in the reading-only sample study English at home language level, while the vast majority of those in the note-taking sample (40 out of 42 participants) study English at home language level. However, this is not a good measure of home language or level of multilingualism, as demonstrated by the statistics in Tables 7 and 8. A very high proportion of participants claimed to be able to speak two or more languages, with English being the first language or home language of only a small proportion of participants, particularly for the reading only sample (15%) and to a lesser extent for the note-taking condition (41.5 %). The former is consistent with national demographics, while the latter is significantly higher than general population statistics (Statistics South Africa, 2012). In the reading-only sample, participants (on average) estimated that 53.3 % of their general language exposure was to English, with this figure being 65.5% for the note-taking sample. Given the ever-increasing prevalence and predominance of the English language in various aspects of modern life, particularly among younger generations, it seems possible that these statistics are fairly close to being representative of national demographics for this age group. Finally, for gender, the reading only condition has a markedly skewed distribution, with more than twice as many female participants compared to male participants (a ratio of approximately 3 males to every 7 females or 1:2.3), while the note-taking condition contains a somewhat less skewed (albeit still uneven) gender distribution, with males outnumbering females at a ratio of approximately 4 males to every 3 females (or 1.33:1).

3. 
   Between-subjects (two-to-three-day delay)
   

Table 8. Between-subjects (two-to-three-day delay) demographic information

	Note-taking
	Paper	Tablet
Total number (N)	16	9
Average age (years)	16.8	16.7
Number of male participants	3 (19%)	2 (22%)
Number of female participants	13 (81%)	7 (78%)
English Home Language (subject)	3 (19%)	3 (33%)
English First Additional Language (subject)	13 (81%)	6 (67%)
Average estimated percentage of time participant is exposed to the English language	32.3%	45.7%
English most dominant language? (% of participants)	25%	55.6%
English as first language of acquisition (%)	0%	0%

The average age of participants in this sample is very uniformly distributed at 16.7–16.8 years of age across both device conditions. The gender balance is, however, very heavily skewed in favour of females. This sample is the only one of the three where a greater number of participants study English at the level of first additional language rather than at home language level. None of the participants in this sample had acquired/learnt English as their first language. Both these are more (although not fully) representative of national demographics than the previous two sections (Statistics South Africa, 2012). The average estimated percentage of time that participants are exposed to the English language is fairly similar to the proportions reported in the one device only section. Given that none of the participants in this sample are English first language speakers, the percentage of participants listing English as being their most dominant language is also comparatively large. As discussed in the previous subsection, when one considers the ever-increasing prevalence and predominance of the English language in various aspects of modern life, particularly among younger generations combined with the prevalence of learning English at the level of home language already displayed in this sample, this very high proportion of participants listing English as being their most dominant language seems not only plausible, but very possible that these statistics are fairly close to being representative of national demographics for this age group. It must be noted, however, that the sample sizes in the sub-section are very small.