Master's Dissertation First Full Draft

5.7. Supplementary analysis

A supplementary analysis was conducted to examine two additional factors, namely: (1) the impact self-report English proficiency on test scores (given the highly multilingual nature of the samples obtained) and (2) due to the gender imbalances observed in some sample groups, whether any significant differences are observed for gender.

1. 
   Impact self-report English proficiency on test scores
   

Bivariate correlation analysis was used to determine whether any statistically significant correlations exist between the self-report English proficiency and test scores. Analyses were conducted both for participants in the between-subjects (one-week delay) and within-subjects sample groups. Self-report English proficiency scores were calculated using three self-report items from the adapted LEAP-Q –estimated percentage of time exposed to the English language, most dominant language, and order of language acquisition (i.e. whether English was acquired as a first, second, third etc. language).

Table 15. Between-subjects – impact of English proficiency on test scores

	N	Pearson Correlation	Sig.
Reading-only
Factual	31	.005	.979
Conceptual	31	.082	.662
Reading and note-taking
Factual	66	.260	.035*
Conceptual	66	.254	.040*

* significant result

Participants in the reading-only condition showed no statistically significant correlation between self-report level of exposure to English and test scores for either question type. Participants in the reading and note-taking condition did, however, show stronger correlations which are statistically significant for both question types. Table 16 below shows the results for participants in the within-subjects sample group.

Table 16. Within-subjects - impact of exposure to English on test scores

	N	Pearson Correlation	Sig.
Reading-only
Factual	38	-.084	.734
Conceptual	38	.207	.395
Reading and note-taking
Factual	82	.284	.072
Conceptual	82	.318	.043*

* significant result

Similarly to participants in the between-subjects (one-week delay) reading-only sample group, participants in the within-subjects reading-only group showed no statistically significant correlation between self-report English proficiency and test scores. However, for participants in the reading and note-taking group, a statistically significant correlation was observed for conceptual questions.

Given the correlations observed in both the between- and within-samples groups, a covariate analysis of variance was conducted to examine the impact of language proficiency. These results are included as Appendix O. When this set of tests was conducted, the following results were observed:

• 
  All previously non-significant results remain non-significant.
  
• 
  The previously statistically significant results for participants in the reading-only factual questions condition from both between- and within-subjects disappear.
  
• 
  The previously statistically-significant result for participants in the within-subjects conceptual questions note-taking condition remains.
  

However, incomplete data from some participants results in the further reduction of sample sizes for this covariate analysis when compared to the previous analyses (which did not look at language). This sample size reduction results in already-small samples such as the between-subjects reading-only condition (N=33, 12) becoming even smaller (N=24, 7). This renders the between-subjects reading only analysis non-parametric, which suggests the ANCOVA is not useful for this particular data subset. Further assumptions required to conduct an ANCOVA include the dependent and covariate variables being continuous (met), that the independent variable be categorical (met) and homogeneity of variances - met (Glass, Peckham & Sanders, 1972). Additionally, the composite measure of English proficiency is still not a bona fide measure of English language proficiency. Meaningful conclusions can therefore not be drawn on the impact of English language proficiency – further research is required. This suggests that further caution need be applied in interpreting the original analyses as well.

2. 
   Gender differences
   

Given the gender imbalances observed in some of the analyses conducted in this chapter, a series of independent-samples t-tests were conducted to determine whether any statistically significant differences exist between male and female participants for the various cells. Table 17 below shows the means for these various cells, with the asterisk (*) marking corresponding means which are statistically significantly different from one another.
Table 17. Gender differences in mean test scores

	Between-subjects (one-week)		Within-subjects
	Mean (N)		Mean (N)
	Male	Female	Male	Female
Reading-only
Factual	.19 (16)	.06 (24)	.09 (16)	.05 (38)
Conceptual	.07 (16)	-.20 (24)	.2 (16) *	-.24 (38) *
Reading and note-taking
Factual	-.17 (32)	.10 (49)	-.07 (48)	.12 (36)
Conceptual	-.06 (32)	.06 (49)	.06 (48)	.001 (36)

* statistically significant difference; p = .037

While differences in mean scores were observed in all cases, only one statistically significant difference was observed – male participants in the within-subjects reading-only condition performed statistically significantly better than their female counterparts for questions which tested conceptual recall.