A meta-Analysis of Teaching and Learning

2.4Developing social understandings

In a paper written as a keynote address delivered at the Higher Education Close Up 4 conference held at the University of Cape Town in July 2008, Haggis provides a review of more than forty years worth of research on teaching and learning in higher education. Her analysis shows that, over the past forty years, work on student learning in higher education has taken a ‘predominantly individualistic approach to the study of student learning’ (p.6) in spite of the fact that other disciplinary areas such as psychology and sociology were developing ‘a range of social and interactional perspectives which explore the relationship between individual and ‘context’ in a variety of much more complicated and nuanced ways’ over the same period (p.6).

In concluding her review, Haggis (2008:10) notes that:

... what we know about student learning depends on where we look, and is always a reflection of specific purposes and interests, which are tied to particularities of special and temporal contexts. One characteristic of the theoretical shifts which I argue have been largely ignored is an increasing demand for critical reflexivity; a call to pay attention not only to issues such as gender and power, but to turn this attention back on all of the practices and assumptions of teachers and researchers themselves. Higher Education has arguably focussed most of its efforts until very recently upon attempting to shore up certainties in relation to knowledge of students as ‘other’, and has not been particularly good at examining its own cultures and ways of being. In the wider world of Education, ‘what we know’ is contradictory and contested, and is understood to be rooted inextricably in value positions. It is only recently in Higher Education, however, that a range of different value positions have begun to emerge, and finding ways to engage with value positions is still arguably problematic.

The range of ‘value positions’ which Haggis notes have only just begun to emerge in higher education relate to work rooted in what, in this report, have been termed ‘social’ understandings of learning and teaching. For Haggis, questions which need to be asked about student learning are not ‘What can we discover about how individuals learn?’, ‘What are the implications of our knowledge about individual learning for classroom teaching and curriculum design?’, How can we get our students to take a deep approach to the content of our curricula?’ or ‘What is going on outside the classroom which might impact on learning outcomes?’ but rather questions such as ‘How does the way we speak, and what we ask students to write, create impediments to students’ learning?’ (p.7).

As the review in the preceding section has attempted to show, this sort of focus on ‘us’ (the academics/the disciplines/the institution) rather than ‘them’ (the students) has long been perceived as offering the potential to offer understandings of the patterns of poor performance in the South African higher education system by experienced practitioners in the field of Academic Development/Teaching and Learning. For reasons also outlined above, this sort of approach has tended to be marginalised in the face of the dominance, and concomitant lack of questioning, of autonomous accounts of student learning. Given this position and also that the focus of an audit is the institution, it is appropriate that any analysis of teaching and learning should also be centred on the institution and draw on social understandings of learning and teaching. This is a central assumption of the research on which this report is based.

If learning and teaching are to be examined through the lens of critical social theory, then it is also necessary to develop an analytical framework for the examination which is also ‘social’. In order to do this, the research has drawn on the work of Roy Bhaskar (1979) and Margaret Archer (1995, 1996, 1998) outlined in the next section of this report.

3An analytical framework for the research

3.1Bhaskar’s Critical Realism

From a philosophical perspective, arguably the most coherent critique of institutional audits in South Africa (Luckett, 2007) is made from a position based in Roy Bhaskar’s Critical Realism. According to Luckett, the methodology used for institutional audits in South Africa is essentially pragmatic in that it assumes that empirical analysis, and more specifically the recommendations and commendations based on that analysis, have the power to bring about change. Luckett is sceptical of this position because of the realist understanding of a stratified and layered ontology.

Bhaskar (ibid) identifies three ontological layers: the empirical, the actual and the real. For Bhaskar, the real, the deepest layer of reality, consists of underlying structures and causal mechanisms which give rise to events in the world. This layer is intransitive. Bhaskar then identifies the actual as the domain of events – what actually happens when structures and mechanisms are activated. The empirical then consists of commonsense experience – what we come to know as a result of sensory experiences, a process which is tempered by our own historicity and social location. Our understanding of the empirical is thus fallible.

This stratified ontology requires an understanding of the world as an ‘open’ system. Danermark et al. (2002:206) define an open system as one in which there are no clear connections between cause and effect. The lack of cause and effect is explained by noting that ‘[w]hen generative mechanisms operate in combination with each other, the more mechanisms involved, the more difficult to anticipate the outcome.’ The purpose of research based in a critical realist ontology is to identify and try to understand the structures and mechanisms which operate as ‘causal forces’ at the level of the real. The concept of ‘causal forces’ or ‘causal powers’ rather than direct causation entails an understanding that objects of study have intrinsic properties (structures and mechanisms) which are capable of generating events. These causal properties exist (and are thus ‘real’) regardless of whether they are exercised or whether they are known (identified). Critical realist research therefore involves the identification of these basic underlying properties and mechanisms.

In the context of institutional audit, the adoption of Bhaskar’s ontology entails an acknowledgement of the existence of structures and mechanisms at the level of the real which give rise to multiple events at the level of the actual – events in life of the institution. Such events would include lectures, meetings, an academic sitting down to assess student’s work, a workshop to develop a curriculum, a student learning that she has failed a course by reading a list of student numbers and marks on a noticeboard and so on. The empirical then refers to contesting interpretations or experiences of those events. An institution, for example, might make claims about its student profile (i.e. the number of students who actually study at the university) at the level of the actual which is based upon one set of experiences (at the level of the empirical) of what is entailed in the production of a just society. The institutional account is thus socially constructed and theory laden and is thus partial.

Luckett’s position is that the ‘flat ontology’ of audit methodology, which essentially operates only at the level of the empirical, ‘fails to penetrate the level of the real and uncover the workings of social structure and social agency’ (2007:7). The account provided by an audit analysis, is thus partial, potentially fallible and does not have the power to bring about the change envisaged by those who developed it.

The position taken in this research is that, while an audit panel might work at the level of the empirical (and this is the level at which any analysis must begin since all of our experience is located at this level) what the panel in fact does, albeit unknowingly, is ask the question ‘If this is what the world (i.e. the institution) looks like (to us), what must the world (i.e. the institution at the level of the real) really be like?’ What the panel then does, again albeit unknowingly, is delve beneath the level of commonsense experience in order to identify mechanisms from which the institutional account at the level of the empirical has emerged. This process then allows them to challenge the institutional account.

An example may help to illustrate this point. In a Self Evaluation Portfolio, an institution might state that its student population is 52% black. This claim exists at the level of the actual and the empirical. 52% of students attending lectures, writing examinations etc. (at the level of the actual) may indeed be of African origin although they would also be of many different nationalities and from different social classes. At the level of the empirical, however, the observation (and experience) of the number of black students can be understood as emerging from the mechanism of ‘aggregation’ – of aggregating all black students regardless of their nationality and social class into a single category. The audit panel which challenged a claim made at the level of the empirical by disaggregating the broad category ‘black’ into South African black and international black would be identifying this mechanism and thus moving beyond the level of the empirical, albeit unconsciously.

Luckett’s claim about the failure of audit methodology to penetrate the level of the real can therefore be seen to be only partially true. The issue with the claim would appear to relate to the extent to which an audit panel might consciously attempt to identify the workings of social structure and agency and the extent to which it delved sufficiently deeply (and analytically) in order to do so.

The research on which this research is based draws on previous work (Luckett, 2008, Quinn & Boughey, 2008) on audit methodology in order to explore further the usefulness of realist ontology in better understanding South African universities. In the context of a meta-analysis of teaching and learning in audit processes, the research aims to identify the structures and mechanisms (and, thus, system level issues) which have given rise to the teaching and learning ‘events’ which exist at the level of the actual and teaching and learning related experiences which exist at the level of the empirical. The identification of these structures and mechanisms will then better allow an evaluation of the extent to which audit processes have contributed to the potential for change.