A corpus-driven investigation of connections between the language of written feedback information and ethnic attainment disparity in UK higher education

Strand: Assessment
Proposal category: Expansion study


Attainment disparity between Black, Asian and Minority Ethnic (BAME) and White students is an ongoing concern within the higher education (HE) sector, and more widely within society. Recent investigation of student degree outcomes in the UK indicates that this disparity, whereby White students are awarded higher degree qualifications than BAME students, at its starkest sits at around 20% nationwide (OfS 2018). This gap in academic outcomes increases during the HE journey of these particular student populations, despite sector-wide recognition of systemic inequality (UUK and NUS 2019).

Our work therefore seeks to better understand institutional practices and policies that may contribute to this ethnic attainment disparity. We specifically attempt to unpick the ways in which the ‘written feedback information’ (Boud 2013) that students receive may be connected to their degree award outcomes. This focus has two main drivers: 1. written feedback information given to assessed work is one of the main channels through which lecturers engage formally with individual students, throughout students’ HE journey, and 2. students at local and national levels have voiced ongoing dissatisfaction with feedback and assessment practices, disproportionate to other satisfaction indicators (e.g. National Student Survey results (OfS 2019)).

We previously looked at whether differences in the nature of written feedback information can be differentiated according to the variable of discipline, for which we developed a framework of feedback functions (Alsop and Gardner 2019). This framework is comprised of five main functions (‘advice’, ‘critique’, ‘observation’, ‘praise’, ‘query’) and a catchall ‘unclassified’ option. Each function is subdivided three times, firstly by focus (e.g. ‘advice’ splits into ‘suggestion’ and ‘instruction’), then by orientation (‘specific’ or ‘general’), and then by aspect (‘future’ or ‘present’). We found that it was possible to differentiate the nature of the written feedback information received by individual students based on the variable of discipline.

In this paper, we explore further whether we can use this framework as a starting point to investigate whether a relationship exists between students’ ethnicity and language patterns within the written feedback information they receive. To do so, we applied our framework to 50 texts written to students from different ethnic backgrounds within a single discipline in a post-92 HE institution in the UK. Each file in the small corpus includes the feedback text written by the instructor to the student, and also metadata information about that student. Metadata variables include socioeconomic information (such as ‘ethnicity’, ‘gender’, and ‘home postcode’) and academic information (such as ‘assignment grade’, ‘discipline’, and ‘level of study’). The variable of ethnicity is self-identified when students apply to HE institutions in the UK (see HESA 2018). After manually encoding all texts, using individual comments as the unit of analysis, we looked at the relationship between students’ ethnicity, grade attained, and the language of the written feedback that they receive.

We will discuss findings from the analysis of this small dataset. Preliminary results indicate that little differentiation is evident in terms of the categorisation of written feedback information at the highest level of overall function in relation to binary categories of self-identified ethnicity (BAME and White). A more granular investigation of feedback function at the level of focus and orientation indicates some differentiation in the language of the feedback written to White and BAME student populations. Further differentiation is evident in the language of written feedback information at the level of overall function between the distinct ethnic populations that comprise the broad BAME category. Overall, there is greater differentiation in the language of written feedback information at the level of particular functions within BAME populations than between White and BAME populations.

As part of a wider research project, what this stage of analysis confirms is the importance of granularity when investigating institutional practices and student academic outcomes. In the context of the ethnic attainment gap, we do not propose a direct causal relationship between the language of written feedback information and student ethnicity, but rather suggest that written feedback information may constitute one means by which certain (already disadvantaged) groups of students are further, and cumulatively, disengaged by higher education practices.

Analysis also resulted in minor amendments to the 2019 working feedback framework; we added ‘neutral’ as a branch of the aspect attribute, alongside ‘present’ and ‘future’. This amendment is intended to capture more finely lecturer intention so that we can later compare this to student perceptions, which we anticipate will be a productive gap to explore.

This small study gives us confidence to progress to the next stage in our broader research project, which is to extend both the scale of the corpus and also the method of analysis to include investigation of the intersection of impacting variables.


References
Boud, D. And Molloy, E. (eds.) (2013). Feedback in Higher and Professional Education. London and New York: Routledge.
HESA (2018). Fields required from institutions in all fields: Ethnicity. Retrieved from https://www.hesa.ac.uk/collection/c18051/a/ethnic
Office for Students (OfS) (2018). Data and analysis: Differences in student outcomes. Retrieved from https://www.officeforstudents.org.uk/data-and-analysis/differences-in-student-outcomes/
Office for Students (OfS) (2019). National Student Survey. Retrieved from https://www.officeforstudents.org.uk/advice-and-guidance/student-information-and-data/national-student-survey-nss/
UUK and NUS (2019). Black, Asian and minority ethnic student attainment at UK Universities: #closingthegap. Retrieved from https://www.universitiesuk.ac.uk/policy-and-analysis/reports/Pages/bame-student-attainment-uk-universities-closing-the-gap.aspx