Understanding Attainment Disparity: The Case for a Corpus-Driven Analysis of the Language used in Written Feedback Information to Students of Different Backgrounds

Background: Disparity of attainment between different groups of students in UK higher education has been correlated with ethnicity (UUK & NUS, 2019). For example, students who declared their ethnicity as Black were 20% less likely to graduate with a top classification than those who declared their ethnicity as White (OfS, 2018a). The causes of such attainment gaps are complex, and one important factor may be the nature of the feedback given by academic staff on assignments written by different groups of students. This paper aims to explore the feasibility of investigating this hypothesis by analyzing written feedback and looking for patterns in feedback given to different groups of students. Literature Review: Research on attainment among Black and Minority Ethnic (BAME) students in the UK has explored a number of aspects, and has generally concluded that there are issues of “belonging” (Richardson, 2015), particularly in institutions where the majority of academic staff and students are White, but that no single variable can explain the disparity. The wording of feedback on lower-scoring papers has been shown to be more impersonal and distant than that given to students on higher-scoring papers (e.g., Gardner, 2004), which has the (unintended) result of increasing the sense of belonging of higher performing students in ways that can build incrementally over the years of a degree course. While there have been many such small-scale studies of written feedback, none have aimed to collect large quantities of authentic written feedback for analysis. Research Questions: The hypotheses that drive our exploration are that written feedback information (WFI) (Boud & Malloy, 2013) is worded differently to different groups of students, and that there is a direct relationship between this aspect of feedback and academic attainment as measured by grades on summative assessments. Specifically, we asked: 1. Can a framework of WFI functions be developed for our data that share a meaningful set of attributes? 2. Can these categories be used to differentiate WFI to different groups of students? Methodology: A small pilot corpus was compiled from written feedback comments on twelve student assignments from two large Faculties. Metadata was added to each file, and the WFI comments were annotated and analyzed according to a framework developed in a branching format through a recursive construction process informed by the literature reviewed and the data in the corpus. This technique was used to characterize the WFI styles of the two Faculties. Results: The results show that all WFI comments could be classified using the novel systematic framework developed, and that its binary nature enabled ready cross-tabulation with metadata variables. Praise and critique were found to be most frequent, with specific praise of ideas (P1A) accounting for 68% of all praise, and specific critique of content (C1A) accounting for 49% of all critique. Observations tend to be the longest feedback comments (average 15.4 words). When the two Faculties are compared, two different feedback styles are evident, with Fac1 providing more advice, query, and observation style feedback than Fac2, and Fac2 providing more praise and critique than Fac1.