PRE CONFERENCE WORKSHOP

WORKSHOP TITLE: Unlocking ‘Assessment Cultures’

Presenters:

Stuart Shaw and Eleanor Andressen

Presenters’ Bios:

Stuart Shaw worked for international awarding bodies for over 20 years and was Head of Research at Cambridge Assessment International Education (Cambridge Assessment). Stuart is particularly interested in demonstrating how educational, psychological and vocational tests seek to meet the demands of validity, reliability and fairness. Stuart has a wide range of publications in English second language assessment and educational research journals. His assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). His most recent book is entitled ‘Is Assessment Fair?’ (Isabel Nisbet & Stuart Shaw) published by Sage (2020). Stuart is a Fellow of the Chartered Institute of Educational Assessors (CIEA) and a Fellow of the Association for Educational Assessment in Europe (AEA-Europe). He is also an Affiliated Lecturer at the Faculty of Education, University of Cambridge. Stuart is also a Member of the Board of Trustees of the International Association for Educational Assessment (IAEA), Chair of the IAEA Communications Committee and Co-Chair of the Scientific Programme Committee (AEA-Europe).

Eleanor Andressen has worked in qualification and assessment development and regulation for awarding organisations in England throughout her career and has a particular interest in the employer role in technical and vocational education and training. On completion of her PhD on this subject, she worked in the research team at Pearson UK with a particular focus on learning design and its impact on learner outcomes, including standards-based apprenticeships. More recently, Eleanor has been designing and delivering training on ‘Innovation and Management of Higher Vocational Education in the UK’, for university customers in China looking to reform their own systems. Eleanor has written and presented a number of times on employers and apprenticeships, from The impact of trailblazer standards on the delivery of apprenticeships’ JVET 7th July (2017) to the most recent on this subject Tensions and innovations: the impact on learning of Apprenticeships in ‘non-traditional’ organisational settings (2019), with a focus on the relationships between policy, design, learning and assessment. More recently, she has researched and presented on other aspects of assessment, including the impact of technology on large scale examinations, and issues of access around this (e-AA conference, June 2020) and the experiences of children taking secondary entrance examinations during the pandemic (AEA-E, November 2021). As a member of AEA-Europe, Eleanor chairs the regular AEA-E Assessment Cultures SiG webinars. She is a peer reviewer for the Higher Education, Skills and Work-based Learning journal, and a judge for the annual e-Assessment Awards.

Workshop contributors (in alphabetical order):
• Eleanor Andressen, UCL Institute of Education, England
• Peter Ludwig, University of Koblenz-Landau, Germany
• Isabel Nisbet, Faculty of Education, University of Cambridge, England
• Raphaël Pasquini, University of Teacher Education, State of Vaud, Switzerland
• Lise Vikan Sandvik, NTNU Norwegian University of Science and Technology, Norway
• Christoph Schneider, University of Trier, Germany
• Stuart Shaw, Faculty of Education, University of Cambridge, England
• Sverre Tevit, University of Oslo, Norway
• Fernando Morales Villabona, University of Teacher Education, State of Vaud, Switzerland
• Andrew Watts, Faculty of Education, University of Cambridge, England
• Lesley Wiseman, University of Glasgow, Scotland

Why AEA members should attend this workshop:

Within the global educational assessment community, we often refer to the concept of an assessment culture. But what exactly is an assessment culture? This workshop is intended to make the complexities around an understanding of assessment cultures less challenging and more readily understandable. All those connected with education should aim to understand the assessment principles which govern their work and why those principles are important. Such principles are influenced by individual perspectives – for example, historical, educational, philosophical, psychological, sociological or methodological – which are variously brought to bear on the work of assessment. The workshop provides opportunities for participants to consider the assessment cultures that affect their own work, through discussion and the exchange of information and ideas. It is hoped that insights can be gained into our own beliefs and assessment practices by comparing them with practices and cultural influences in other countries and contexts.

Who this Workshop is for:

Everyone who works in education and assessment will benefit from a shared understanding of assessment cultures and the opportunity to reflect upon the ways in which these influence their assessment practice. The workshop is envisaged as a resource for students of educational, psychological and vocational measurement and assessment, for key practitioners in assessment organisations who wish to gain a deeper understanding of the nature and influence of different assessment cultures, for those with an academic interest in assessment, for educational policy-makers and for the assessment novice who should be able to benefit from attending the workshop.

Overview:

Following an introductory overview, Stuart Shaw and Eleanor Andressen will open with a session which explores the links between language, culture, assessment, and ‘assessment cultures’. Assessment is a cultural practice shaped, for example, by belief systems, teaching and learning styles, epistemologies intrinsic to the learners’ cultural experiences, and the socioeconomic conditions preponderant in their cultural groups. Moreover, assessment design is shaped by societal values.

The following session, led by Isabel Nisbet, will look at what is meant by ‘values’ and their link with assessment culture(s) and will encourage participants to reflect on the values they bring to their own work. With the participants, Isabel will consider two issues relating to what constitutes a good assessment, and the value of assessment, drawing conclusions on whether ‘’assessment values’’ are different from, or a sub-set of, ‘’educational values’’.

Christoph Schneider and Peter Ludwig will then focus on specific features of the interpretations and connotations of the term ‘education’ in both academic and colloquial languages across European countries and regions. They will briefly outline how the German terms for education (namely ‘Erziehung’ and ‘Bildung’) have affected educational and assessment cultures in German-speaking countries. In dialogue with the participants, the session will seek to explore the different ways of how understandings of ‘education’ translate into practice in different cultures and contexts.

Next, Andrew Watts will provide an example of the use of history to explain the original nature of the English and Welsh national examinations systems. Andrew will focus on the use of university-based exam boards to run the system and the choice of multiple examination boards to implement a system which has been claimed to be over-academic and administratively complex. The aim will not be to find fault with the decisions that were made but to understand the reasons why they were arrived at. Participants will be invited to describe a situation in their own national assessments and begin to seek historical reasons for the structure of their own systems.

The next session will be a joint one on the broad subject of Assessment for Learning (AfL) and will focus on two jurisdictions: Switzerland and Norway. In the first half of the session, Raphaël Pasquini & Fernando Morales Villabona highlight how in the Swiss state of Vaud, there is a huge challenge to promote and implement AfL practices at all school levels. Raphaël and Fernando will invite participants to reflect on aspects such as AfL principles and teacher mindsets, to better understand the dynamic dimension of these and its role in AfL implementation. In the second half of the session, Lise Vikan Sandvik discusses findings from a nationwide research project conducted in Norway to determine the impact of the national programme called AfL on assessment cultures. Lise will show how a conceptual mapping tool used to investigate AfL practices in Norway could help identify disciplinary signature assessment and thereby could expand our understanding of characteristics of assessment cultures in different contexts.

In the final session before the plenary, Lesley Wiseman will raise the apparent issue of mistrust of educational measurement. If qualities of good measurement are the same in all fields (such as health, science), why is it then, that tensions and suspicions remain in the educational assessment context, which are not apparent in other measurement fields? To what extent, and why, is the mistrust and readiness to call into question the outcomes of educational measurement apparent internationally?

The concluding session will consist of a group discussion about how to tackle the issues raised in the workshop and how workshop participants can continue to actively engage with the Assessment Cultures SIG.

Preparation for the workshop:

Questions for participants to think about in advance:

• When implementing new routines or practices related to assessment: What kind of tensions or contradictions have you experienced in the assessment culture that you are a part of?
• To what extent is there mistrust of standardised summative assessment in your country or context, and why?
• Would you be able to identify and name ways of how understandings of ‘education’ translate into assessment practices in your culture/context?
• In your context, how does teachers’ assessment or testing culture impact AfL practices?
• What do you think good assessment looks like? How does practice in your context rate against your description? Do you think that your view is shared by others in your context – colleagues, policy-makers, student, parents, the wider public?

Suggested Chapter reading:

Alarcón, C. & Lawn, M. (2017). Introduction: Assessment Cultures. Historical Perspectives. In C. Alarcón & M. Lawn (Eds.). 11-22. Studies in the History of Education. Vol. 3. Peter Lang: Frankfurt a. M. pp.11-22

Tentative Schedule

Duration of each contribution: 15 minutes (maximum) followed by participant discussion/interaction (average 30 minutes).