PRE CONFERENCE WORKSHOP

WORKSHOP TITLE: Item Banking and Assembly of Test Forms

Presenters:

Angela Verschoor, Sebastiaan de Klerk

Presenters’ Bios:

Angela Verschoor is Senior Researcher at CITO, the Netherlands. With a background in discrete optimization, her interest is the development and application of automated test assembly (ATA), optimal design and computerized adaptive testing (CAT). She has been the driving force behind well over 30 operational CATs in Europe, as well as major improvements in large-scale projects such as the Dutch Final Primary Education Test, the Central Examinations, and the examinations for Dutch as a Second Language.

Other recent projects included the introduction of ATA in, amongst others, Russia, Kazakhstan, the Philippines and Italy.

In 2010 Angela was accredited as a Fellow of the AEA-Europe and in 2018 she won the Lifetime Contribution Award of the e-Assessment Association.

Sebastiaan de Klerk studied psychology at the University of Amsterdam. His research focuses on industrial psychology and statistics. It comprises the influence of high-power group collaboration on group members’ individual negotiation style and individual factors. In 2016, he finished his PhD at eX:plain, Cito, and the University of Twente. He focused on the efficiency and effectiveness of multimedia-based performance assessment as a replacement of performance assessment in vocational education. Besides building innovative assessments, he also worked on the psychometric modelling of the data flowing out of the assessment.
Currently, he works as a senior researcher at Cito, with special interest in innovative assessment types, detection of test fraud, and the relationship between technology and assessment.

Why AEA members should attend this workshop:

The workshop will offer an introduction into Item Banking and applications for test assembly from a practical point of view. Participants will gain insight in the do’s and don’ts when using an item bank for the purpose of developing assessment instruments, and will receive practical guidelines to use metadata and psychometric theory to assemble optimal tests based on an existing item bank. participants will have hands-on experience in using automated tools to make linear or adaptive tests, based on Item Response Theory (IRT) or Classical Test Theory (CTT). Main features of these applications will be addressed in the workshop. Participants will be able to understand and assess the usefulness of item banking in their own work.

Who this Workshop is for:

The workshop is aimed at those who want to know more about item banking and test assembly with a focus on applications. Participants might be novice or more experienced users. No prior knowledge is required to attend the workshop, although some knowledge on CTT and IRT would be welcome.

Participants will practice using software for some examples and are invited to bring their own laptops for practicing (Windows).

Overview:

The workshop starts with an introduction to item banking as part of the test development cycle, from the perspective of the test developer.

The first session of the workshop starts with some theory and best-practices: why is item banking an important issue; how can we make item banking be profitable for us, the test developers? An overview on item banking systems will be given, and participants will be encouraged to share their views and experiences with item banks.

In the second session participants will learn about the main features of the test construction process. The participants will practice specification of test requirements, using examples from the test construction experience of the presenters and, when available, from the participants themselves.

A short introduction in the use of Item Response Theory and Classical Test Theory in different types of test design will be given. The participants will practice with hands-on exercises to assemble their own test based on an example item bank.

Special attention will be paid to the development of multiple parallel test forms. The use of these multiple test forms will be discussed, as well as the requirements that must be fulfilled.

The topic of the last session will be various aspects of item bank maintenance and renewal: how can we identify potential shortcomings in the available item pool, what role do security issues and item renewal schemes play in a project? Developing long-term views in item banking will be the main topic in this session.

Preparation for the workshop:

No special preparation is required, the workshop format will be interactive allowing participants to discuss their own experience and/or problems. If available, participants are encouraged to bring their own item bank data for discussion. It is the belief of the workshop leaders that sharing experience in applications will stimulate and enable participants in solving educational measurement problems that they encounter in their practice or anticipate encountering.

Tentative Schedule