Emagister Eventos

Organizado por SERVICIO DE IDIOMAS UAM

RATER RELIABILITY IN ACLES CEFR-RELATED PRODUCTIVE SKILLS EXAMS

  • Fechas:

    Del 17/06/19 al 20/06/19

  • Lugar:

    calle Freud 1, Pabellón A, Universidad Autónoma de Madrid, Madrid, España (mapa)

Web del evento

Descripción ↑ subir

Much work has been carried out by ACLES members to ensure the production of valid test tasks for the written and oral production/interaction parts of their exams. However, for the productive skills if we cannot depend on the reliability of raters it is immaterial that test tasks are valid measurement instruments (Shaw "> Evidence needs to be provided that raters consistently apply appropriate ratings based on a sound understanding of the rating scale if the interpretation and use of test scores is to be valid.

This practical course will provide participants with tools to help them to produce more reliable ratings for their productive skills ACLES exams.

 
 
How to get to UAM
 
By BUS:
 
Línea 714: Plaza Castilla - Cantoblanco - Comillas 
 
Línea 827: Alcobendas - Cantoblanco  
 
Línea 827A: San Sebastián de los Reyes - Alcobendas - Cantoblanco      
 
Línea 828: Campo de las Naciones – Aeropuerto – Cantoblanco  
 
 
By TRAIN (Cercanías)
 
Línea C-4a Parla-Atocha-San Sebastián de los Reyes
 
Línea C-4b Parla-Atocha-Colmenar Viejo
 
 
 
Summary Schedule:

DAY 1 - Rating scales as a representation of the construct for productive skills (Gold standard) 

DAY 2 - Rater reliability

DAY 3 - Rater reliability

DAY 4 - Rater training "> For further details, please check the program.

Lugar ↑ subir

Programa ↑ subir

DAY 1

17 JUNE

9:30 - 12.00

12:30 - 15:00

(5 hours)

 

 

 

 

 

 

 

 

Rating scales as a representation of the construct for productive skills (Gold standard):

 

Relating exams to the CEFR

(Brief discussion of test tasks)

 

Examination of rating scales used by participants: problems and solutions

 

Analytic versus Holistic rating scales

 

Development of CEFR-related rating scales for productive skills

 

DAY 2

18 JUNE

9:30 - 12.00

 

12:30 - 15:00

(5 hours)

Rater reliability:

 

Consensus estimates

 

Consistency estimates

 

Measurement estimates: Introduction to Many Facet Rasch Measurement (FACETS)

 

DAY 3

19 JUNE

9:30 - 12.00

 

12:30 - 15:00

(5 hours)

Rater reliability:

 

Running FACETS

Analysis of FACETS output

 

-problem raters

-problem scripts

 

Rating scale functioning:

Brief analysis of FACETS output

 

DAY 4

20 JUNE 

9:30 - 12:00

12:30 - 15:00

(5 hours)

Rater Training:

 

Benchmarking and standard setting

Monitoring and feedback

 

Rater design plans for live administrations

 

 

Practical session:

 

Participants will have the opportunity to analyse their own data (or a data set which will be provided) in order to draw conclusions about possible future rater training

 

 

Ponentes ↑ subir

CAROLINE SHACKLETON

Teacher and test developer at the University of Granada's Modern Language Centre (Centro de Lenguas Modernas).

She holds an MA in Language Testing from the University of Lancaster and a PhD in Applied Linguistics (Language Testing) from the University of Granada.

Currently, she is an expert member of ACLES (Association of Higher Education Language Centers in Spain) and regularly provides training in language test development.

Eventos relacionados