e-ISSN 2231-8542
ISSN 1511-3701

Home / Regular Issue / JTAS Vol. 29 (S3) 2021 / JSSH(S)-529-2021


The Effects of Different Rater Training Procedures on ESL Essay Raters’ Rating Accuracy

Souba Rethinasamy

Pertanika Journal of Tropical Agricultural Science, Volume 29, Issue S3, December 2021


Keywords: Assessing writing, rater training, rating accuracy, standardisation, validity and reliability

Published on: 30 November 2021

The study investigated the effects of three commonly employed rater training procedures on the rating accuracy of novice ESL essay raters. The first training procedure involved going through a set of benchmarked scripts with scores, the second involved assessing benchmarked scripts before viewing the scores. The third was a combination of the former and latter. A pre, post and delayed post-experimental research design was employed. Data were collected before, immediately after and one month after training. Actual IELTS scripts with benchmarked scores determined by subjecting expert IELTS raters’ scores through Multi-Faceted Rasch (MFR) analysis were used for the training and data collection purposes. Sixty-three TESL trainees were selected based on their pre-training rating accuracy to form three equally matched experimental groups. The trainees’ scores for the essays before, immediately after and one month after the assigned training procedure were compared with the official scores for the operational essays. Although the findings indicate that generally, rater training improves raters’ rating accuracy by narrowing the gap between their scores and the official scores, different training procedures seem to have different effects. The first training procedure significantly improved raters’ rating accuracy but showed a decreasing effect with time. The second training procedure showed immediate as well as delayed positive effects on raters’ rating accuracy. The third training did not lead to significant immediate improvement, but rating accuracy improved significantly after some time. This paper discusses the implications of the findings in planning efficient and effective rater training programmes.

ISSN 1511-3701

e-ISSN 2231-8542

Article ID


Download Full Article PDF

Share this article

Recent Articles