The Federal agency in charge of NAPLAN is rushing through with plans to have robots mark next year’s NAPLAN tests despite their justifications being discredited by world-leading research.
Parents, teachers and principals have not been consulted about these radical plans and students have not been prepared for such a dramatic shift. NAPLAN has been a high-stakes and controversial test and this latest development will further bewilder youngsters who are subject to such testing.
The Australian Curriculum and Reporting Authority (ACARA) announced in a newsletter — quietly released in the recent school holidays — that they intend to use computer-robots to mark the 2018 NAPLAN tests. In justification they cited their own anonymous 2015 report that they claim guarantees that robots can mark children’s work just as well as teachers.
However, the ACARA Paper has been carefully examined by one of the world leaders in Assessment and Writing from the Massachusetts Institute of Technology [M.I.T.] , Dr Les Perelman, who describes the ACARA report as, “So methodologically flawed and so massively incomplete that it cannot justify any use of AES [automated essay scoring] in scoring the NAPLAN essays.”
Dr Perelman also reveals:
- That robots can only detect low grade attributes of writing;
- That robots cannot detect “the most important elements of a text”;
- That the ACARA “Report” uses evidence and data selectively and poorly;
- That ACARA ignores any authority critical of robot marking;
- That robots and humans should not be used in conjunction in marking;
- That even robots as grammar checkers are seriously flawed;
- That overseas evidence shows robot marking discriminating against some social groups;
- That introducing robots will encourage regressive teaching practices.
Dr Perelman’s full report, commissioned by the Federation, can be found at https://www.www.nswtf.org.au/files/automated_essay_scoring_and_naplan.pdf.
ACARA is trying on the ruse of introducing shared marking between their robots and what they call, in an Orwellian phrase, “human markers”. All of the perils of robots marking young kids’ NAPLAN efforts are still contained in such a process. Any degree of robot marking to assess students’ writing is simply wrong.
The ACARA plan to introduce robots into the marking of writing is revealed at
http://www.acara.edu.au/news-and-media/acara-update-archive/acara-update-archive-2017/acara-update-september-2017#3
The way forward is clear:
- Ministers and state education leaders must reject ACARA’s plan to bring robots into the marking of extended pieces of children’s’ work in NAPLAN;
- ACARA must withdraw its discredited “research” paper that purports to justify their Brave New World visions;
- Extensive and unhurried consultations must take place with parents and teachers before any substantial or radical changes to the marking of NAPLAN are even contemplated;
- ACARA must reveal the contractual arrangements around NAPLAN testing and marking to indicate all private arrangements that have been entered into that so vitally affect the futures of our young.