Assessment of mathematics assessment: Discrepancies between standardised testing and teacher judgements in an Australian primary school context
Abstract
Withmoves afoot to review Australia's National Curriculum, perhaps it is timely toassess teacher assessment practices. This study compares the judgments thatteachers make on their students' mathematics achievement with results takenfrom Australia's National Assessment Program: Literacy and Numeracy (NAPLAN). Usinga sample of 2144 students, drawn from the Longitudinal Study of AustralianChildren (LSAC), the study develops two regression models: one with teacherratings of achievement as the outcome variable, and the other with NAPLANnumeracy results as the outcome. A number of individual and environmentalfactors are then regressed onto these outcome variables, and the magnitudes oftheir effects are compared. The results indicate a consistency between the twoassessment approaches, except for the assessment of students with specialneeds, where teachers significantly under-estimate the mathematical abilitiesof these students. Implications of these results are discussed.References
Asparouhov, T., & Muthen, B. (2006). Multilevel modeling of complex survey data. Paper presented at the Joint Statistical Meeting. ASA Sectiion on Survey, Seattle. Retrieved from: http://www.amstat.org/sections/SRMS/Proceedings/y2006/Files/JSM2006-000803.pdf
Australian Association of Mathematics Teachers. (2008). Position paper on the practice of assessing mathematics learning. Adelaide: AAMT.
Australian Curriculum Assessment and Reporting Authority. (2010). NAPLAN Achievement in reading, writing, language conventions and numeracy: National report of 2010. Canberra: ACARA. Available from: http://www.nap.edu.au/results-and-reports/national-reports.html
Australian Curriculum Assessment and Reporting Authority. (2012a). NAPLAN Numeracy Example test: Year 5. Canberra: ACARA. Retrieved from: http://www.nap.edu.au/verve/_resources/Example_Test_Numeracy_Y5.pdf
Australian Curriculum Assessment and Reporting Authority. (2012b). NAPLAN Reading Example test: Year 5. Canberra: ACARA. Retrieved from: http://www.nap.edu.au/verve/_resources/Example_Test_Reading_Y5.pdf
Australian Curriculum Assessment and Reporting Authority. (2013). Australian Curriculum: Mathematics (Version 5.1). Canberra: ACARA. Retrieved from: http://www.australiancurriculum.edu.au/Documents/K10/Mathematics%20curriculum.pdf
Bantick, C. (2014, January 25). Generalists not the answer to teaching woes, The Weekend Australian.
Bates, C., & Nettelbeck, T. (2001). Primary school teachers' judgements of reading achievement. Educational Psychology, 21(2), 177-187.
Begeny, J. C., Eckert, T. L., Montarello, S. A., & Storie, M. S. (2008). Teachers' perceptions of students' reading abilities: An examination of the relationship between teachers' judgements and students' performance across a continuum of rating methods. School Psychology Quarterly, 23(1), 45-55.
Bennett, R. E., Gottesman, R. L., Rock, D. A., & Cerullo, F. (1993). Influence of behaviour perceptions and gender on teacher's judgements of students' academic skill. Journal of Educational Psychology, 85(2), 347-356.
Bergin, D. A. (1999). Influences on classroom interest. Educational Psychologist, 34(2), 87-98.
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148.
Blakemore, T. J., Strazdins, L., & Gibbings, J. (2009). Measuring family socio-economic position. Australian Social Policy, 8, 121-169.
Bobis, J. (2009). The Learning Framework in Number and its impact on teacher knowledge and pedagogy. Sydney: NSW Department of Education and Training. Retrieved from http://www.curriculumsupport.education.nsw.gov.au/primary/mathematics/assets/pdf/cmitreport08.pdf.
Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513-531.
Byrne, B. M. (2001). Structural Equation Modeling with AMOS. Mahwah, NJ: Lawrence Erlbaum Associates.
Daraganova, G., & Sipthorp, M. (2011). LSAC Technical paper no.9: Wave 4 weights. Melbourne: Australian Institute of Family Studies.
Demaray, M. K., & Elliot, S. N. (1998). Teachers' judgements of students' academic functioning: A comparison of actual and predicted performances. School Psychology Quarterly, 13(1), 8-24.
Donnelly, K. (2010, October 18). Content drowns new curriculum. The Australian.
Donnelly, K. (2013, March 16). History curriculum sacrifices Western values at the altar of political correctness. The Australian.
Eckert, T. L., Dunn, E. K., Codding, R. S., Begeny, J. C., & Kleinmann, A. E. (2006). Assessment of mathematics and reading performance: An examination of the correspondence between direct assessment of student performance and teacher report. Psychology in the Schools, 43(3), 247-265.
Feinberg, A. B., & Shapiro, E. S. (2009). Teacher accuracy: An examination of teacher-based judgements of students' reading with differing achievement levels. The Journal of Educational Research, 102(6), 453-462.
Greenlees, J. (2010). The terminology of mathematics assessment. In L. Sparrow, B. Kissane & C. Hurst (Eds.), Shaping the future of mathematics education: Proceedings of the 33rd annual conference of the Mathematics Education Research Group of Australasia. (pp. 218-224). Adelaide: MERGA, Inc.
Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Abingdon, UK: Routledge.
Hay, P. J., & Macdonald, D. (2008). (Mis)appropriations of criteria and standards-referenced assessment in a performance-based subject. Assessment in Education: Principles, Policy & Practice, 15(2), 153-168.
Helwig, R., & Tindal, G. (2003). An experimental analysis of accommodation decisions on large-scale mathematics tests. Exceptional Children, 69(2), 211-225.
Hinnant, J. B., O'Brien, M., & Ghazarian, S. R. (2009). The longitudinal relations of teacher expectations to achievement in the early school years. Journal of Educational Psychology, 101(3), 662-670.
Hoge, R. D., & Coladarci, T. (1989). Teacher-based judgements of academic achievement: A review of the literature. Review of Educational Research, 59(3), 297-313.
Jussim, L., & Harber, K. D. (2005). Teacher expectations and self-fulfilling prophecies: Knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review, 9(2), 131-155.
Kaiser, J., Retelsdorf, J., Südkamp, A., & Möller, J. (2013). Achievement and engagement: How student characteristics influence teacher judgements. Learning and Instruction, 28, 73-84.
Kenealy, P., Frude, N., & Shaw, W. (1991). Teacher expectations as predictors of academic success. The Journal of Social Psychology, 131(2), 305-306.
Lowrie, T., Greenlees, J., & Logan, T. (2012). Assessment beyond all: The changing nature of assessment. In B. Perry, T. Lowrie, T. Logan, A. MacDonald & J. Greenlees (Eds.), Research in Mathematics Education in Australasia 2008-2011. Rotterdam: Sense Publications.
Martínez, J. F., Stecher, B., & Borko, H. (2009). Classroom assessment practices, teacher judgements, and student achievement in mathematics: Evidence from the ECLS. Educational Assessment, 14(1), 78-102.
Mashburn, A. J., & Henry, G. T. (2004). Assessing school readiness: Validity and bias in preschool and kindergarten teachers' ratings. Educational Measurement: Issues and Practice, 23(4), 16-30.
Muthén, L. K., & Muthén, B. (2011). Mplus User's Guide (Sixth ed.). Los Angeles, CA: Muthén & Muthén.
National Center for Educational Statistics. (nd). Early Childhood Longitudinal Study - Kindergarten (ECLS-K). Washington, DC: Department of Education.
Perso, T. (2009). Cracking the NAPLAN code: Numeracy in action. Australian Mathematics Teacher, 65(4), 11-16.
Pianta, R. C. (2001). Student Teacher Relationship Scale (STRS). Lutz, FA: Psychological Assessment Resources.
Ready, D. D., & Wright, D. L. (2011). Accuracy and inaccuracy in teachers' perceptions of young children's cognitive abilities: The role of child background and classroom context. American Educational Research Journal, 48(2), 335-360.
Ritter, D. R. (1989). Teachers' perceptions of problem behaviour in general and special education. Exceptional Children, 55(6), 559-564.
Rubie-Davies, C., Hattie, J. A. C., & Hamilton, R. (2006). Expecting the best for students: Teacher expectations and academic outcomes. British Journal of Educational Pscyhology, 76(3), 429-444.
Sanson, A., Nicholson, J., Ungerer, J., Zubrick, S., Wilson, K., Ainley, J., . . . Wake, M. (2002). Introducing the Longitudinal Study of Australian Children (LSAC Discussion Paper No. 1). Melbourne: Australian Institute of Family Studies. Retrieved from: http://www.growingupinaustralia.gov.au/pubs/discussion/dp1/index.html
Sideridis, G. D., Antoniou, F., & Padeliadu, S. (2008). Teacher biases in the identification of learning disabilities: An application of the logistic multilevel model. Learning Disability Quarterly, 31(4), 199-209.
Südkamp, A., Kaiser, J., & Möller, J. (2012). Accuracy of teachers' judgements of students' academic achievement: A meta-analysis. Journal of Educational Psychology, 104(3), 743-762.
Thomson, S., De Bortoli, L., & Buckley, S. (2013). PISA 2012: How Australia measures up. Camberwell, Vic.: Australian Council for Educational Research. Retrieved from: http://www.acer.edu.au/documents/PISA-2012-Report.pdf
Van Houtte, M., & Demanet, J. (2013). Curriculum tracking and teacher evaluations of individual students: Selection, adjustment or labeling? Social Psychology of Education, 16(3), 329-352.
Wright, R.J. (1994). A study of the numerical development of 5-year-olds and 6-year-olds. Educational Studies in Mathematics, 26, 25-44.
