Measuring the Tough to Measure

Published:
Topics: Education Policy

By Jesse Moyer

Ever since the convening KnowledgeWorks hosted focused on creating an assessment and accountability system that supports competency education, which you can read about here and here, I have been on an assessment bent. Specifically, I have been thinking and reading a lot about how to assess non-cognitive competencies (non-cognitive competencies are also known as disposition, interpersonal and intrapersonal skills, or 21st century skills or competencies).

There is a lot to think about when assessing non-cognitive skills. StriveTogether has done some great research around figuring out which of these skills actually matter when it comes to academic achievement. While I think this work is extremely interesting and very worthwhile, I am more interested in how to measure these skills.

That is where a new report from the Asia Society comes in. For someone without an assessment background, like me, this report was really helpful. In addition to outlining which competencies they believe have a direct impact on academic achievement, similar to the StriveTogether report linked above, it provides major themes to consider when attempting to pick an assessment: instructional, practical, and technical. For more detail:
Instructional

  • Formative (informs learning) or summative (validates learning)
  • Provides actionable info for teachers
  • Useful feedback for students
  • Grade/context appropriate
  • Meaningful, engaging, authentic for students
  • Encourages effective teaching/learning
  • Practical
  • Cost
  • Ease of training (for teachers to administer)
  • Ease of scoring (for teachers)
  • Ease of administration (for teachers)
  • Ease of technological implementation (computers)
  • Technical
  • Reliability (consistently produce the same score across time, absent more learning)
  • Validity (measures what it supposed to measure)
  • Fairness (measures across student populations)

Finally, the report provides a great chart containing the different types of assessments (Multiple choice, open response, self-report, performance, portfolio, cross-cutting) and how they should be used.

The more work KnowledgeWorks does around competency education, the more I believe assessing non-cognitive skills will be one of the most important, and difficult, things to accomplish. I look forward to learning, and sharing via this blog, more about it.