At KnowledgeWorks we are well into the third year supporting four districts pioneering student-centered learning practices. As part of the North Dakota Personalized, Competency-Based Learning Initiative, we are working with WestEd to undertake a formative and summative evaluation of the work.
We sat down with WestEd’s Natalie Lacireno-Paquet, director of research and evaluation, and Lauren Agnew, research associate, in the beginning of our engagement to reflect about how this project might be different from more traditional evaluation projects they have been part of. The North Dakota initiative is not a traditional model that has a clear intervention with pre/post measures at set markers along the way.
“I really came to understand that systems transformation work that you’re trying to support from the ground up as opposed from the top down,” said Lacireno-Paquet. This kind of evaluation poses some challenges for researchers because it’s not a set intervention and not always easy to find other schools or districts to compare against.
Another notable difference in this evaluation process is that the focus, especially in the early years, is on collecting formative evaluation data and helping the school districts plan and improve their practices. According to Lacireno-Paquet, “One of the things that we really appreciated is the partnership – it’s not like we’re doing this evaluation for KnowledgeWorks or for the districts. It very much feels like a partnership with all the stakeholders involved.”
Learn about one North Dakota high school learner’s experience in an eye-opening internship developed her skills, shaped her post-secondary plans and helped her navigate uncertainty.
Read now >
After the first site visits, the districts seemed excited about having information they can share back with their teams. Sometimes it’s easier to have somebody else hold up the mirror of what are you hearing and what are you seeing – and getting that feedback in real-time, rather than waiting until the end of the year or the end of the initiative. Districts are surprised by how much they have in common despite many apparent differences in district size, location, student body and the fact that they may be at different stages in the visioning process.
The North Dakota Personalized, Competency-Based Learning Initiative is complex and not a cookie-cutter approach. It needs to be built from the ground up but with support from the top. “It’s going to look different in every place, and that’s ok – it’s actually kind of the point,” said Agnew. “But even with these differences, the outcomes we are looking for are the same.”
What are the right outcomes if not standard academic outcomes?
At KnowledgeWorks, we previously examined the impacts of personalized, competency-based learning in the RSU2 school district in Maine in a study that focused almost exclusively on academic outcomes. Analyzing academic outcomes is important but not sufficient. Some takeaways from this analysis were:
- Schools and districts must prioritize developing a culture of collecting and utilizing data to understand what works for students and to improve classroom practices.
- A strong culture of data should include academic data as well as data that illustrates student progress in building the social-emotional skills and dispositions necessary for success in an uncertain future.
Access to this data is important to understanding a key value proposition of personalized, competency-based learning: that this approach helps build student mastery, as well as student agency and ownership of learning.
Student reflection, self-checking and conferencing ensure this North Dakota middle school teacher has what she needs to meet students where they are.
Learn more >
Here are a few things that are not well-measured in readily available assessment data but will be important to understanding the success in North Dakota:
- Transparency of learning targets, mastery expectations and flexible assessments of student progress
- District and school staff regularly talk about what and how students are learning and how they prefer to learn
- Students of all ages can readily explain what and how they are learning
- Teachers are passionate about their work but honest about the challenges they face and what they need or can do to improve
- Regular demonstrations of student agency through voice and choice as well as flexible pacing
- Teachers listen to students and provide opportunities for peers to talk and work together
Together, with KnowledgeWorks North Dakota learning communities, we are implementing innovative research practices – and teaching and learning practices – to enable effective system transformation to support every learner.
Formerly with KnowledgeWorks, Geoff Zimmerman is the Principal at Shared Measure, a consulting firm that works with organizations and schools to develop innovative strategies around shared measurement and data systems, outcomes and indicators, evaluation, continuous improvement, and organizational learning and development.
Measuring the impact of personalized learning isn’t always easy. A more nuanced approach will be critical in understanding the full picture of this systems transformation work. Learn more about how to measure progress towards personalized learning.
