Here’s a quick recap of parts I and II…
In applying lean methodology to guide the implementation of our reading strategies initiative, we defined our problem, crafted a hypothesis, and used lessoncasts as the communication vehicle to define the minimum viable practice (the essential elements required for effectively implementing the identified teaching practice). We developed a two-week cycle to facilitate collaborative planning, reflection, iteration, and refinement.
Evaluating the impact of an educational tool, strategy or professional development initiative has always faced the steep challenge of tying adult efforts to student growth. True, it’s a quixotic errand – attempting to draw a straight line from a 2-hour afterschool workshop to improved performance on an end-of-the-year assessment. And, the truth is, a 2-hour workshop typically has no longterm benefit for improving classroom instruction. However, I propose that by using lean principles to reshape professional learning practices in schools, we can hone in and measure impact on classroom instruction. Traditional evaluation efforts try to gauge increases in student achievement over extended periods of time; with such a wide scope and myriad variables, who is to say what had an impact on student learning? In contrast, by using a minimum viable practice (MVP) and focusing professional learning efforts on collectively grasping a teaching practice, implementing it, and refining it, the connection between those efforts and the impact on classroom instruction and student learning becomes much more tangible. Since this might sound like a bucket of hogwash, I’ll explain the process we used.
As I mentioned in Part II – What to Build, our school established a two-week cycle to implement and refine instruction for each reading strategy. Week 1: watch the lessoncast, plan instruction, implement in the classroom. Week 2: discuss challenges and successes, refine, assess student learning. During this cycle, we measured impact on two fronts.
First, we developed lookfors to observe during classroom instruction. Here is where using a lessoncast as the MVP made a huge difference. The content of the lessoncast focused our collaborative planning sessions on the minimum viable practice, the essential elements for instruction. This focused conversation led to the collective development of instructional lookfors. We were able to define what teacher and student activities should be present when the strategy was being implemented effectively. We had a shared picture of what we should see. These shared lookfors laid the groundwork for two-way feedback. An observer could provide specific feedback, and the teachers could communicate if they needed more clarification or wanted additional information about a particular aspect of the strategy or a related instructional practice. These conversations led to requests for more PD related to other items like academic rigor, student engagement, higher-order thinking questions, and classroom management techniques. These requests, in turn, led to the creation of additional lessoncasts to support overall instruction. Teachers also shared how they enhanced the strategy implementation by using multi-media or technology or dramatization or improved rubrics. We were able to measure our classroom implementation and, more importantly, identify ways that we could improve.
Our second challenge was to measure student learning. The creation of each lessoncast started with a focus on what students would be able to demonstrate as a result of engaging with the instructional content. With this focus, the teachers and I worked together to create shared assessments for each grade level. We started an assessment item bank and then used the items to develop quizzes to capture a snapshot of student progress. Why create our own assessments in this day and age of purchased, off-the-shelf options? In fact, our district provided short cycle and benchmark assessments, but these measures didn’t occur on a level that was granular enough and at the frequency that we needed. For example, our typical short cycle assessment might be given after four weeks of instruction, but it would have one or two questions on a particular item that we were trying to measure. There’s not much that we could glean from a student correctly answering one out of two questions on a particular concept/skill. District-level short cycle and benchmark assessments provide predictive measures (who is more likely to score proficient on a state assessment) not prescriptive measures. We needed better information about how our students were doing on a day-to-day, week-to-week basis. Daily, teachers used informal classroom assessment (exit tickets, anecdotal notes, observations). And every two weeks or so (we allowed ourselves flexibility when need be), students completed a more formal assessment. After scoring the quizzes and examining the data, we had the students review, reflect and discuss their thinking.
We looked at each student, each class, and each grade for each biweekly snapshot and over a series of snapshots. These periodic measures helped us to see when we were heading in the right direction and when we needed to shift. This process led us to a huge realization about how our students were interacting with complex texts and exposed a major comprehension hurdle for which we didn’t properly account in our original planning of the initiative. I’ll give more details about how we determined the need to pivot in my next entry on validated learning.