Morgan Polikoff visited the show on episode 076 Standards-based Instruction & Belonging Interventions to discuss his recent publication on successes and failures of different districts moving toward standards-based instruction.
Laurence Woodruff
Hello, my name is Laurence Woodruff, And I am so glad to be done with grading this year.
Michael Ralph
And I’m Michael Ralph and I am apprehensive of my coming summer projects.
Laurence Woodruff
Professional development requires ongoing reflection and dialogue.
Michael Ralph
So join us as we spend our Saturday discussing education research and drinking beer.
Laurence Woodruff
Today, we are drinking tequila barrel aged chocolate cinnamon stout from the Empyrean Brewing Company.
Michael Ralph
Feel like you’re ending your cinnamon themed year strongly with a literal picture of cinnamon sticks on the label.
Laurence Woodruff
Oh, we got one more buddy.
Michael Ralph
Oh my gosh. One more.
Laurence Woodruff
June Yeah, June. June is a standard month. July is the freebie but we got one more we got a June one to do.
Michael Ralph
That’s alright, I like chocolate. It smells. I’m, smelling the chocolate pretty strongly.
Laurence Woodruff
Yeah. What are we doing today, Dr. Ralph?
Michael Ralph
Many schools have adopted new instructional standards in recent years. But some schools have struggled in helping teachers align their instruction. We talk with author Morgan Polikoff about his team’s work studying why flexible specificity is so important for success. Later, we look at a very large study of a belonging intervention that helps students persist in their first year of college. We consider what that could mean for teachers in both K 12 and higher education settings. Let’s get started. For our first segment, we read the good struggle of flexible specificity. Districts balancing specific guidance with autonomy to support standards based instruction.
Laurence Woodruff
This was written by Amy Sornaiuolo, Laura Desimone, and Morgan Polikoff.
Michael Ralph
This is published in American Educational Research, journal and 2023. And we are grateful to welcome one of the authors from this paper to join us for this segment. Thanks for joining us, Dr. Polikoff.
Morgan Polikoff
My pleasure. Glad to be here.
Michael Ralph
Dr. Morgan Polikoff is an associate professor of education policy at the USC Rossier School of Education. In 2021 he published Beyond Standards which critique standards based reform policy in the US and advocated for the role of high quality instructional materials. He is also a 2011 Jeopardy champion. So Morgan, you’re really active on social media. And I’ve enjoyed following you and watching some of your education threads and even engaging sometimes for the thoughtful way that you navigate nuance and some of these policy conversations. And I think when you and your co authors, were talking about this new publication that you’ve got out, what really caught my attention was what I think of as the core of the paper being this specific flexibility, flexible specificity. And that dovetails really well, that aligns with some work that actually Laurence and I have done together on a past paper. And so that got my attention immediately of I want to read this paper. And it’s a really robust qualitative study that looked at a lot of the experiences of different people across a lot of different school districts. What What was it like putting together or working on this pretty large project?
Morgan Polikoff
Well, this paper was part of a five year federally funded, center focused on college and career ready standards implementation. So this project, which was case studies of five school districts was actually part of a much larger project that included State Representative surveys that included work that I led, which was about instrument development, and content analyzing standards and curriculum materials. And then we had these deep dive case studies where we chose districts and really spent a lot of time in those districts understanding what was going on in terms of standards implementation. So it was one piece of many for us to try to gather, looking across the studies make sense of, okay, what’s happening with college and career ready standards? Are they being implemented? You know, well, and where, and why and why not?
Michael Ralph
If papers include a theoretical framework at all, I’m gonna say it’s, there’s a lot of folks like, oh, and that’s the framework and they move on. But to be able to very clearly lay out some of the components, like some of the pieces that make up the lens for your analysis. I think this might be the first paper I’ve ever read, where it actually helps me understand what you are analyzing, as opposed to like, Oh, it’s over there, whatever. I could have skipped that section and it wouldn’t have mattered it Actually, it was useful to think of what were the five components they were. Power is one of them that stands out in my mind, because it really seemed like it was important.
Morgan Polikoff
Yeah. So the end those five components, this so called policy attributes theory, which isn’t really a theory, but it’s just sort of like a way of thinking about how to make policies more effective, that actually guided the whole, larger center. So when we submitted our proposal to the Institute for Education Sciences, we wrote about the policy attributes theory. And we carried that through the qualitative work, but also through the surveys, we asked questions about each of those components. And we know there’s a number of published papers that that relate policy attributes to standards implementation, and I’ve written about that, I’ve used that policy attributes framework, even going back to my own dissertation, that was my conceptual framework as well. So it and it comes from Andy Porter, who is, was the PI, the principal investigator of the whole center. He is a retired dean of the Penn grad school of education also happened to be my PhD advisor. And so, you know, I think it’s, I think it really is, I mean, it’s almost like a heuristic for thinking about like, Okay, well, what are the components of policies that seem to matter in terms of their implementation and education? And, and I agree with you that, you know, we really worked hard to follow it through the whole paper. And you can see we, and that’s how we ended up with the term flexible specificity. You know, we, we get that really from the policy attributes framework.
Laurence Woodruff
So before we get into the weeds of this, can I take a shot at trying to, reframe it and summarize it for audience members that haven’t read it is that can I give that a shot? So there are five districts that were explored, and all of them had new standards that were being adopted by the districts, and the districts had different approaches for supporting or not, in some cases, not supporting the teachers relationship with those new standards, and you did extensive interviews, to develop an understanding of how teachers felt, and their perceptions of their ability, and their support and engagement in implementing those new standards based on different district approaches.
Morgan Polikoff
Yeah, I think I think by and large, that’s right. I mean, I think, you know, we were especially folk, you know, there are kind of like, two big policy levers that school districts have at their disposal for standards implementation, and those are curriculum materials and professional learning. And so really, what when it comes down to it, what we did was, we investigated these five districts, and we looked at how were they structuring their curriculum materials in terms of their choices and their expectations for teacher implementation? And how are they structuring their professional learning as well. And we analyze them according to this policy attributes framework to try and understand, Okay, well, which place we’re being more successful in terms of actually getting the standards implemented as intended in the classroom, and which places weren’t and why. And so, yeah, absolutely. And involve not just talking to teachers, but also talking with school district leaders and curriculum leaders. We talked with, you know, school principals, we we even observed some lessons, although I don’t think observation data made it into this paper. So, yeah, it was it was a huge undertaking, but really, fundamentally, just to understand, what were districts doing with curriculum materials and professional development to support standards implementation, and why was it working or not?
Michael Ralph
What struck me, I was really trying to understand Texas, both because I have existing relationships in Texas, but also because they seem to have a fair amount in common with the two districts that you talked about that were generally successful, and supporting the sort of standards based reform efforts. And then the three districts that were largely unsuccessful, Texas was unsuccessful, but had more stuff in common with the two successful districts. And so that was kind of watching, like, how was how was, how were the differences manifesting what you saw in the first part of that table. And it looks like differences in consistency and stability were really important, which I understood was like a lot of what you analyze coming out of that was, can you just can you tell us a little bit about what consistency and stability mean?
Morgan Polikoff
Sure, yeah, yeah, absolutely. So you know, consistency, I think is, you know, you might just think, you know, it has a lot of sort of synonymous terms that we’re all familiar with. So consistency is kind of like a line mentor, it’s kind of like coherence, right? It’s basically, to what extent are all of the policy things happening in the district pushing in the same direction? Are they sending the same message to teachers about what it is that they’re supposed to be doing? And you know, and anytime you talk to teachers, of course, about like, their professional learning or standards implementation, they’ll readily tell you, right, like, oh, you know, we got this PD, it was a one off, it was totally unrelated to what we were, you know, what we’re told we’re supposed to be doing right. So so that’s what consistency means everyone has sort of a clear understanding of that. And stability is really just this idea that, you know, you need time to, to implement really any reform. And I think one of the things that we all again, if you talk to teachers, or if you study education for a while, you know, that there’s kind of this like, turn of reform, right? Because the policies aren’t very stable. It’s like you put in something, you try it for a couple of years, you see it, as I’m working, you have some new district person, you know, some new superintendent comes in, they have a different priority, or there’s some new fad of the day. Right. And that, also, I think, undermines, you know, a sort of consistent implementation and long term vision when you’ve got this constant churn. So So those are two of the of the main attributes. And yeah, and were a focus of our analysis. And I think in the case of the Texas district, I would say, actually, that to some extent, I mean, it was about differences in, you know, consistency and stability. But I think it was, you know, there…, I think it right in the Texas district, there was this sense, there wasn’t as clear a sense, when we talked to teachers about the coherence of the vision and the coherence of this, the district’s different approaches to supporting standards implementation, where in California, in particular, because that’s the place where I spent the most time, I’ve actually never been in a place where there was as unified a voice about what the goal was, and what the expectations were for teachers about what they should be doing. And that was just because of that consistency, right? You’re hearing the same message over and over again, you really start to believe it and buy into it.
Michael Ralph
Laurence, I, I couldn’t help but think of the paper just from last month that we read, looking at Nationwide teacher evaluation reforms. And like that being one of the primary critiques was we changed things last year, why aren’t things different now? And like, well, it takes a while to show you’ve got the commitment to the changes and to help understanding like be cultivated and disseminated across the the organizations. And so I appreciated seeing in this in your paper that you are attending to the importance and the role of that consistency over time, the stability over time, because that has been a major flaw in teacher evaluation reforms, which we just happened it like just sheer coincidence we were talking about last month.
Morgan Polikoff
Yeah, I mean, pick a reform. Right. And that’s the kind of the story. And so Eva, and I think even within form within reforms that have been quite stable, like I would say, standards based reform actually has remarkable stability as as kind of a high level policy instrument, we’ve been doing standards based reform for like 30 years. But at a sort of micro level, we get, you know, waves of standards, where now all of a sudden, it’s you know, it’s Oh, the the standards say that it should be more student directed. And before it’s that something different or now we’re supposed to do more procedures. And before we’re supposed to do more conceptual understanding, or, you know, we’re doing standards now, you know, we’re doing standards, but we don’t actually have we don’t get aligned curriculum materials until five years in to, to the effort. Right. And so, absolutely, I think that’s a trend and teacher evaluation, I think is another prime example.
Laurence Woodruff
So when I’m thinking about this paper, your school districts you divided them into into three different kinds of categories, the rural districts, the urban districts in the suburban districts. And just to simplify things, the suburban districts had teachers who had positive perspectives of that their relationship to implementing the standards and the rural and urban districts did not communicate that same optimism or positivity. And so for like, maybe framework of this discussion, since we we kind of like to talk about this podcast as getting to the shoulds of the paper. So maybe, let’s discuss what went wrong. Let’s talk about maybe some of the perceptions and the approaches of the districts that did not seem to work. And then we can like, discuss, okay, now we’ve got that, because I think a lot of I think a lot of teachers will be able to identify with some of those frustrations. I mean, teaching is a high turnover profession, because there’s a lot of dissatisfaction in it. And so if we can acknowledge the states that are dissatisfying, we can then say, Okay, now let’s discuss the good stuff. So let’s talk, let’s, let’s talk about one of those districts, you can choose or we can start where are we gonna make a proposal, but let’s talk about one of those districts where things weren’t working out. And let’s talk about why they weren’t working out.
Morgan Polikoff
I think this paper ultimately, like a, frankly, a lot of education research is a little bit of like, sort of common sense is, like, I could have written the paper with common sense before I actually analyzed all these heaps of data. So I think that, you know, if you were to start with the rural district, which is located in Massachusetts, you know, this is a district that at its core, you know, is fully bought into the idea of, like, local control is everything. Right. And so, you, so you could talk, you would talk to district leaders, and the it was almost like the district leaders didn’t believe that there was any role for the district to play in shaping, teaching and learning in the district, which I think, you know, sounds like a bit of a wacky idea, but actually, I think there are lots of places all around the country where both states and districts, leaders really believe that right that, like, the district’s job is not to tell teachers what to do or not to, you know, have, you know, have expectations for them? And so that, you know, and that was how it was experienced, right. So, so, curriculum materials, it was like a local, like, do whatever you want. It was like, totally laissez-faire, which, you know, I think on the one, you know, people say, Oh, well, we don’t want to be telling teachers what to do, or, you know, we don’t want to, you know, we want to empower teachers, but not giving anyone, any support is not empowering, right. And so what we see in, you know, what we saw on that district was, well, of course, you wouldn’t get coherent standards implementation when you’re not giving anyone, you know, clear materials or expectations for how to use them or support that’s aligned with the standards. Right. So that was kind of the one extreme, which I think is, you know, you might think of these districts as emblematic of various kinds of districts around the country, right. And then the Pennsylvania District, which was an urban district, was kind of, I think, had actually vacillated between different reforms that as well as a lot of, I think, large urban districts, in particular are, are seen as doing that, right. So it had been a place that was like, the other extreme for Massachusetts. And so really tightly scripted, we expect you to be on this day saying this thing at this time. And then there was a backlash to that, because, of course, there would be. And then they went to the opposite extreme, which was, you know, really, again, totally unstructured. And in that case, it was experienced, because it’s a, it’s a, you know, a large urban district. In some, in some schools, the principal was still all about scripted, and you’ve got to be on this thing. And then in other schools, there was, again, no guidance at all. So it was very disorienting. And teachers would, you know, we’re almost like living in like a culture of fear around what is it that I’m supposed to be doing? So that might be emblematic of a different kind of district. Right. One that. I mean, you know, started with real strong scripting, which is, which is one way of approaching curriculum. You know, of course, if you want curriculum to be implemented, you could just literally force people to do it, which is what that district had been doing. And then as a reaction to that moved, again, to sort of local control that every school do what it wants, which was experiences this wide degree of variation.
Michael Ralph
Laurence and I with a couple of other co authors, that’s one of the papers that we wrote that like, I think you could cite it like, I think it is highly relevant to this to this kind of work, where we were talking about collaborative autonomy, which is specifically about navigating that tension of empowering professionals to be professionals while also pursuing a shared vision for what you’re doing in the building and in the district and that those things are not necessarily at odds with one another. But I don’t see very many places that are holding two truths in their hands simultaneously. And I think that that’s a big piece of what’s coming out of the narrative of these districts that are struggling is they see it as either 100% scripted curriculum which we’ve got the democratic critique of scripted curriculum, that’s another past episode, like we know that that’s, that’s something that we are, like philosophically opposed to. But then also you have you have to support teachers, like, do whatever you want, is it remarkably dis-empowering.
Laurence Woodruff
In the paper that we published, One of the sort of the questions from leadership that that was, we kind of propose is what the question that leadership should be asking is, how would you like to improve? And how can I help? And when we start looking at these districts, and we start looking at the approaches of these different districts, the rural district was granted freedom, we respect your professionality, do whatever you want to do this, but they didn’t ask, How can I help? They didn’t ask that, how can I support you in in exercising your professional autonomy? They didn’t offer any and so there wasn’t perceived to be any support. And so there was that frustration. Whereas when we come back and have some discussions about some of the districts that had more positive perspectives from their teachers, you we will will, we saw that there was response, there was that question was asked questions were asked, and then they were responded to, in a manner that respected the professional professionality of the teacher. So you give freedom with no support, you’re still just encouraging frustration in your teachers.
Morgan Polikoff
What I would say is that the clearest difference for me, and, you know, across across these districts, but I think that the Texas example is one in particular, is the consistency of the messaging about what good teaching and learning look like, and the organization of absolutely everything in the school day around that. And, you know, as I said, I spent the vast majority of I lead the vast majority of the California data collection. And this was a district where everyone was singing the same tune from the superintendent, all the way down to individual classroom teachers. I mean, there was, there was literally I and I, and I remember him, because he was so notable, there was one teacher who wasn’t buying it. And this was like, one of these guys who was in his last year does an extra last year, and he was like, I’m not doing this, I’m just closing the door and doing what I want. But everyone else was really bought into, they just believed in the curriculum materials. They they understood that they that there were clear expectations for implementation, but that they were also allowed to supplement and encourage to supplement as needed to do it collaboratively with other teachers in their school. So that kids will be getting the same experience, you know, no matter who they were, who they were in a class with. And so that, you know, frankly, that the teacher wasn’t having to, you know, go on Pinterest the night before and find some, you know, activity of dubious quality. There, the whole school week was structured around supporting data analysis and curriculum implementation, right, with PLC time on Fridays, and with, you know, this district had a decent number of English learners. So they, you know, weren’t getting the outcomes that they wanted for their ELs. And so they added an EL block at the start of the day for everyone across the whole district. And actually, one thing one, you know, sometimes you could hear teachers grousing about, well, I only have two wheels, and I don’t really feel like this is a great use of my, you know, precious instructional time. But this was the vision, right? Because the whole district really believed in that, that this all should be, you know, one machine operating together. So it’s about, yeah, it’s about building the structures that are aligned with the vision, it’s about, you know, you know, I mean, all the same things we’ve already said, right, it’s about having clear expectations, and high quality tools, like curriculum materials, but also high quality assessments that are common across classrooms. But also, you know, empowering people to make decisions. And, and it really is, you know, one of the other things we talked about, in the paper, one of the other sort of leverage points is around authority, which is basically just buy in, there really is about all of these things together are, you know, they’re working together to cultivate buy in, right, and teachers are bought in because they’re seeing success. They’re feeling like they’re, you know, they have good relationships with their peers, and they’re and they’re implementing together successfully. So, you know, this wasn’t a study focused on primarily like Data Analysis. So I didn’t spend time observing data meeting. So I can’t say what it is about their use of data that was different from some other place. I mean, I do know that data, you know, like data driven decision making can also be a kind of, you know, quasi scripted performative activity, right, that isn’t really about, you know, fundamentally understanding what students need and what you need to do different. And I think that’s often how it’s implemented in places that are sort of dysfunctional or not particularly coherent. Right. It’s just another thing to do. And it’s not part of some some vision for how do you actually move instruction forward.
Laurence Woodruff
So one of the things that separated the suburban schools from the urban schools, and we kind of touched on this earlier was the willingness to gather feedback and then respond and make changes to the communication of the of the teachers involved. And I think I think it was at Ohio, Ohio suburban school, they’re the one where they said, Okay, for the first year, we really are mandating that you do everything by the book, very scripted, am I am I right about that? Or was that Ohio?
Morgan Polikoff
Yeah, I mean, both. I mean, Ohio was a little bit more extreme on that. But even in California, that was pretty much the expectation, it was like in year one, we really want to try this thing is written for the purposes then of in subsequent years figuring out what didn’t work and and doing better.
Laurence Woodruff
And that was very, very similar to what was going on in Texas. But what made Ohio different from Texas is that in year two, they said, Okay, now that we know what the landscape is, and we see what the execution is, we’re now going to give you the opportunity, we’re going to get feedback from you, we’re going to make some changes, we’re gonna let you make changes. Whereas in Texas, it just seemed like we don’t want to hear from you. We don’t need to hear from you. And responsive teaching is we say that phrase, almost every episode that we want to make sure that you are, are getting feedback from your students getting feedback from the ecology of your school, and you’re responding to your students are responding to your school with the appropriate interventions that they need. And the districts that were successful, were at the next tier up doing that with their educators and the districts that we’re not, we’re just not doing that with our educators. And if I were gonna boil this down to one thing, that would be a responsive leadership to the needs of their their educators.
Michael Ralph
And they did too, because that’s the namesake of their paper is the the good struggle, which I expect it to be a classroom thing when I cued up this paper, but it was actually a description of that process of the give and take.
Morgan Polikoff
Yeah, no, I think that’s exactly right. But the only caveat that I would add was, yes, all of that. But in the context of a district that has provided teachers with materials that they think are good, and that they have that they expect them to use, that provides them with other supports, like, you know, reasonably good interim assessments or high quality professional learning with time during the school day. So yes, it is about that responsive leadership that you just said, but in the context of providing those other supports. I feel like it’s a little bit common sense. And I feel like if you talk to teachers, you know, if you spend any time in schools at all, you might, you might have come up with a similar, a similar conclusion. But, you know, everyone knows that any particular curriculum material isn’t perfect, right. That’s, we all believe that. Every, you know, but and but I would say that the vast majority of teachers that you talk to want to have something to organize their instruction around. There are certainly exceptions to that. But high quality core materials. I think most teachers want that.
Laurence Woodruff
I hate to do this, because I love your paper. You keep using this phrase, common sense is the approach of the suburban districts common?
Morgan Polikoff
Well,
Laurence Woodruff
I mean, I, yeah, I don’t think you should use that term. I mean, I’m sorry. But I like and I, I understand it emotionally, I understand where you’re coming from, like, when you see everything, it should be straightforward, and it should be laid out and the the Ifs-then all line up like this should be. It feels obvious to maybe you and me, but I don’t think it’s common sense.
Morgan Polikoff
Yes, well, maybe I shouldn’t say it should be common sense. Right. And no, but I think your point is very well taken. And yeah, I don’t think it’s well, anyway, I’ve already
Laurence Woodruff
Yeah, sorry.
Morgan Polikoff
I say I don’t think it’s rocket science. Right. But then that’s also like, well, but it maybe you know, maybe it’s more hard than rocket science right?
Laurence Woodruff
Exactly, yeah.
Michael Ralph
So there’s a linear flow of logic. That is my Yeah,
Morgan Polikoff
I get your point. Right. Yeah.
Laurence Woodruff
So I don’t know. I mean, I would be less frustrated if it were that common. And we were just got some, like rare places…
Morgan Polikoff
It’s sensible. Take out the word Common.
Laurence Woodruff
Yes. It’s sensible.
Michael Ralph
Well, thank you. Once again, this has been a terrific conversation. If our listeners have enjoyed hearing this conversation and want to read more of your work, or learn more from the work that you do, where can they find your materials?
Morgan Polikoff
Absolutely, it’s been a real pleasure. It’s a fun paper to talk about. And I’m, you know, I like the back and forth. I love your point about common sense. I’m going to be thinking about that for a while. So I have a book already out there. It’s called Beyond Standards, which you can pick up wherever, on Amazon or whatever. And I’m working on a second book. And I think that this common sense point I want to I’m gonna stick it in my head because my my, my second book is all about bad ideas in education policy. So each chapter is a bad idea that I sort of refute with evidence, one could be on early high school start times, which is absolutely a terrible idea. Except that I need to write a whole chapter on that because it’s so obvious that you could refute it in 500 words so anyway, but But lots of things that we know are bad are still implemented in schools. And so I think that that’s really the point of the second book and and I’m excited to read it I’m working on it now. And yeah, it’s been a pleasure.