Heather C. Hill, Harvard Graduate School of Education
Heather C. Hill is a professor at the Harvard Graduate School of Education. Her primary work focuses on developing new measures of mathematics teaching quality and using these measures to inform current policies and instructional improvement efforts.
Heather spoke to Emily Freitag about her research on effective professional development programs for teachers.
Watch the full conversation or read the abridged Q&A below.
EF: Please share a story from your own journey as a learner and what that has taught you about learning.
HH: I’ll tell you the story of a non-academic learning experience that I had, which is learning to play Ultimate Frisbee. Two things you should know about me is that I’m not super coordinated, and I have never played a team sport playing before playing Ultimate. I picked up Ultimate sometime during my adulthood and was never any good. Still, I started playing consistent pickup with a bunch of folks that I found through the internet. The first interesting thing is that the Ultimate community is terrific because they were so patient with me as a learner. The fact that I would drop the disk or make a terrible throw 80% of the time, didn’t deter them from involving me in the game. That’s a unique feature of Ultimate, but it’s also a way to think about learning environments that can help people grow and thrive. The second thing is that they persisted at this for a very long period of time. I’ve been playing with them for almost 10 years, and I would say it took until year seven or eight for me to actually become mediocre at playing Ultimate. Over a very long period, I was able to learn to play the game at a level where I’m comfortable and I’m not the worst player out there anymore. The third thing that’s interesting about this is there’s one particular person who taught me to play without ever speaking to me. This guy is a little shy, but he’s a pretty amazing Ultimate player. In many cases, if you’re that good at playing ultimate, you take it upon yourself to be responsible for training other people and teaching other people to play. So, he taught me to play not by saying anything to me, not by coaching or giving tips, but by simply throwing me the disk where he wanted me to go, or teaching me how to short kinds of passes and cuts, by simply reinforcing the good behaviors that I was already doing and trying to improve them. It was really interesting to watch that process and think about learning as a nonverbal activity between two people.
EF: You’ve spent a lot of time thinking about professional learning and how we collectively support teachers in their learning. As schools are considering dramatic changes, it is such an important time to be anchored to what we know to be true about what works in professional learning. Please share with us the best practices that we should be anchoring to.
HH: I’ll start by telling you about a study that I did with Katie Lynch and a couple of other colleagues at Harvard. This study is a meta-analysis of teacher professional development and curriculum programs. We found 95 studies that looked at STEM teacher professional development where teachers were learning new practices, or where they were handed new curriculum materials, or where the programs were a combination of professional development aligned to curriculum materials. One of the things that we found was that when you combined professional development with curriculum materials, outcomes from those studies were more positive than outcomes for studies that had only professional development or only new curriculum materials. Oftentimes, teachers will come away from professional developments and say things like, “Well, that was a lot of ideas, but I need something I can take back into the classroom.” I think teachers are indicating that they need something to help them carry out the ideas of professional development. Teachers may be enthusiastic about new ideas for learning, they may be enthusiastic about new instructional practices, but unless they have something concrete to support them every day in the classroom, it’s much harder to implement those things.
There were several other findings, one of which was that when the professional development focused on pedagogical content knowledge and content knowledge, those programs boosted student outcomes beyond typical professional development programs. The same was true for programs that had summer workshops, which was a surprise to everyone because the interest has been in distributing learning across the year; however we found positive impacts for these summer workshops above typical impacts. We also found above-typical positive impacts when teachers had a chance to come back together after beginning implementation of a new program to the facilitator and each other, and say, “What’s going well? What’s not going well? Can we troubleshoot this issue?” Often all it really took was a three-hour meeting where teachers could get back together with the program leader or a coach and discuss how the program was going. It’s really interesting to think about these components for building a better professional development program. We don’t have a lot of fine-grain information about what those ingredients would look like, but we know some of the structures now that can really help.
EF: Can we dig into each of these components? Let’s describe what pedagogical content knowledge is.
HH: Pedagogical content knowledge is knowledge that is unique to teaching. One example I like to use is if you’re simplifying a fraction like two-fourths, there’s a procedure for doing that—you divide two by two and you divide four by two. Oftentimes, teachers get asked why that works. The answer is you’re dividing by two over two, which is one. So, you’re not changing the value of that fraction, you’re just re-expressing it in a different format. This idea that you can re-express quantities turns out to be a really important idea in upper-level mathematics and in statistics. That is teaching knowledge, because it’s going to come up in the classroom and few other places, and because it’s going to sort of seed knowledge up the curriculum. If kids have that kind of meta-knowledge, they will be able to thrive with other content.
EF: So the content knowledge would be that I can do it, but the pedagogical content knowledge is that I know why it works and I know how to teach it.
HH: Exactly. It also encompasses knowledge of kids’ errors and mistakes as they’re learning content, and how to represent content to kids in friendly ways.
EF: I feel like there have been conflicting messages around professional development on content knowledge. Tell us a little bit more about that.
HH: The STEM study actually has a little wrinkle in our findings. The original study used student outcomes as the dependent variable. Then, we ran a model that looked at teacher knowledge and teacher practice as mediating variables, asking whether the effects on student outcomes ran through changes in teacher knowledge, teacher practice, or both. We found that the effects on student outcomes correlated more with the changes in teacher practice than the changes in teacher knowledge. There were programs in our dataset where teachers did improve their knowledge. Those programs didn’t see as high of gains in student outcomes as the programs that also included a practice component. I think one of the issues—and this goes back again to what teachers tell us over and over—is that they’re learning content that doesn’t directly apply, and they can’t see how to use it on a day-to-day basis in their classrooms. If you’ve learned this new idea about what’s really going on when you’re simplifying fractions but your curriculum materials don’t cue that, you’re not going to be able to implement that in your class. I think that’s probably what’s going on.
EF: And the curriculum alone doesn’t seem to then lead to change in practice?
HH: That seems roughly right. The teachers need a little bit of the content push to make good use of their curriculum materials.
EF: What else do we know about professional learning?
HH: Let’s go back to the STEM study that we did. When we wrote the study proposal, we said, “We’re going to code for the district surround.” That this professional development or curriculum program is in a particular context. We know that when programs go into contexts, those contexts reshape them. We thought we would be able to collect from these 95 studies, something about the leadership support for the program and something about the alignment with other sets of curriculum materials or other instructional guidance offered to teachers. But, the studies had none of that information. It didn’t allow us to really investigate the extent to which the district surrounds make a difference for promoting better student outcomes.
That being said, I think there’s strong evidence from other single studies that the district surround makes a huge difference in the efficacy of new professional development and curriculum materials. We’ve talked to teachers who’ve gone through intensive professional development, and they say they loved the professional development, and we say, “But we didn’t see that you doing anything differently in your classroom, so can you tell us what that’s what that’s all about?” The teachers would say back to us, “Well, my principal has a pacing guide and my principal walks by my classroom at least once or twice a week, and I feel like I can’t do the things that I learned about in the professional development because I need to be on the pacing guide, which means I need to be moving along at a pretty fast clip.” Another teacher would say, “Well, I loved professional development and I did it for a year, but then the district had an RTI-type thing where we were required to do remedial instruction for the first 20 minutes of our math class. And that meant that I couldn’t enact the professional development ideas in the way that I wanted.” A huge part of the success of any professional development or curriculum program is the alignment and the coherence into which they are occurring. So you can’t have a program that is at cross purposes with your teacher evaluation system, with your pacing guide, with your assessment, or with other sources.
EF: Or with other PD, right?
HH: That’s exactly right. This is the number one problem facing improvement efforts because it takes a long time for teachers to develop expertise in something. Just like my example of Ultimate Frisbee, it really took me seven or eight years to get good at the game. It’s probably going to take a teacher two or three years to be an expert in teaching a particular set of curriculum materials, but often the cycles change much faster than that because a new superintendent comes in or somebody gets replaced in the district, and suddenly there’s a new idea about what to do. There are huge benefits to staying the course.
EF: That makes me think of our conversation with Candice Bocala about the Internal Coherence Framework. And in our Curriculum Support Guide research, we also found that one of the biggest pitfalls of systems that didn’t see the success of curriculum implementation was that teachers were getting mixed messages—whether it was the formative assessment system, or leader feedback, or pacing guides, it was all creating a lot of confusion.
So, what doesn’t work? What do we know we shouldn’t do?
HH: This is now moving outside of the STEM meta-analysis that we did. I’ve been reading reports on teachers’ study of data for a long time. This is a really common activity in schools. Many schools have grade-level teams, and what the grade-level teams spend a lot of their time on is studying student assessment data, often interim assessments, assessments that teachers design or they’re collecting data from kids in other ways. I located 11 or 12 studies of these kinds of programs where some kind of data was brought in and there’s a process for teachers to study it and think about what it means for practice. The results there were just flat across the board. So I think of 19 impact estimates, there was one positive and one negative and the rest were zero. I think across that many impact estimates in that many programs, what you’re looking at is probably a true zero for that particular intervention. I’ve seen some of these and I’ve had doctoral students write dissertations on these types of efforts. I think what often happens when I’ve watched these is that a focus on the students and the data doesn’t turn into a focus on instruction, and it doesn’t go back to, “Let’s look at our curriculum materials and think about how we’re teaching, and maybe do a rehearsal or think about how we could be doing that better.” Often, the conversations end up moving toward, ” Johnny scored really poorly on this assessment, but he was having some problems that week and it was really just an aberration.” Or a teacher saying, “I’m sorry to hear Johnny didn’t do well in the division problems. There’s this Tarzan division worksheet on Pinterest, you should go find it.” This further makes the content incoherent for kids.
EF: So, it doesn’t lead to a fundamental rethinking of the content or about developing teachers’ expertise.
HH: If you think about ways to develop expertise in teaching, it is through the practice of teaching, it’s through reflecting on teaching, it’s through developing better decision-making capabilities. And if you think about the study of student assessment data, it doesn’t support many of those things as it’s currently constituted.
I published this right before COVID hit. I received a lot of emails about it, sometimes from people who are really distressed to think that something that we’re doing isn’t effective. But, as important as it is to identify what works, it’s also important to identify what doesn’t work and to be able to say we need to do less of it, or to do a better version of it, or to move to something that we know is much more effective.
EF: Do you have hypotheses on the root of the distress?
HH: The folks I was hearing from were teachers and principals, and I think they’re so genuinely concerned about improving for students, and they’re driven to do it. Being told that you have invested something in something heavily that doesn’t work is just heartbreaking.
EF: I’m thinking about our conversation with Elaine Allensworth. I do think the data systems and processes she was describing around their Freshmen On Track work are actually quite different. Can you explain that difference?
HH: I have no problem with administrators studying data. I think that’s a terrific idea. Freshmen On Track is a wonderful system that is so necessary right now in which they’re tracking kids and making sure that kids are staying on track, logging on and completing assignments, so that we can catch folks who are not, for whatever reason, able to do those kinds of things, and start to figure out what’s going on with them. That’s a great idea. Same with principals or district folks that want to study broad assessment data and say, “Where are the problem spots? Is it fractions?” And then start to think about ways to target professional development around that content. The piece was about having teachers study student data, and that’s where I think things break down.
EF: Is there anything else you see people try that we actually know doesn’t work?
HH: This is just a hypothesis on my part—I think the professional development that is more abstract in nature that tries to change minds and beliefs needs to be accompanied with something that is much more concrete, like new instructional practices. I think particularly in the equity space, there are a lot of bets that if you can make teachers more sensitive to racism and aware of their own privilege, that’s going to change behaviors. But I think it’s the same kind of issue that you’re looking at in the math PD, where teachers say, “Well, if you just give me some math ideas, and sort of teach me about math learning, that doesn’t help me very much, I need much more concrete instructional strategies.” I think we are making progress towards some of those more concrete strategies in the equity world, which has been terrific to watch. I know for math, we may fairly soon be able to say, “Here are some really tightly controlled things that teachers can do to really engage kids in a different way in classrooms, and particularly kids who have not profited in the current system of math instruction in the US.”
EF: I think we do still see a lot of general pedagogy professional development, do you feel like we know enough to say that’s not the best use of time?
HH: That is like a huge bucket, so it’s too hard to say. I think about MyTeachingPartner at UVA, which has been shown over and over again to be highly effective in engaging kids more and improving their learning. I would count that as general pedagogy professional development, because it’s not content-specific. They have a nice system of ways to engage adolescents, which is the age group where kids get turned off from schooling, and it’s a way to build rapport and trust and relationships between teachers and students. So, I would say, go for it, especially around the engagement piece and the idea that you want classrooms to be more humane places for kids. There’s some good evidence that those kinds of things work.
The other place that I’m a little bit more skeptical about is the SEL programs. Not that they’re not effective—many of them have been shown to be effective, but I think their effects are overblown. When I took a look recently, I actually went through all the studies that were in these meta-analyses of SEL programs. At the end of the day, I was not convinced that you end up with the huge bump that people claim once you throw out the studies with bad designs and you throw out the studies that I would not count as SEL studies. I think the bump is much smaller, much more in line with professional development in mathematics and other subjects.
EF: I feel like these interviews just keep leading us back to this inescapable truth that it’s a lot of different things adding up coherently.
HH: If the teacher has beautiful math pedagogy and is using great curriculum materials, but the kids aren’t going to engage, they’re not going to learn a thing. That is the number one reality. And that’s why the engagement of kids is pretty high on my list, even though I don’t study it. Professional development that addresses that can be super beneficial for kids and teachers. I mean, it’s just better as a teacher if your kids are into it.
EF: I’m thinking of the general, “hot skills” professional development. Those seem like they are less effective if they don’t get content-specific and practice-specific.
HH: That would be my guess. I would worry about any kind of professional development where a teacher can’t take it back into the classroom in the form of a specific routine or a practice that they can use consistently with kids or a set of curriculum materials that supports it.
