Is there a secret weapon for improvement? Yes! John and Andrew discuss how students fit into improvement projects - and how that translates to businesses. TRANSCRIPT 0:00:02.0 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. The topic for today is "Engaging Students is the Secret Weapon for Improvement". John, take it away. 0:00:28.0 John Dues: Andrew, it's great to be back with you on the podcast. Yeah, this is sort of a revelation to me when I was working with... It's actually working with David Langford, and we were talking about, "How do you bring about improvement in schools," and at one point, he said to me, to kinda give it away at the top of the program here, "Students are sitting right in front of you, and they are the secret weapon when it comes to school improvement." Engaging them in those improvement processes is really the secret to improvement, because almost everything we want to improve in schools has to do with students, but we almost never directly engage them in this improvement process. It was so obvious they're sitting right there in front of me, but it wasn't until David said it that way that I said, "Oh, my gosh, all this time." Of course, as a classroom teacher or a principal, students were sometimes tangentially involved in improvement efforts, but how many times are they central to it, how many times do we put the data that we want to improve right in front of the students and elicit ideas for improvement as we watch that data move up and down over time? So it was a real sort of eye-opener for me to start thinking in that way. 0:01:50.8 AS: It's funny 'cause when I first started teaching many years ago, teaching finance, I was always worried that I would get a question that I couldn't answer. And what I came to learn from that was that a question that I couldn't answer is a great opportunity for a discussion. And then I would basically say, "Hmm, well, what do you think is the answer?" Now, in a way, I was playing a little bit of a trick 'cause I was deflecting the fact that I didn't have an answer. But I said, "What do you think? Okay, what do you think?" And then we started to construct and answer to that as best we could. And it took a lot of pressure off me because I realized that that discussion was a fine discussion to be had in the classroom around a topic that I wasn't exactly sure how to answer. 0:02:37.9 JD: Yeah, I think all of us try to hide our weaknesses, especially early on. We gain experience, it gets more comfortable to say, "I don't know," which is a fine thing to do as a experienced classroom teacher as well. And I'm thinking about in this context involving students, probably the best ideas for improvement are living right there with them, just like even if you didn't know the answer in your early classrooms, that sort of elicited a discussion that maybe was richer than it would have been otherwise. So I think, yeah in either case, involving students is a real sort of key to this improvement process, whether it's a single teacher in front of a classroom of college students in your case, or in my case, where we're trying to improve our system of schools. 0:03:30.7 AS: In my Valuation Master Class Boot Camp, which is like an online course, I have so much more flexibility than you have in high school. But I found one of the students was just really engaging and really supportive of the other students, so I hired him. And I said, "Why don't you become student experience? That's... Your job is about bringing that great student experience." And then whenever I kick off the Valuation Master Class Bootcamp, I ask prior graduates to come and speak and tell the students, give them some advice, and tell about the transformation that they went through in that course. So on the first day of class, they're inspired and encouraged, and then throughout the class, they've got a prior student guiding them and helping them get through where he knows are the most difficult parts. But you don't have that kind of flexibility, I would guess in your setting. Tell us more about that. 0:04:30.2 JD: Yeah, I think well, one I think that example that you just told is outstanding, and I actually think... I think it's a little bit of a misnomer. There are a lot of regulations, there are a lot of handcuffs on... To certain things in terms of what we can do and what we can't do. But actually we have fairly wide latitude. We're a small public charter school network, so we maybe even have more latitude than the typical traditional public school. We have the latitude of a district, so we're making decisions for district of schools, basically. And we're small, we're pretty nimble. We think innovation is pretty important. We think continual learning is important, and we put some processes in place to elicit that. Where there can be some roadblocks here and there, I think one of my jobs is actually find a way around those roadblocks, if they're in service of our mission and in service of helping students be educated at a higher level. 0:05:29.4 AS: So what I do is I ask the students at the end of the whole course, I say, "Tell me what you learned. What is the number-one thing that you took away," that type of thing. And I'm putting them in a pretty intense situation for six intense weeks, but then they've got a record of that, they've thought through that. And then when they come back, then they can share, "Here's what I went through, and here's my advice on how to get through it." And it is an idea in a school to say, having a list of the people who made it through the class on the wall. 0:06:03.4 AS: And then another idea is to find one or two students that would say... Come back and talk to the students to say, "Okay, this class is about American history, and the one thing that just lit me on fire is the story of Philip Sheridan when he was attacking... The US cavalry, was attacking the southern cavalry, and how he knocked out Jeb Stuart, and it just got me reading all this stuff from blah, blah, blah, blah, and then... " So, of course, that's big brainstorming, but that's an idea. 0:06:33.5 JD: Yeah, I think that... You said a six-week course. What you're describing is essentially that Plan-Do-Study-Act cycle, so I could see a scenario where in your six-week course where you run a PDSA on how are we gonna improve the class. And at the end, when you get to that act, you sort of decide with the class... What should I focus on for the next PDSA with the next class. And so in that way, you'd sort of be... Assuming you're re-teaching this class on an ongoing basis, you'd be sort of continually improving, and that's really the sort of... We talked about the Plan-Do-Study-Act cycle, the PDSA cycle, the last couple sessions, that's really what it is. It's where you leave off at the end of that cycle and you decide what you're gonna do next, feeds into that second cycle of improvement. So whether you called it that or not, it sounds like you're basically running PDSAs with the finance classes that you're teaching. 0:07:34.6 AS: Yeah. In fact, at the end of the class, I ask them another question, which is, what could we do to improve? And... [chuckle] 0:07:43.3 JD: It's perfect. 0:07:45.5 AS: So the question... The problem that I faced was that the students said I want more one-on-one feedback, that they submit their assignment and they just get pass/fail or a grade and they don't get the feedback that they wanted. And really, I have to say, I was kind of upset about this reply because I felt like, "I can't do it, it's too many students," and my goal is to grow it so that I've got 100 or 500 students. How am I gonna scale it if it's about personal feedback? So we talked about it a lot for the next Boot Camp that came up, because we had seen this complaint coming up, and we came up with this idea. And I said, "Maybe... " And this... Part of this is talking to people like you and David Langford and others, maybe we need to do more work on clarifying the assignment. And so we went back and I said, "Look, every week we need to make it super clear on Monday what's the assignment for the week." 0:08:48.3 AS: And we even provided them kind of a score card of the way we're gonna look at it. "Did you do this? If you did that, you get a point. Did you do that? Did you check your grammar," whatever. And so we got much more clear, and then what we decided to do was to say, "Look, the teams will meet in the week, they always meet once a week, and they need to pick one or two people to present that on Friday." And then what we had is, we had the students present their work, just the best of the best, and I would say not the best of the best, but the ones that shows... Said, "I'm ready and I can do that," and then myself and my team gave them feedback after they presented, and said, "Okay, see that? Try to fix that. Make sure that you don't... " And then once we did that, what I then did is I took notes throughout those and recorded those, and then I improved again the description of the assignments and the common mistakes that people made. And so the next time that we did it, the next launch of the Valuation Master Class Bootcamp, we now had an even more clear focus on what you've got to do by the end of this week. 0:10:00.1 AS: And then finally, what I did is I called it Feedback Friday. And I said, "A whole week, we're working on a bunch of stuff but the end result is on Feedback Friday. One person, two people from your team is gonna present and you're gonna get critiqued and see how you do, and everybody's gonna watch that, it's gonna be recorded. Anybody can go through that." So we've been doing Feedback Friday now for three bootcamps and I would say all of the complaints related to feedback and not enough personal feedback are gone. And it wasn't through personal feedback that we resolved the issue of not getting enough personal feedback. 0:10:35.0 JD: Yeah. Well, that's a PDSA cycle for sure. Another thing I think of is what you... Sounds like you did over time as you iterated this class, and how you gave feedback, you actually found the actual root cause that was causing the problem. And the third thing I was thinking of, 'cause you talked about complaints, one thing we can do is overreact to complaints. So that's another thing that you could do is put the complaints on a process behavior chart, and if you get to a certain number, that might sort of signal that you have an issue. Otherwise, there may be an acceptable level... Number of complaints and... Or a third level analysis is, it's a stable number of complaints but the number is not acceptable to you as the instructor and so you wanna go about improving the whole system. It sounds like that's exactly what you did, sort of what we've talked about the last couple sessions, is you chart something, whether it's quantitative or qualitative data, you're keeping track of that, and then you're tagging it to this structured improvement process. And, yeah, it sounds like you're running the PDSA cycles for the class. It's pretty cool. 0:11:57.1 AS: That's a comforting message for the listeners and the viewers because what it tells you is that you don't have to be super official and have all of the tools that we learn from Dr. Deming's teachings that... First, is to start with the thought process. And my first thought process is, "I want my Valuation Master Class Bootcamp to be the best course in the world." That's all I want, just the best in the world, so I'm constantly wanting to improve. The second thing, I do not ever focus on competitors because my course is just so different, and all I focus on is the students. The third thing is I'm getting feedback on a consistent basis from the student about what they like and what it's worth to them. Because I also ask them, "Now that you understand exactly what's in the Valuation Master Class Bootcamp, what is the price that you think I should charge for this?" And my goal is that that price continues to rise as the perception of the value of the course rises. So I'm getting feedback, and then I'm looking at that feedback and I'm trying to identify what I think is the most important feedback that we've got to somehow resolve. 0:13:13.3 AS: And then I'm coming up with a theory that how, "Okay, wait a minute, if we clarify more about what we want, maybe that's gonna help, but even if we clarified our assignment, it wouldn't have helped the feedback. They still could have had the same problem of, "We're not getting any feedback." But then it was the idea of coming up with the Feedback Friday and really naming it. And that's what I've learned from the world of marketing and all that, is that you've got to name something and repeat it. And so all of that is... And then I keep wanting to repeat that process, which is why I love doing the bootcamp 'cause it's six weeks, every 10 weeks or so I do it again, and that gives us a perfect opportunity. And that's what teachers are doing, they're doing again and again, right? 0:13:53.9 JD: Yeah, yeah, and it makes me think... I'm obviously living in a different world than you in terms of who the students are and who the customer is. But we... In our network of schools, we have two elementary schools, two middle schools, and a lot of your description makes me think of this first ever PDSA cycle we ran a few years ago when we were working on an improvement project we called Eighth Grade On Track, which is just like what it sounds. How do we make sure that our eighth graders are on track to go to high school? We don't have a high school in our network, but we have a high school placement process. One of the things that parents expect of us is that their child is well-prepared to go to a good academically-oriented high school once they leave us, and, of course, high schools are also expecting that from us. So the parents are the customer expecting certain things from our schools. The high schools that we feed into are expecting certain things from our schools. So of course we can't fulfill our mission, we can't be an important part of that sort of education system, if we're not preparing our students to leave us as eighth graders and matriculate into a solid high school. 0:15:09.4 JD: And I remember working through, what does it mean to be on track in eighth grade to predict that you're gonna go on and be on track and do well in high school? One of the interesting things that, as I was reading some research out of the University of Chicago on this, was that when you look at students in middle school, you see grades start to drop that's actually a leading indicator of things to come in high school, which makes a lot of sense, 'cause if you start to experience academic issues in middle school, high school is a little harder, academically, plus some of the supports that are in place in elementary and middle school start to drop away so that makes perfect sense. Bs drop to Ds in middle school, and Ds drop to Fs in high school. And, man, if you're off track, even in your ninth grade year, students have a lot of trouble bouncing back from that. So I remember there was a student I was working with named James and this exact thing happened. 0:16:19.4 JD: I was looking at his grades, I was looking at his GPA, his attendance, his discipline record in sixth grade and everything was on track. In seventh grade, it was mostly on track. Things were looking pretty good. And then all of a sudden, here we are in the first trimester of eighth grade, and his reading grade dropped from a B in seventh grade all the way down to a D in that first trimester. In a lot of places, that's not gonna... Especially if the rest of his grades are pretty good, good attendance, he never was in trouble, he's not gonna get on a lot of people's radar. But we have this on-track system in place, so once we saw that data, our team, we said, "Wait a second, James was on track in sixth grade, on track in seventh grade, and now all of a sudden, he's off track in eighth grade," and we started asking why. So we're adults, we're sitting around the table in a conference room, "Why is James off track? Why is he off track in eighth grade? Well, his B in seventh grade dropped to a D." 0:17:25.8 JD: "Well, why is that? Why did that grade drop from B to D?" We're looking at his scores, and he's got pretty high reading test scores in his class. And then we look at his homework grade. His home grade's really low in his eighth grade reading class. And so then we asked this next question, "Why is James' eighth grade reading homework grade low?" And then we get stuck, and this goes back to this whole point of this episode, which is students are the improvement secret weapon. So we're sitting around this table and we say, "How do we figure this out? Why is his reading grade and homework grade, low? Let's go get it. Let's go get it." [chuckle] 0:18:10.5 AS: And for the people who are working in a manufacturing company listening to this, it's like sitting in your office above the factory... 0:18:19.1 JD: Exactly. 0:18:19.9 AS: And looking at the chart and thinking, "I wonder why this is happening." 0:18:24.0 JD: "Why this happening?" Yeah. So this is when the conversation gets really interesting. We had never done this before. We go get James on the spot from his eighth grade classroom and say, "Hey, we're doing this thing where we're just trying to figure out what's going on with your grades." 0:18:40.0 JD: We're asking some "why" questions. We're basically using the 5 why tool, we have a piece of chart paper and listing these things out, and so now we've invited him into the room, we just say, "Why is your reading homework grade low?" And he says, "Well, I really do the easier less-time-consuming homework first, math and science and history are fairly easy for me. So I do those first." A pretty typical answer from an eighth grade boy, and so what he's basically saying is he does his reading homework last, "Well, why do you do your reading homework last?" 0:19:28.9 JD: "Well, I don't like doing my reading homework, it's too much work." "Why do you dislike doing your reading homework?" "It's too much work. It takes too much time. I wait to the last possible moment." And we said, "Well, what is the last possible moment mean?" And he said, "Well, I usually not only do it last, I do it on the bus ride to school in the morning." It's dark. It's bumpy. That's the worst possible place to do the hardest homework, but that's what he's doing 'cause he does wanna get it done, he wants to turn in something, but he's not putting any type of effort, so now what we've uncovered is, why is it exactly that his homework grade is low? 0:20:02.9 JD: Now, if I was just an administrator sitting in my office, I could see that D and say, "James needs to go to after school tutoring," or "James needs to do this," or "James news to do that." But none of that, like what you were talking about with the student complaints in your class, none of that would have been the actual root cause of James' reading homework problem. So then we said, "Okay, now we know what the problem is, what are we gonna do about it?" 0:20:35.6 JD: And I had just learned about this Plan-Do-Study-Act cycle thing, and I said, "Well, let's sit down and write... Literally write out a simple PDSA with James." And the basic question was, if we could do reading homework first, could we raise that homework grade to at least a 70%? So we kinda looked at what his homework grade had been and what he needed to do to pass, and so that's what we settled on. We said, "James, what do you think about doing your reading Homework first?" And again, typical eighth grader here's where some of the psychology comes in, he says, "I don't know... 0:21:09.6 JD: I don't know if I really wanna do this. I don't think it's gonna work. I hate reading, I hate my reading homework," I said, "James, I hate getting up in the morning and running, but when I do it... When I get myself to do it, I feel better." He said, "Oh yeah okay, I could buy that." And I said, "Could you try this for just five days," and he looked at me, he kind of nods and I said, "No," I literally got up out of my chair, he got up out of his chair to go back to class and I said, "I need you to look me in the eye and shake my hand and tell me you can do this for five days." And he said, "Sure, I think I can do this." 0:21:54.1 JD: He said, "At the end of the day, when I'm sitting in my homeroom class, when we work on homework... Do you mind if I sort of sit in the back of the room so I can concentrate better?" "No problem. Great." So we write this down for the next five days, James is gonna work on is reading homework first, we tell the teacher that's in the homeroom so she can check to make sure he's actually doing that, and then we talked to the reading teacher and said, "Hey, can you give us James' last five assignments in reading and can you put the next five in this table to see if this is working." So just over the next five days, he does this, I check in with his homeroom teacher... Yep, he's working on his homework at the kidney table back in the back of the room. 0:22:33.5 JD: And then we start seeing the data come in, so whereas... Right before this intervention, it's one out of five, that's a 20%, three out of five, that's a 60%, three out of five... Those types of grades. First homework comes in five out of five, second homework comes in a three out of five. It's gonna take some time. Then it's 6.75 out of 10, it's five out of five, four out of five. And you start seeing this sort of momentum building, and after five days, he not only has a C on those assignments, he's very nearly got a B. And so we're studying this. And we're saying, "This seems like it's working pretty well." He sort of out-did even our predictions, and so we go to him and say, "What do you think about this sort of reading homework-first intervention?" And he says, "Yeah, it's going pretty well." 0:23:27.3 JD: "You're gonna keep doing it?" "Yeah, let's keep doing it." We design a second PDSA, We're gonna check in in two weeks now, so you can kinda see how this process goes, you create this very small plan, the student was hesitant, but he gave really great answers, insights into why his grade was low, he helped develop the plan, so now he's intrinsically motivated, 'cause this wasn't something that we did to him, we did it with him, and he's starting to get some momentum, he had some immediate success. 0:24:06.7 JD: And so you can see even just the small little change had this huge impact on this kid, and then he starts to build this momentum, and this has really changed a lot about how we approach changes, whereas before we'd sit in a room and plan, plan, plan, plan, plan, and then you go implement, you're like, "Oh man, I wasn't expecting this to happen, I wasn't expecting this to happen, I wasn't expecting this... " In five days, we went and saw "What would happen if we put this plan in place," and it worked pretty well, and all of a sudden we're gonna do it for 10 days instead of instead of five days. 0:24:40.3 AS: So let's just break it down for the... For the listeners to understand, we often talk about PDSA and all that, I think the first lesson that I would take from what you've said is that, yes, it can be a formal thing where we sit down and write down Plan-Do-Study-Act, and we go through it in a formal way, but it can also be just an informal process that we go through, but let's just break it down. They'll... Explain to us, P-D-S-A, how does that break down for this specific thing? 0:25:14.1 JD: Yeah, so the P would be the plan. And so I think the important thing to keep in mind here, is one... We wrote it down. You know, that sounds simple, but that's a big first step. We wrote the plan down, we made a prediction, we said, if James does this reading homework-first intervention, we predict that he'll have a 70% or higher on each of his homework assignments over the next five days, so we've quantified what we think is gonna happen. Then it's just the who, what, where, when, and of the plan. So it was literally like on March 22nd, 23rd, 26th, 27th and 28th. 0:25:54.7 JD: So on the next five school days, James is gonna work at the back kidney table, he's gonna work on his reading homework first, his homeroom teacher, Ms. Kramer, she knows that this plan is being put in place and she's just gonna check that he is working on his reading homework and then his reading teacher who is. Dr. Brennan, she said, "I'll record his homework scores as I get them over the course of each of those five days, so we can see if this is in fact having the sort of success that we think it is." So the plan is our hunch, it's really our theory about how we're gonna improve James' reading grade, but we don't know if that's actually connected to the real world in terms of if it's gonna be successful or not, until we actually put it in place. So we made a prediction, we made a plan that plan included who's doing what, when, and then it also included a plan for collecting just a little bit of data to see if this PDSA is on track so that's the P, the plan. 0:27:03.9 JD: In terms of the Do then after those five days, we just came back together as a team and just said, "What actually happened? Did James go and sit at the back kidney table? Did he do that each of the five days? Did he work on his reading homework first?" And it actually... This is a pretty simple plan and it's only over the course of five days, so in terms of implementation, the Do, everything matched exactly with what we put in the plan, and part of that is because the plan was simple, straight forward and on a really, really, really short time frame. 0:27:43.1 AS: So that is him doing the plan executing the plan. 0:27:44.4 JD: Us running the tests. Yeah us running the tests. Now on the study, the difference between the Do and the Study is the Study has... That the plan has been run, the test has been run, and now we have the data in and so we looked at what happened for the five days before we started the study, and over the course of those five days, he had gotten a 53% average on his homework on those five days pre-intervention, after the intervention began over those next five days, he earned a 79% average on his homework, and that 79% was 9% higher than what we had predicted, so the intervention actually went better than we thought at the outset, so that gives us evidence... 0:28:36.1 JD: That one, we know what we're doing in terms of creating the plan in the first place, and two, that the implementation can actually be put into effect in a real school, in real classrooms, with all the constraints that you have with time and all that, all that stuff. And I think another big thing besides writing it down, besides having the structure of this PDSA, we had James, we had James there, so I think it was a pretty good plan because of that. And so... 0:29:01.6 AS: And then, so what happened? Okay, so we've got the Study and what about Act? What does act mean in this case? 0:29:09.3 JD: Yeah, I think I mentioned this in one of the last two podcasts that when I think of Act, I'm gonna do one of three As, I'm gonna either Abandon the idea that we put in place 'cause it didn't go so well, I'm going to Adapt the idea 'cause some of it went pretty well, but maybe there are some things that need to be tweaked or iterated on, or I'm gonna Adopt it, this intervention went so well that it's gonna become a part of my system. 0:29:41.3 JD: So in this case, because we've only done this over five days and it was successful, but that's a pretty short time period, now we're gonna adapt it, we're gonna adapt it. And in this case, we're gonna... We call this thing reading homework first, that's the name of the intervention or change idea. Now, instead of five days, we're gonna do this for 10 days, and then one other piece of the Act is going back to the appreciation for a system component, that you can improve one part of a system and destroy the rest of it, that sort of idea. We're focused on reading. And everything else was pretty good when we started this focus, but we wanna look at James' whole system. In this case, we're talking about grades, so we're not gonna do sight of writing class and math class, and science and social studies, those types of things, because we're all focused only on reading, so we added that as a sort of a second component to the second cycle so we're gonna run a little longer. We're also gonna add his other grades to the data we're collecting just to make sure that those things stay on track. 0:30:53.4 AS: And one of the lessons I've learned, John, in the stock market where I basically spent most of my life is that you have to also double-check that your process didn't go wrong in some particular area or is biased, for instance, just the fact that we're paying attention to James... And maybe the teacher is gonna grade things slightly different now because they know that we're looking and we're trying for improvement, and so you also have to ask questions and try to understand where the biases are because you may come to a conclusion, "Wow, this is great." And then you wanna think, "I'm gonna apply this more across more students or across more systems," and then you find out that it starts to fail and why does it fail because there was some kind of fatal flaw in the process. Do you have any thoughts on that? 0:31:43.1 JD: Yeah, I think that goes back that I think... Something we've talked about very early-on in this series, maybe in episode one or two. This idea is there are goals for accountability and then there's goals for improvement, and I think... Now bias can happen at any point, for sure, because we are paying attention to this more, but when you're in a system and the goals are accountability-focused, you're much more likely to get that sort of nefarious or "I'm gonna change my behavior, maybe not in the best way, because I have to meet this goal - my job's at stake, or... " 0:32:30.2 AS: In other words, when you talk about goals for accountability, let's say that you went to that teacher and said, "You're gonna get a bonus if this one particular case is able to really improve." Okay, now you've brought in a whole another element into this thing. 0:32:41.1 JD: Yeah, that would be the carrot side or the stick side, you know, "If this kid's homework doesn't improve, you're gonna get a bad rating." Something like that, and that... Like I said, I think there's still always potential for bias or doing some things subconsciously in terms of how hard your grading him or something like that, but I think it's much less likely to happen if we're sitting down and saying as a group, "Hey, what can we do to improve James' approach to reading class?" Versus those things that we just talked about, whether it's a carrot or a stick, a ranking that could be impacted, those types of things. 0:33:20.1 JD: I think in focusing on improvement-oriented goals, you're much less likely to sort of see that... See that type of behavior. I think a lot of it goes back to what's the aim, what's the aim or objective of the PDSA, what's the aim of your system in general, what's the orientation you have in terms of how you manage as a principal, the teachers or the teachers managing the students, I think that's where you have to be careful. And this sort of improvement orientation, I think helps overall rating and ranking accountability-driven system. 0:33:58.4 AS: So let me try to summarize a little bit about what we've talked about, and for the listeners, this is kind of our way of trying to make sure that we all learn from what John's sharing here. So the first thing you were telling the story about how it's important when eighth graders leave because you're preparing them for high school, and you talked about the idea of being on track and when somebody starts to fall that it's hard to bounce back, and then you identified James. 0:34:26.8 AS: And then you said, "Okay, we saw something sliding there from a B to a D, and that was his reading grade," and you thought, "What could we do about this?" And your first reaction was to sit back in your offices looking at the data and thinking about it, but instead you say, "Well, let's just bring him in here and talk to him." So this is the secret weapon you're talking about is getting the student involved, then you went through a PDSA, so let's just try to review that briefly, so the plan is, you guys came up with an idea and you wrote it down, and you had... 0:34:58.6 AS: Like who, what, where, when? So that it was clear what was gonna be done, also you made a prediction, because if you don't make a prediction, you don't have some sort of theory, it's very difficult to really understand what happened, and as Dr Deming says, "Without a theory, there is no knowledge." And you also had a plan for collecting the data too, to make sure that you had that. Then Do, meaning that you ran the tests and James did what was planned... In this case, it went well, 'cause he actually did it, and then after that, you have to Study where... 0:35:36.1 AS: After the test was run, so the test has to be run first, the Do has to happen first, then you started to compare the outcome to your prediction, and as you said, it was slightly better than the prediction that you had made, and then you came to the Act section, where you had to think, "Well, do I abandon... " "Do we abandon this? Do we adapt it? Or do we adopt it?" And part of what you said was that it's about adapting a little bit, maybe and saying, "Okay, what we've... " "We've identified this as reading homework first, maybe now we're gonna test that on a 10-day basis," but you also had to think about it. 0:36:07.9 AS: This is a critical part, you had to think holistically, you had to think systems thinking, because your objective was not to increase the performance in one area at the cost of another, so you had to look at it holistically, and finally, we talked about the risks that something like, this can go wrong. And if you're tying goals to accountability to the people involved, and all of a sudden they're being punished or rewarded based upon the outcome of that result, it's gonna be much more risk that it's not gonna go properly compared to looking at goals for improvement, anything that you would add to that? 0:36:45.2 JD: I think that was a really perfect summary. I think it was spot on. Spot on and the only thing I'd say is not every PDSA is gonna work that smoothly, I think, but it illustrates the key points of what a PDSA is, how simple it can be, how to connect ideas that are sort of in the universe to reality, what actually happens in actual classrooms and schools when you try something. I think that's the power of the PDSA. I think you nailed it in your summary. 0:37:17.2 AS: So ladies and gentlemen, now it's your turn. What's something that you can do a PDSA on, just like John has described to us? And what improvements could that bring, and most importantly for the teachers and the administrators out there, my question to you is, do you realize that engaging students is the secret weapon for improvement? John, on behalf of everyone at the Deming Institute, I wanna thank you again for this discussion and for listeners remember to go to deming.org to continue your journey. This is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming, and that is people are entitled to joy in work.