What Works in Schools: A Research-Based Approach to School Improvement

Presenter: Bob Marzano, Marzano and Associates, Centennial, CO

This session is presented in separate parts. Use the buttons at the end of the transcription to navigate between each part.

Part Three

BOB MARZANO: Now, you might say, gee, I don't like that rubric very much. Good. Make up your own. Actually, no kidding, make up your own. Can you see the power — no kidding -- the power in an individual school? Step number one is identifying, for these nonacademic, non-achievement or, better term, lifelong learning skills, what are the ones that we want to give students feedback on? Which ones do we want to give students feedback on?

Then you make up your school-wide rubric that teachers systematically use in the classroom. How often? At least once a week. Now, remember, you have computer software to help you out here. At least once a week.

Now, does that take time? Yes, it does. Can you see an individual classroom teacher, though, maybe every Friday — let's take a high school teacher with 200 kids. Can you see, every Friday, the last thing you do in class for five minutes is you go through the class and you tell the students, look, based on my perceptions of you this week, I'm going to give you a score in the rubric for assignments, for effort, for participation, for behavior? Five minutes.

Can you see the possibility of student self-assessment in this area? Yes, absolutely. Why not? What a powerful message that would send to students. And that is, hey, we'll give you feedback in these areas and you give us feedback.

Actually, a little anecdote. I was working in Indiana a couple of years ago now. And when I presented this, there was a high school teacher who had done just that. She had her own rubrics and it was just in her class. I forgot the areas. But she kept track of about four or five things, as I recall. And it was every Friday. That's where I got the example. The last five minutes of class she would give rubric scores to the students. They would score themselves.

She would take that home over the weekend and just compare informally students' self-evaluations with her evaluations. Now, I've asked this question a number of times, too. And that is, do you think students tend to give a score that's higher than the teachers, lower than the teachers, or about the same? I have no data to back it up, but the common response seems to be students. You would be surprised at how tough they are on themselves.

In this particular example, the teacher said that she had one girl that she had scored quite high in terms of participation. For her, the girl was attending and right there at the edge of her seat. The girl had scored herself very low. And so she actually took the time to talk to the student and said, well, I don't get it, Mary — or whatever her name was. I thought you were right there with me the whole time. Why did you give yourself such a low score? And the student said something that went like — and I'm paraphrasing — did you think I was really paying attention because I was doing this?

And she kind of told on herself. Apparently she knew how to sit up straight, get your head off your shoulders, look at the teacher, and every once in a while put your hand up. And that meant that she was there. She said, but my mind was totally not there. So, the possibilities for student self-evaluation are huge here.

Well, I've probably belabored this factor. I apologize. But let me look at some of the questions that deal with this factor. Again, think of your school. Just kind of rate yourself on these. Challenging goals and effective feedback. In my school, we have an assessment system that is used that provides for timely feedback, at least every nine weeks, on specific knowledge and skills for individual students. Specific achievement is set for the school as a whole. Specific achievement goals are set for individual students. And performance on school-wide and individual student goals is used to plan for future action.

Please, just turn to the person next to you, how well do you think you're doing on those?

[Pause]

BOB MARZANO: Let me go back to the model here, just to remind you. In my synthesis of the research, I have said there are 11 factors that, within the current system of resources and time available that you can work on to dramatically increase the effectiveness of your school relative to student learning. School-level factors. It is a rehash of the school effectiveness literature, but I think there are some things we know now that we didn't.

Let's look at the teacher-level factors. These is not new research, right? What does an effective teacher do? They use good instructional strategies. They use good classroom management and classroom curriculum design. You might want to write that down; I'm sure you've never heard that before. Like I needed to tell you that.

Well, again, a lot of old information. There are some new things, again. One is the relative impact of schools versus teachers. Now, let me show you my numbers, and I'll do this in a metaphor.

Let's say that you took my family. And I've got four kids that go from ages 24, 25, 31, and 34, so they're all out of school obviously. All of my kids went to public school by design. And let's say I moved out of Colorado and I moved into your State, and my kids were of school age. And let's take my daughter, Carmen, 25, and let's make her a fifth-grader. And in mathematics she is at the 50th percentile. So, we move into your State, and Carmen happens to be assigned to an average school and she has an average teacher for two years running.

Now, here is what I mean by an average school. In terms of those five factors there, average is a pejorative term. That's unfortunate. Average is competent. They're doing okay. It's an okay job in terms of those five factors, challenging goals, effective feedback, guaranteed and viable curriculum. And not great. There is a lot of room for improvement, but they're doing a pretty good job.

Now, Carmen walks into the 50th percentile. Two years later she walks out of the 50th percentile. An average school, average teacher, for two years running. Has Carmen learned? You bet she has. Things are going just fine. She is keeping up with her cohort group.

A scenario that's not so good: Carmen is assigned to a highly ineffective school and a highly ineffective teacher. So, at the school level, things aren't going well at all and she happens to have a teacher, for one reason or many reasons, who is not very effective. She walks in at the 50th percentile and walks out at the 3rd percentile. Has Carmen learned anything?

Well, the reason I use mathematics is because it's more cumulative. Can you see where Carmen could have learned things, but it's so spotty and chaotic that she is really not accumulating the necessary knowledge and skill? So, she is losing ground relatively rapidly.

Carmen is assigned to a highly effective school but a highly ineffective teacher. So, the school has got everything going for it and doing everything it can on those five factors and doing a great job. However, the teacher, for one or many reasons, is highly ineffective. She walks in at the 50th and walks out at the 37th percentile.

A better scenario: A highly ineffective school and a highly effective teacher. She walks in at the 50th and walks out at the 63rd.

The best scenario: A highly effective school and a highly effective teacher. She walks in at the 50th and walks out at the 96th percentile.

And actually the scenario that gives me the most hope is a highly effective school and average teacher. She walks in at the 50th percentile and walks out at the 78th percentile.

Now, a number of researchers have done the same thing. They come up with slightly different numbers. So, take my percentiles — you always watch out for percentiles, first of all — take them with a grain of salt. Take the pattern seriously, though, because virtually every researcher or synthesizer or research who has done this has basically come up with the same pattern. So, I stand behind the pattern very strongly.

So, assuming that the pattern is valid, what are some generalizations you could draw about the impact of the school versus the teacher? Again, just turn to the person next to you, please, relative impact of school versus teacher.

[Pause]

BOB MARZANO: It's the teacher, right?

Actually, look at scenarios 3 and 4. That's where you see it. Scenario 3, a highly effective school and a highly ineffective teacher, Carmen loses ground. The opposite scenario, a highly effective teacher and a highly ineffective school, Carmen gains ground.

Now, I have said that the 6th scenario actually gives me the most hope, and that is highly effective school, average teacher. Here is why. I believe we have close to or getting close to about 3 million teachers in the profession. We have a normal distribution in terms of effectiveness. Does that make sense, 3 million people? Most people are clustered near the middle — they're competent. You have a few way out there on the skinny branches, incredibly effective, like the Jaime Escalantes of the world.

Unfortunately, you have some down here, highly, highly ineffective. Here is what that 6th scenario says to me. Everybody doesn't have to be Jaime Escalante. Now, I've never met the gentleman. I've read about him. I've seen the movie. I just loved it. He's a spectacular person. But if you watch the movie — and I assume that it's accurate — he gave a lot of his life to teaching. As I recall, he had a heart attack, correct?

Everybody cannot be Jaime Escalante. Maybe we all don't want to be Jaime Escalante. I don't think you can train people how to be Jaime Escalantes — no kidding. I mean, that is stuff that is innate, God given, all of the above. But I think you can take virtually anybody who wants to and make them competent in relatively short order. No kidding.

So, you don't have to have everybody operating at the 99th percentile in terms of their teaching ability, if the school is doing everything it can. So, if you're a building administrator, I'm going to harp at you again. If you're doing everything you can relative to those five factors and you have a competent staff, you will get learning that is astronomical by today's metric. You really will.

What about the people down here at the bottom end of the distribution? Well, I think, again, what we know about competent teaching is so straightforward that anybody who wants to, in very short order, can be a competent teacher if they want to.

The reason I use my daughter Carmen is that she graduated from college. She was in one of the States represented here. And she was offered a job as a full-time teacher in a district. Having never taught a day in her life, having no certification, having never taken an education course in her life. The reason is she is a warm body with a degree, and they were so hard up for teachers that they saw this young, bright woman who wanted to teach and they were going to offer her a job.

By definition, Carmen would have been at that end of the distribution. However, I think, again, we know enough about good teaching that, in very short order, she could have been competent.

What about the people at that end of the distribution who don't want to get better? Well, my good friend Ray Mackie was, I would go so far as to say, one of Al Shanker's right-hand men for the last five years. Of course, Shanker is dead. And I just saw Ray just two weeks ago, but I've been in touch with him over the years. And you could see it in Shanker's public appearances but also more in terms of his personal discussions. That was one of his big pushes at the end of his life. We don't police our profession very well. This is not the profession to hang out in. No kidding. And there really is no excuse for somebody not being competent.

That doesn't mean we all have to be great every year. But not to be competent and refuse to try new things just doesn't make sense at all. Again, this is not the profession to hang out in.

[Applause]

BOB MARZANO: So, what does it look like to be an effective or a competent classroom teacher? Well, I already mentioned that the factors are fairly straightforward. Use of good instructional strategies, use of classroom management strategies, use of classroom curriculum design. I will just hit a couple of these briefly.

Use of instructional strategies — well, I would assert that an effective classroom teacher has a wide array — first of all, this is perhaps the most obvious characteristic of an effective teacher — and it seems logical that an effective classroom teacher has a wide array of instructional strategies that he or she draws from. If you're familiar with a little book, called "Classroom Instruction that Works," that myself and colleagues at McREL — one is here, Dianne Paynter — worked on, what we did is an analysis, technically called a meta-analysis, of about 4,000 studies. And we asked this simple question: Are there any general categories of instructional strategies that have a decent, good track record in terms of enhancing student achievement, regardless of the content, regardless of the grade level?

So, we were on purpose looking for general categories. We came up with that list. As I say, they're in the book, "Classroom Instruction that Works."

Now, if you look at that list, we might have used some funny terminology, but is there anything new there? No kidding. Isn't that pretty obvious stuff?

Now, what you will find is that within each area, again, I think there are a few twists that we didn't have before. But that is pretty straightforward stuff. What we don't know is, are some more effective in certain content areas. We didn't analyze the data at that level. We don't know, are there some more effective for certain styles of learners. We didn't analyze the data that way. So, that is not "the" list.

I think, though, within the next five to 10 years, what you will see is subject matter organizations, like the math people getting together, and I think they will be very specific. They will be able to say things like, when you're teaching this type of student at this grade level, maybe even for this topic, here are the strategies that work the best. I think we are on the verge of some real breakthroughs in instructional strategies that have real specific influence.

Which brings up kind of an interesting question, for me at least. Apparently, Ben Bloom, when he was doing his — he didn't call it a meta-analysis, but that's what he did — for "Mastery of Learning," which eventually turned into "Mastery of Learning" — actually, the book "School Learning and Human Characteristics" contains all of his effect sizes. At that time, apparently he said we know something about effective teaching. There is fairly straightforward data here. We know what works better than other things.

And he posed a question, and I'm paraphrasing, that went something like: Will there ever be a time when there is such a thing as malpractice in teaching as there is malpractice in medicine?

Now, we're not there yet. I can't hold that list up as the list. But I think we are close. So, it brings up for me an interesting question: When we do have more specific lists, what will be the responsibility of the classroom teacher to use the strategies on that list?

Let me state it as a question. If you were running a school and you had the magic list — and again, that's not it — but if you had them, you know that this strategy works best with this student, what would be the responsibility of your classroom teachers to you, and how would you enforce that if you say there is one?

Just turn to the person next to you. What are your thoughts on that?

[Pause]

BOB MARZANO: That's a tough one, isn't it? Very tough. Sometimes, when we first started doing this, I remember thinking, well, by golly, if the research says this strategy works, then they should use it. But that actually contradicts something I've always said: I think it's valid to tell teachers what to teach, to a certain extent, but it is not valid to tell them how to teach. So, what do you do? That is incongruous now with a research list of strategy.

I actually have an answer. Let's see if it works for you. And I got the answer from my son. I mentioned I have four kids, 34 down to 24. If you have ever heard me speak, I speak about my kids a lot. I commonly tell stories about my son. Because the thing that got him into college was the movie Top Gun. Prior to seeing that movie, he was not going to go to college.

Now, I come from two Italian immigrant parents, who said, "Get an education." So, for my firstborn to say he wasn't going to go to college, it was like a dagger in my heart. His logic was good. He hated school. He was good with his hands. He was a mechanic. As a matter of fact, in his school he was called a gearhead. I don't know if you have gearheads in your school. And it's just what it sounds like. They like cars and fast things.

He saw the movie Top Gun, and he comes home and says, I want to be a fighter pilot. And I'm thinking, well, this will pass soon. However, he went to school, within a few days, I would say, and he actually figured out that you have to have a college degree to be a fighter pilot. So, I'm thinking, anything to get him into college I'll support, and then I'll talk him into something sensible.

Well, he got into college. He didn't drop this idea of being a fighter pilot. He couldn't drop it. I actually didn't want him to get hurt from not accomplishing a goal. I went so far as to call a friend of a friend, who was an instructor in the Air Force, and asked, what does my son have to do to be a fighter pilot? He said, send him to one of the academies. I said, no way, he's not going to make it into an academy. He said, well, he's got a tough row to hoe. He better have a degree in engineering. His physical ability, he better be in great shape. His eyes better be 20/20 or better. He should have his own private pilot's license.

I'm kind of writing this all down, and he figured out what I was trying to do, talk my son out of his dream. Now, what happened was the guy said something — and I'm paraphrasing — he said, don't rain on his parade. And I remember thinking, who is this guy to talk to me like that about my son?

My wife overheard the conversation, and here's a quote from my wife. She said, "Get out of his face."

[Laughter]

BOB MARZANO: Either support the kid or shut up. So, a stubborn Italian like his old man, let him make his own mistakes.

Well, a long story short, he graduates from college. His eyes actually were 20/20, but not quite as good as he wanted. He spent his own money and did eye exercises, which I had never even heard of. Eye exercises? What do you do, eye push-ups? What's an eye exercise?

[Laughter]

BOB MARZANO: There were actual cards that he would look at and, if you were a lizard, it was easy to do, because the eyes were on the side here. For a human being, it actually hurt. He religiously did his eye exercises, goes to sign up with the Navy, and they say, we don't need many fighter pilots anymore. He was crushed. They said, we give 75 percent of the slots to the Annapolis guys, 20 percent to the ROTC people, and only 5 percent for people like you.

Did you ever see that movie Officer and a Gentleman? That was the program he went through. And he was undaunted. He signed up and, to make a long story short, this day he flies a $40 million F-18.

[Applause]

BOB MARZANO: Yes, good for him. He deserves a hand.

Two and a half years ago, life came full circle for him. He graduated from Top Gun. So, he went through the course. Of course, my wife and his sisters, we all went out there. We're all real proud of him. And I'm real excited. I had seen the movie like 40 times before.

[Laughter]

BOB MARZANO: So, I go up there and I'm expecting it to be like the movie. Not even close. There was no champagne by the pool. It was a very Spartan graduation. I think there were nine students in his class.

There was no Kelly McGinnis to be found. I looked. I guarantee you that.

[Laughter]

BOB MARZANO: There was no volleyball that could be played or anything like that.

When he got done — it's way up in Fallon, Nevada — we went to Lake Tahoe for the weekend. He graduates on a Friday and we went to Lake Tahoe. And I said, so, what was it like? And he's explaining everything to me. Now, when they walk in the front door, they are given a syllabus, if you will, of 22 advanced tactical strategies that you have to master. Now, they were scored, by the way, on what we would call a six-point rubric, although he was not familiar with the term "rubric."

Now, remember in the movie there was the competition for the number one pilot, or crew. Not so. You either are or are not a graduate of Top Gun. There is no number one. Every pilot who has ever graduated from Top Gun has their picture up on the wall. It's kind of interesting, isn't it?

Now, it's still an honor to be asked to go to Top Gun. What scores do you think they get for the first few weeks in these 22 advanced tactical strategies?

Very low scores. Now, they think they're real hotshots, and they're failing everything. So, they're crushed the first couple of weeks. By the end, though, everybody has to get consistently high scores in all 22 or you do not graduate.

Now, there is a day that is two days before graduation, which is ceremonial only but important. He says that what they do is you know you're going to graduate now — in his class there were nine students — and you're pitted against an unknown adversary. You're told to be at a certain point in the sky at a certain time.

Now, you don't know who your adversary is going to be. He said they will get Air Force pilots, flying F-15's or F-16's, and sometimes they will get Israeli pilots. Every once in a while they will get a MiG up there.

Now, as an expert, soon to be expert, in these 22 advanced tactical strategies, the assumption is you know all 22 and can do all 22. However, you pick and choose what you consider appropriate. Do you see where I'm going with this metaphor?

Even with this list up here, no kidding, I think those are generic and straightforward enough, if a teacher calls him or herself a professional educator, I think they should not only know what those are but know how to do them. But as a professional educator, they pick and choose. If they don't want to do cooperative learning, they don't.

The metaphor is not over yet, though. When the soon-to-be Top Gun graduates go up against their unknown adversary, there is one requirement: They have to win. And if they don't win, they go up a second time. Believe me, they don't lose the second time.

So, what does that look like in the classroom? Classroom teachers, you pick and choose what you use and what you don't, but you have to win. And for you, winning is your students learn. That doesn't mean they all have to walk out in the 90th percentile. But it does mean, if they walk in here, they walk out there. And, by golly, if things aren't working this quarter, isn't it the teacher's responsibility to try something different? That makes total sense to me. It really does. There is accountability. There is expectation. There is professionalism and you pick and choose. But, by golly, if it is not working and, you correct me if I'm wrong, isn't it the case where some quarters, with certain students, your strategies work just great, the very next quarter, the same topic, different students, try something different.

Now, there is really nothing new there. There really isn't. But maybe our way of looking at this is new. Just take a second and turn to the person next to you and react to what I've said relative to a way of looking at instructional strategies and the research on instructional strategies, please.

[Pause]

BOB MARZANO: Let me go back to the big model again. I won't go through classroom management and classroom curriculum design. They're fairly straightforward. Let me jump to the student-level factors. Actually, I think this is an area of the greatest potential. Now, I have no data to back this up, but do you remember the results of the top 1 percentile schools? Do you remember that? I asked myself the question, so what are they doing that is different?

My guess, and it is a guess, is that they are actually addressing all three levels here. They are not only addressing what the school can do, what the teacher can do, but also they are looking at the student background factors. And what they are saying is, let's do something to mitigate against kind of negative starts relative to those background factors. So, I'm going to look through all three of these obviously very quickly.

Let me start with some funny terms here, I realize that. Let me start with home environment. Notice it says home environment and not socioeconomic status. That is an important distinction, it really is. And I don't know about you, I got my Ph.D. in 1974 from the University of Washington, and that's a long time ago, I realize that. But wasn't it kind of handed down from on high that socioeconomic status is one of the strongest predictors of academic achievement? Don't you remember that? Was that kind of a given? Of course. If you know there socioeconomic status, you can predict their academic achievement.

Not so, says Karl White. Karl White did a meta-analysis of, I believe, 600 correlations across I forgot how many studies. Now, it was done in the late-eighties and I don't believe it has been updated, but I believe you would find the same findings here. When he looked at how they defined socioeconomic status, he got real different answers to what is the relationship between socioeconomic status and student achievement.

In case in point, when socioeconomic status is defined as income only of the parents, it accounts for about 10 percent of the variance in student achievement. When socioeconomic status is defined as education of the parents, it accounts for a whopping 3 percent. When it is defined as occupation of the parent, it's only 4 percent. When it is defined as home atmosphere, it accounts for 33 percent. And then, finally, even income and education together, it's only about 5 percent.

Now, I don't know about you, but do you see any hope there? Well, I sure do.

Can you change the incomes of the parents? No.

Can you change the education? Can you change the occupation? No.

Can you do anything to impact the home environment? Yes, I think we can.

Now, a way to look at this is as value-added. So, if you say, gee, we have to get all the parents, or 90 percent of them, before we are doing any good. No, if you get one family that you have influenced, you have done something good. So, this is value-added. And it can be really very straightforward. And that is as simple as how about a little seminar that you give one night a week for four weeks, an hour and a half. We'll buy the donuts. You all come on in and we'll tell you some things to do that will help your home environment be more conducive to learning.

What are some of the things you tell parents to do? What do we already know? This is not rocket science. What would you tell parents to do? Primary people, what do you tell them?

AUDIENCE: Read.

BOB MARZANO: Of course. We've known that for years, read to your kids. Read to your kids. Sit down and talk to them. Try to have dinner together and talk even more aggressively. How about homework? Can you tell them some things about homework? Yes, absolutely. We know a lot about homework.

I think Harris Cooper just came out with a new book, "The Battle Over Homework," published by Corwin Press. Harris Cooper did the meta-analysis about 10 years ago. He is still the reigning expert on homework. There are very straightforward results from that, I would assert. There are still some questions, but pretty straightforward in terms of what parents can do.

Parents shouldn't actually do the homework, of course, but parents should set up a situation for the homework, a time, a place. Check with them: Do you have homework tonight? Communicate with the school.

Given that this is free to parents, you can even be somewhat intrusive. Why not teach parents about parenting styles? Now, I had my first child at 21. I would have loved to have some information about what parenting style was. I had no idea in the world. To this day, I remember — my oldest was my son Todd — I remember when my mother- and father-in-law left, after that first four days. And he was in one of those little bassinet things, those little chairs. And I remember thinking, he's staying.

[Laughter]

BOB MARZANO: He's staying for a long time, and it's my responsibility. I was just terrified. Well, I think the research is fairly directive in terms of parenting style and what we can do.

I will do an injustice to this in such a short time, but aren't there, in general, three parenting styles, gross categories? There is authoritative, authoritarian and permissive. I always get these mixed up. Authoritative goes something like, kids, I want set some rules and procedures for the next semester here. Mom and I have some ideas; we want your ideas. It's kind of a family meeting, kind of a Glasser family meeting if you will. We come up with the rules and procedures. And here are the consequences if you break the rules. There is always time for negotiation, too. That's authoritative.

Here is what authoritarian looks like. Kids, here are the rules. Go.

Here is permissive. Kids, life will be your teacher.

[Laughter]

BOB MARZANO: Mom and I will give you some input if you want, but you make your decisions and you take your lumps.

Now, I say the research is fairly clear on the effectiveness of those three. Out of the three, which has the highest positive correlation with student achievement? Well, obviously the authoritative.

Which has the second highest positive relationship with student achievement? It's authoritarian.

Permissive has, if anything, a negative relationship with student achievement. And it's funny, if you read the research of some theorists in that area, I think they are fairly clear. They say, if you're going to err on the side of anything, err on the side of structure. Err on the side of structure.

Now, how does that fit in the home environment? Well, I say, even giving parents information about good parenting style, again, you're not going to hit everybody; you don't have to hit everybody, but this is value-added. Even one family, you add an awful lot.

Let me do the background knowledge. And this is a funny one. Background knowledge and learned intelligence. I realize that is probably the one out of the list where people said, well, what is that? That's a weird term. It almost sounds like an oxymoron — learned intelligence and background knowledge. Let me give you the theory behind this.

There are a lot of theories on intelligence. I actually say that the underpinning of all of those, they are fairly consistent in one area. And that is most theories say that there are really two types of intelligence, which can be broken into more subtypes. One is called fluid intelligence, which is innate. You can't change that at all. The other is crystallized intelligence. And that is learned intelligence. And that is not innate.

Now, here is the $64 question: What is more strongly correlated with academic achievement, fluid intelligence, which is innate, and we are different on that, or crystallized? And the answer is: It's crystallized, the learned intelligence. As a matter of fact, fluid intelligence, you certainly have to have a certain amount of it, but you would be surprised how big crystallized is in the equation here. So, it's background knowledge, background knowledge.

Can you see the possibility that — and let's take the innate intelligence — that you can be born in either high or low on that? Does that make sense?

Let's give another axis, though. How about your experiential base, can you see a continuum, from very rich experience to very limited experience?

Now, let's start plugging in some kids here. Can you see that if you have high fluid intelligence and rich and varied experience, you, by definition, have high crystallized intelligence? Does that make sense?

Can you also, though, see the possibility that you can have relatively low innate intelligence but rich and varied experience and have pretty decent crystallized intelligence? Does that make sense? That is a really important point for me, because here is what it says: There are some kids who, even though they are not highly, highly, highly intelligent, have strong crystallized intelligence because of their background experience.

Guess whose kids those are? Your kids and my kids. See, my four kids had all the advantages. My guess is yours, too. My kids were taken on planes when they were 4 and 5 years old to different cities. We made sure they had cultural experiences. My wife dragged them to the opera whether they liked it or not. She actually dragged me to the opera, whether I liked it or not, but that's a whole other story.

[Laughter]

BOB MARZANO: So, by definition, my kids don't have to be high fluid intelligence. They had the background experience. They developed this learned intelligence.

Let's do the other scenario, though. Can you see where a student can have high fluid intelligence, be smart as a whip, but still not have high crystallized intelligence? Yes, that's the real tragedy. You know kids out there who have high innate intelligence, they just don't have the background experience.

Read Bob Sternberg's work here on practical intelligence, which is a very practical book to read. He has a book called "Practical Intelligence." And here is what he did — and the guy is a genius in terms of his look at intelligence, but he also speaks in a language we can all understand — he looked at people that have low intelligence as is measured by the typical intelligence test. But he looked at what they did in life and he looked at the complexity of what they did. And here is what he found:

There are people who score very low on some intelligence tests, but when you look at what they do, it is so incredibly difficult. Can you imagine a street kid, can you picture a street kid who bombs out on the test? But if you look at the life that street kids lead and the complexity of that life -- if you put me in that environment, I would perish within a week. So, just because of low crystallized intelligence does not mean that students don't have high fluid intelligence.

And the toughest area of all is if you come in with low fluid intelligence and you also don't have much background experience. So, here is the trick. You cannot change the innate ability, but can you change the experience of students? Absolutely.

So, let's increase the richness of their experience, and you will get an increase in the crystallized intelligence of people all across the spectrum. What might that look like?

Well, let me actually go through the items in this one, because I think you can be pretty prescriptive on this. Remember, this is meant to be in our school that we do this. So, just think of your school. In our school, students are involved in school-wide programs that directly increase the number and quality of life experiences and, number two, students are involved in school-wide programs of wide reading that emphasize vocabulary development. And the final one, students are involved in a school-wide program of direct instruction of vocabulary and terms and phrases that are important to subject matter content.

Let me go back again. Please, just turn to the person next to you and, thinking of your school, how would you score yourself?

[Pause]

BOB MARZANO: Let me go over these with you. Number one, students are in our school in programs that increase their background experience. This is obviously the most direct way of doing it — obviously. So, take kids on fieldtrips. Take them to different places.

Now, we're limited in that, aren't we? If we could put every kid in airplanes and bring them places, that would be the thing to do. But, obviously, we are very limited on that. So, are there some other ways to do it? Well, I think the research on mentoring programs is great. It really is. Academically, those mentoring programs, in general, I think have a profound effect, can have a profound effect, on student achievement. But we are somewhat limited in this area of increasing the direct experience.

Are there some things we can do, though, that are indirect? I say there are — tried and true strategies. Look at number two there: school-wide reading. Remember the old sustained silent reading. It still works — no kidding.

Now, I think we kind of misused it. I shouldn't say that. I think we expected too much of the sustained silent reading. I think we looked at it as a way of increasing comprehension skills, and it doesn't do a very good job of that. But if you look at it for its ability to increase the background knowledge of students, you get a whole different picture of it.

Now, notice that this has a twist on it: wide reading, with an emphasis on vocabulary. Let me explain that a little bit. If you look at the literature on intelligence, here is what you find kind of ringing out. Vocabulary knowledge always seems to be a good proxy measure for general intelligence. Does that make sense?

Remember, actually, Coleman used verbal aptitude, and he used vocabulary primarily as his indicator. Here is why. What cognitive psychologists tell us is that we have experiences, we store them in our head, and we put a label on those. And the more experiences we have, the more labels, with finer and finer and finer distinctions.

Now, if you look at the research on the relationship between vocabulary and academic achievement, here is what you find: a very strong correlation, too. So far, good news.

Here is the bad news, though. When kids come into kindergarten, given their differences in background experiences, they come in with a huge different in academic vocabulary — by some researchers count, there some kids from lower socioeconomic static backgrounds who come in with one half of the vocabulary terms that other kids come in with — like your kids and mine. And over time, what happens is they get further and further and further behind.

A fellow named Stanovich, a researcher in reading, coined the term the "Matthew effect." Here is the Matthew effect. A hypothetical student, Matthew, who walks in the door, not with enriched experience, has less academic vocabulary than kids from environments like yours and mine where they have a lot of experiences, they already start a leg behind. They don't learn to read as quickly. And what happens is they end up not reading at all.

The kid who walks in with heightened vocabulary likes to read, reads more, and the numbers on that are astronomical — how much reading some kids do versus others in a given period of time. Can you see what happens? The Matthew effect is they get further and further and further and further behind. And the very thing that they should do, which is read to enhance their vocabulary, they don't do. Hence, reading people said to have kids read widely. Have kids read widely. However, if you tack on to that, as they read, have them identify the vocabulary they're interested in and keep track of that, the impact on vocabulary is dramatically increased.

Now, let me add the third piece to this. And that is direct vocabulary instruction. Now, that used to be heresy. Actually, there are some groups, if you look at the science standards, it is kind of implied, but there is a strong implication that teaching vocabulary directly is not a good idea. Here is what they were reacting to: the inappropriate use of vocabulary instruction, teaching a whole bunch of science words by definition and never teach science. And obviously that makes no sense at all.

But if you look now at what the vocabulary researchers and theorists are saying now, well, no, let's be logical about this. You don't teach 25 words in a week; you teach a handful. You don't teach by definition; you give them an example and they draw a picture of it. What we are looking at is, over time, to develop this background knowledge, if you will, through vocabulary instruction.

The meta-analysis by Stahl and Fairbanks in 1986 I think is compelling. If my memory serves, it goes something like this: that if students involved in direct vocabulary instruction, and the instruction is on terms that they will run into in their content area — that's the key, not just those high-frequency lists — terms that they will run into in their content area, have a percentile gain of 33 points, on the average, in their understanding of the content. That's dramatic.

That is just compelling, if we do it right, these three together — increase their background experience, wide reading, but all along kids are asked, are there some words you're interested in, they keep a vocabulary notebook and they're taught vocabulary by their classroom teachers, and they keep track of that too.

Let me go to the last area, and then I will leave you alone. I'm impressed you stayed this long. I figured, after an hour, everybody would go, so I only had an hour prepared. So, this has really been a challenge for me.

[Laughter]

BOB MARZANO: Motivation. I'll do this quickly. Actually, the book I commonly recommend is by Martin Covington, called "Making the Grade." And I've never met Covington, but I certainly hope to some day. This book is, for me, this phenomenal synthesis of the research and theory on motivation over the last five decades. And basically he draws from certain theories, attribution theory — I won't go through all of that. But, fundamentally, he comes up with some very specific suggestions for what a school can do to tap into the motivation of students, some a little bit, the others a lot.

Now, the good news is this is an area where you can really turn kids around. The bad news is, to do this right would mean doing business very differently. Let me show you here.

There are five questions, I believe, for this area. In my school, students are provided with feedback on their knowledge gain. That's a direct, I believe, recommendation by Covington. And what he says is that, hey, if you can show kids that they are gaining — actually, a little background to this. He says, if you have kids who don't want to fail, and you're in a system where you compare one student with another, can you see the trap they're in? See, there can only be a few winners in the system.

As a matter of fact, he goes so far as to say that, given that probably the majority of students relative to academics are what he would call failure avoidant as opposed to success oriented, we have a system where there are a lot of kids who say, hey, what I'm being asked to do, it's going to hurt so bad when I fail that I'm not going to do anything. As a matter of fact, they engage in what he calls self-handicapping strategies. It would be easier for them to subvert their own learning than to try real hard and fail.

He says that one small way of changing some of that is to give kids feedback relative to their knowledge gain. So, you don't pit one student with another; you pit each student against him or herself.

Now, I hope you can see — remember the record-keeping system I was talking about — can you see where that possibly fits in here? If you have kept track of students on specific knowledge and skill areas using rubrics or whatever system you use, now you can show me — hey, Bobby, you're not at a 4 yet, but look what you've done. You've come from here to here.

Does that motivate students? I think it does. I really do. Actually, a little homey example of that. When my kids were really young, we went to see the movie Karate Kid. Remember, that Karate Kid? So, after the Karate Kid, this is my two oldest, Todd and Christine. After the Karate Kid, what do you think they wanted to do? Take karate, of course. So, the last thing in the world I want to do. Unfortunately, Mr. Roz Dojo is right down the block and he is having a special. So, we go into Mr. Roz Dojo and meet Mr. Roz, a wonderful gentleman, a Korean gentleman, and he spoke very broken English at that time. Now, one hand was missing, which immediately frightened the heck out of me. I thought, oh, my God.

[Laughter]

BOB MARZANO: As I got to know Mr. Roz, it was a boyhood accident that had nothing to do with karate. He was a great teacher and a wonderful man.

Well, what he did, he got me into the belts. You know the belts you get? Now, I tested and got my yellow belt. Now, I'll tell you right off, six-year-olds have their yellow belt.

[Laughter]

BOB MARZANO: However, as an adult, to get your yellow belt, you have to break a board with your hand with your feet. So, I actually broke a board with my hand and my feet. Now, I was so excited about my yellow belt, I still carry it. Do you want to see my yellow belt here?

[Laughter]

BOB MARZANO: And it really hit me — I didn't care that a nine-year-old had a black belt and I wasn't going to be a black belt. The knowledge gained really got me excited. So, a small thing, knowledge gain. And I won't go through all of these.

He talks about games and simulation that make total sense. Now, let me talk about number three: Students are provided with opportunities to construct and work on long-term projects of their own design. When you add the theory and research of two people — one whose name I'm going to slaughter and I've never done it right — Csikszentmihalyi — did I even come close to that? He wrote the book "Flow." He talks about peak performance. I've never heard his name pronounced. I believe it's Csikszentmihalyi. He is the one who studies peak performance, when people are doing things, in the very act of doing it, engages them.

He wrote a book called "Flow." When the Dallas Cowboys won their second Super Bowl in a row — and this was a number of years ago — Jimmy Johnson I think was the coach — and Jimmy Johnson was asked: To what do you ascribe your success here, two Super Bowls in a row? No kidding, he held up the book "Flow." It's not an easy book. He held up this academic book, and he said, I follow this.

Now, one thing that Csikszentmihalyi emphasizes is, he says, look, the $64 question is what makes people passionate about running a marathon or writing a book or getting good at music. Nobody knows. But apparently everybody has one or two of those things that they are passionate about. If you can engage people in that, and give them the wherewithal to do it — he says you have to have the resources to engage in your goal and the time and energy to do it — he says, when that happens, skills and abilities come out that you didn't even know that you had.

Can you go back to my son story? He was a nonacademic until he decided that he wanted to be a pilot. He saw a movie, Top Gun, and all of a sudden skills and abilities came out he didn't know he had. I sometimes wonder, what if he had gone to see Texas Chainsaw Massacre?

[Laughter]

BOB MARZANO: But I don't think that had anything to do with it.

Now, you add that to the work of K. Anders Ericsson. K. Anders Ericsson writes under the name K. Anders Ericsson, so I don't know if this is a man or a woman, but I'm going to say "he." If you look at K. Anders Ericsson's work, he looks a lot at expertise across the different fields — athletic, musical, really across the centuries. He would ask, what is it that made Mozart such a phenomenal musician?

Obviously, Mozart was way out on the skinny end of the branches in terms of talent. But it is also true that Mozart's father, I believe, was the first to ever write a book on how to teach young children how to play music. So, maybe it was the confluence of that incredible skill, along with that nurturing, that made him what he was.

K. Anders Ericsson has developed what he calls the 10-year rule. He says that if you are willing to engage in 10 years of deliberate practice — now, deliberate practice is not just going and hitting the tennis ball 100 times a day, it is being brutal with yourself. It is practicing like six times a week, an hour each day, and picking your weaknesses and going over it and over it and over it. If you are willing to engage in 10 years of deliberate practice, he says you can become expert — not world-class, but expert — in almost anything you want. He says there is only one thing the human body can't change about itself, and that is your height. So, if you want to get taller, I guess you're out of luck.

Now, that excites me at age 56, the 10-year rule. No kidding. Think about that. Ten years from now. Sometimes I think, I'm going to try this puppy out. If this thing works, 10 years from now the Denver Broncos are going to have this really, really old quarterback.

[Laughter]

BOB MARZANO: That's not true. But 10 years from now you might see this old Italian guy who can actually ski moguls even though he is genetically programmed not to be able to do that. Now, that excites me at 56. Imagine if I had heard it at 46. Or how about 26? Or how about 16? Or how about 6?

So, this one right here. If I were king for a day, no kidding, every high school, every middle school, every elementary school, we would have projects for kids. I don't know where it would fit, but we would say to them, what are you passionate about? And as long as it is not immoral or illegal, we are going to help you work on that. And we are going to really hold you to setting goals, what is the next step, what is the next step and, along the way, we are going to teach you things about yourself, about problem-solving, about decision-making.

Now, for me — I'll go back to where I started — I really think that we know what works. I truly do. And there is nothing new up there. If you don't like that model and you use somebody else's, you're going to find the same factors, basically. Or make up your own.

My question is: Do we have the will to attack the areas that give us the biggest bang for the bucks? And I think, at the school level, it's those first two. I think, in terms of the classroom teacher, we are doing a good job. I really do. But relative to the student, I think there are things we can do which take kids who walk in the door without advantages and still make them incredible learners.

When you get to motivation, maybe take some kids and turn them into mayors and congressmen and presidents, who would have never otherwise done that except for the fact that they got excited about something.

Well, thank you very much. I really appreciate your being here.

[Applause]

PREVIOUS PART