A simple syllogism to begin:
1. All people are ideologues*.
2. Teachers are people.
3. You know what goes here.
If you reject the premise, you probably want to stop reading. You are the problem.
If you accept the premise, you also probably want to stop reading. Some unpleasantness flows from it.
There is a secret that snakes through the history of education research. In fact, it’s not even a snake. A snake could easily slip into the rushes and go unnoticed. What I’m writing about looks more like a roaring springtime river bloated with snowmelt. Don’t fall in.
In their 1968 study The Teaching-Learning Paradox: A Comparative Analysis of College Teaching Methods, Robert Dubin and Thomas C. Taveggia analyze 40 years of research comparing the effectiveness of a range of college teaching methods, including lecture, discussion, tutorials, independent study, small group work, and TV courses (1968′s equivalent of online education). Their book can be summarized in two words: “No Difference.”
Dubin and Taveggia poured over the data of nearly 100 studies that compare teaching methods by using final examinations as dependent variables (a potential weakness I’ll discuss later). What they found should be disturbing to any instructor who has ever flown the flag for a particular teaching method, for example, favoring small group work over lectures.**
They repeat their conclusions frequently throughout the 86-page study, anticipating, rightly, that no one would listen:
In the foregoing paragraphs we have reported the results of a reanalysis of the data from 91 comparative studies of college teaching technologies conducted between 1924 and 1965. These data demonstrate clearly and unequivocally that there is no measurable difference among truly distinctive methods of college instruction when evaluated by student performance on final examinations. (35)
Lecture? Lecture plus discussion? Small group work? One-one-one tutorials? Self-directed independent study.
No difference. Regardless of method, students will earn the same grade on the final examination.
Rubin and Taveggia also compared small classes with large classes, as well as so-called instructor-centered vs. student-centered classrooms (a bizarre, Orwellian construct if I’ve ever heard one. More on that in future installments.)
Just to repeat: When measuring the performance on college final examinations, lecturing is no worse or better than other methods (despite the lecture’s oh-so authoritarian overtones). In fact, it does not matter one whit which teaching method is employed.
Sure, 1968 is a long time ago, but The Teaching-Learning Paradox has been cited over 200 times since then, and there is widespread agreement on its conclusions. Medical educators seem particularly drawn to (and perhaps repulsed by) its conclusions. This is unsurprising given the importance of final exams in medical school, and the heavy content-knowledge required to become a medical professional (again, Rubin and Taveggia are measuring the kind of end-of-semester knowledge acquisition that many of us may find limiting).
Olle Ten Cate, a medical school professor and former president of the Netherlands Association for Medical Education, published an article in 2001 called “What Happens to the Student? The Neglected Variable in Educational Outcome Research” that is largely a response to the problem presented by Rubin and Taveggia. Ten Cate summarizes the problem (and the accompanying feeling of frustration). He also, however, begins searching for a way around the paradox:
Yet, is it conceivable that there really is no difference in the effects of such different treatments in education? How can we sustain the idea that systematically different educational approaches, not during one hour, not a day, or a week, but during four or six full years and thousands of hours of ‘experimental treatment’, will show hardly any measurable differential effect other than student opinion? (83)
He also points to the money that is being wasted on such studies, since it has been clear for decades that the overall conclusion is “No Difference.”
If we put so much money, time and energy in such huge curriculum experiments, some day the community might not remain satisfied with the consistent finding of ‘no difference’.
You could easily connect the conclusions of The Teaching-Learning Paradox to today’s hot teaching technology, online education. A 2009 meta-analysis of online education by the U.S. Department of Education showed no significant differences in the learning outcomes of three different teaching “mediums” (online, web-blended, and face-to-face). The study’s conclusions claim that blended students performed “modestly better,” but if you dig into the study a bit more, it stipulates that “the studies in this meta-analysis do not demonstrate that online learning is superior as a medium,” only that many of these course required more from students and instructors and “It was the combination of elements in the treatment conditions (which was likely to have included additional learning time and materials as well as additional opportunities for collaboration) that produced the observed learning advantages.”
This lines up nicely with Dubin and Taveggia’s conclusions. If I can take some liberties here and subvert Marshall McLuhan, it’s the message, not the medium.
In fact, as Dubin and Taveggia note, there are only two factors that are consistent in all 91 studies they analyzed: students enrolled in a course, and each course featured a textbook. Lecture at them. Make them watch you on TV. Make them do the work on their own. Make them log into a website. Tutor them.
As long as they are enrolled in your course and reading a textbook……
You guessed it: No Difference!
HOW CAN I MAKE A DIFFERENCE?
Are you drowning yet? Rethinking your teaching method? Wondering whether or not this huge push for more interactive, student-centered learning environments (think ice-breakers, small group work, group projects, student agency, one-one-sessions) has been a complete waste of time?
Well, yes. Yes it has.
That is, if your central goal is to deliver content. The evidence seems clear. At the end of the semester, students will know about the same amount of stuff regardless of teaching method***. Read The Teaching-Learning Paradox and then sit through some faculty training on how to engage students. If you’re not furious, you can’t do math. Dubin and Taveggia’s work is lucid and straightforward, and seems to be supported with each new study on teaching methods. If your goal is for your students to obtain content knowledge by the end of the semester, you should be in open revolt against anyone who suggests that one teaching method is superior to another. It simply isn’t true.
What if content delivery isn’t your ultimate goal? And what might Rubin and Taveggia be leaving out? In future posts, I will consider other studies and books that present the issue from a slightly different angle. For now, let me return to Ten Cate’s paper for some possible solutions. First, he provides a potentially depressing anecdote that (after some reflection) presents a way forward from The Teaching-Learning Morass:
Some call it the VanderBlij Effect, after the Dutch math professor who delivered remarkably clear lectures. However, students attending his lectures usually received lower grades at the test than those who had not attended his teaching. The latter were forced to study so hard to master the material that they really grasped it. But the effect we are discussing may affect students in both groups.
Oh my. Even skilled teachers are wasting their time? This story actually offers an important (and hopeful) truth: authentic student-centered environments (and student effort and study time) can have an impact. In fact, this was the only thing that Rubin and Taveggia found that did make a difference:
We found two studies in, the literature which compared some form of study with no study and evaluated their respective outcomes on examinations covering ability to recall or prove knowledge of course content.These studies had a total of six comparisons between groups of students who studied and those who did not, all of which were independent comparisons. The results are significantly in favor of study. (26)
The grand irony of many so-called student-centered learning strategies is that they are just more instructor-centered strategies in disguise. It’s the soft authoritarianism of ceding control. Above, we find that if students actually take their learning into their own hands, it can make a difference. It seems to be the only thing that does. As one of my colleagues says, “I don’t teach no one nothing.”
The lesson of The Teaching-Learning Paradox is that if instructors apply their own methods (whether instructor-centered or student-centered) it will not make a difference. Hence, Ten Cate’s question, “What Happens to the Student?” He claims that the studies Rubin and Taveggia analyzed (and almost all subsequent studies that support their conclusions) have three massive flaws: First, they confuse an independent variable for a dependent variable. That is, the results of a final examination are not really the result of the teaching method, they are an extension of it. This is why, potentially, all of the final exam results do not vary. Second, these studies are not truly blind, and can never be. If they students know they are being taught, they will act differently. Third, the effects on the student are not being measured. Is education simply about inputs and outputs? Is it merely about transferring knowledge? Shouldn’t we be looking for models that measure the effects on student behavior, which is, ultimately the one factor that can make a difference, if we extrapolate from the above mentioned studies on “studying,” and on the true meaning of the VanderBlij Effect?
Maybe The Teaching-Learning Paradox does not present a paradox after all. It might simply be an infinite regress. When the twin mirrors of content delivery and final examination are made to face one another, you get a perfect, endless, pointless reflection.
END OF PART 1
*Perhaps I’m abusing this term. I simply mean that everyone operates from within a particular perspective or set of perspectives, and that we often, consciously or not, make judgments about the world based on assumptions that our perspectives are superior to others. I’m doing it right now. One purpose of this blog post is to point out that educators often charge forth into the classroom under the assumption that their methods of instruction (whether cutting edge or traditional) are the most effective ones available. Evidence to support such claims does not exist.
**The results of the study hold true for different mixtures of methods, such as combing lecture, discussion, and small group work.
*** Later, I hope to discuss the difference between “teaching method” and “teaching style.” I will also discuss some more recent cognitive research. It may well be that “style” is another “method,” and that style will also make “no difference.” I hope not.