Prologue
The Dalmatian and Its Spots
Why Research-Based Recommendations Fail Logic 101
2004
"The Dalmatian and Its Spots" appeared in Education Week, January 28, 2004. It attacks the unsound reasoning educational researchers use when they specify features that programs should have to be "research based". The attack is directed to virtually all recommendations the field had formulated since "Becoming a Nation of Readers" in 1985 (including the report of the 2000 National Reading Panel).
An indication of the validity of the arguments the dalmatian article makes comes from the lack of response from various researchers and policymakers. The educational community has not presented a single argument to justify the practices or the logic used to formulate "research-based recommendations."
There has been some talk about replacing the category research-based programs with something like evidence-based programs. One of the reasons there is resistance to evidence-based programs is that the field still believes that it cannot identify specific programs but must somehow deal with “features” of programs. Get the hook. Until educators recognize that they are not dealing with physics and that the program-design game is more akin to inventing things than describing broad and amorphous categories like "phonics," the field is going to progress only with baby steps.
The Wright brothers had to orchestrate hundreds of specific details to make a flying machine. Yet, if any one of those pieces had been out of place or misconstructed, the machine would have failed. So it is with educational programs. You can't provide teachers with simplistic slogans and expect them to create programs that are highly successful because the criteria expressed through the slogans are far too general to result in effective programs. The field has to recognize that because highly effective educational programs are inventions, there is no intellectually honest way to describe their structure or why they are highly successful without presenting a myriad of criteria. These would not paint in brush strokes the size of a bulldog, but in picky details of how the tasks are formulated, how the example sets are designed, how the details of lessons are organized and sequenced from lesson to lesson so that only about 10-15% of each lesson presents brand new material, how exercises are designed so they are unambiguous about details of the content, and therefore, how the analysis of the content permits the progressive and systematic transmission of content to average and low-performing students.
If you think about it, you see that the program has to be an orchestration of detail. If it weren't, the moment-to-moment performance of the students would be lumpy, with no control of the details that could make it smooth.
Anyhow, until the research community becomes informed about the necessary features for programs that are effective with low performers, we'll just have to do the best we can with recommendations that are blatantly unscientific and far too general to be sound. About the only ones who benefit from the current research-based criteria are the publishers. They don't have to become concerned with details that make a program superior; they just have to plug in activities, label them "phonics" or "phonemic awareness" and lo, they will probably be adopted in states like California. Go to article.
