On Effects

Timothy Burke has posted one of the most sensible assessments I’ve seen of the problems with “effects” research, spurred on by the vastly over-reported study recently released suggesting a correlation between time spent in day care and “disruptiveness” in school. Burke extrapolates outward to think about the persistent problem of “media effects” research, which has for decades attempted to create causal links between a series of social problems and the consumption of media texts (i.e., Teenage Mutant Ninja Turtles cause schoolchildren to kick one another, Marilyn Manson causes teenage suicide, violent videogames cause school shootings, and so forth). For whatever reason, I keep expecting us all to have moved beyond such simple causalities, and am always taken by surprise when any study suggesting that mode of cultural consumption x causes social problem y seems to achieve such wide purchase in the public imagination.

But then, by and large, we all want something to blame for such social problems other than ourselves, something external to our family structures and our under-supported schools, something that we can demonize without having to ask more difficult questions about our culture and its values and inequities. Burke concludes with a pledge that we could all bear to take:

Do not endorse research about social behavior or social psychology without first looking very carefully at the methodology and the effect size. If you would disregard the study on those grounds when it contradicts your own social views, disregard it when it endorses your views.

I’d add to this, though, that we might all bear growing a bit more skeptical about causality in general, resisting the suggestion that a correlation between phenomena can tell us anything more than that there’s a correlation between phenomena, particularly when the putative “effects” of the phenomena under study are, as Burke points out, “teeny-tiny.”

6 thoughts on “On Effects

  1. I wonder if this is at all inspired by seeing my children at breakfast the other morning. And, if so, whether I should be pleased or not at the notion we should approach these effects with skepticism. Hmm…

  2. That’s a riot. I swear, the potential for connection between these thoughts and your own brood didn’t occur to me until just now. But now that you mention it…

    Kidding! Your kids are adorable. And given their totally endearing response to running into their music teacher, it’s hard to imagine either of them being disruptive.

  3. I agree with both you. To my mind the “effects” problem is proof of the importance of humanities training for social scientists. If there’s anything that we teach – and there’s a lot – one of the most important ideas is the material effect or signifying power of the representations that we produce and consume. That is, the idea that there is no space outside culture called “data.”

    Unfortunately by the time these researchers pass their doctoral quals we’ve usually lost them.

  4. Global warming… not teeny tiny and not a social science question, unless the social scientists want to explain why people look at a melanoma and insist it’s a hicky.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

To respond on your own website, enter the URL of your response which should contain a link to this post's permalink URL. Your response will then appear (possibly after moderation) on this page. Want to update or remove your response? Update or delete your post and re-enter your post's URL again. (Learn More)