Yesterday I pointed out the curious manner in which journalists wrote up a study showing that non-believers, Catholics and evangelical Christians have smaller brains than Protestants who don't claim a "born-again" experience. But at least that study, though having a small sample size, was done by real academics at a real university using typical methods of analysis.
Today I see that ABC News is peddling sheer quackery with a big story about a "survey" that was done "on-line." This is sort of like if Rush Limbaugh asked listeners to call in and give their views on how President Obama is doing and then announced the results of the "survey." It really doesn't tell you much.
Anyway, this story is just abominable. The headline tells us "Atheists Have Best Sex Lives, Claims Psychologist." Which is a pretty weird headline, first off. Like, why do we care what a single psychologist has to say?
Let's get to the lede:
Darrel Ray, raised a fundamentalist Christian in Topeka, Kan., shed a heavy cloak of guilt surrounding sex after he left the church in the late 1970s, and wondered if his experience reflected that of others.
Today, he has finished research that he said bore out his hypotheses -- that religion and good sex don't mix. In an online survey of 14,500 people who had come from a religious background, he discovered that once they had abandoned their churches, their sex lives improved.
In his survey, "Sex and Secularism," which he publicized last week, Ray drew a direct correlation between guilt and sexual behavior. Not surprising, but he also learned that guilt eventually subsides.
"We find guilt is a pretty big thing," said Ray, the author of, "The God Virus: How God Infects Our Lives and Culture."
OK, then. I wonder if Ray -- clearly a subtle man -- uses the term "fundamentalist" to mean, you know, "fundamentalist" or to mean "Christian of some kind." Perhaps we could learn more about the particular upbringing, but I guess that's asking a lot considering that reporter Susan Donaldson James thinks an unscientific online survey with absolutely no scientific validity deserves the hype for her report.
The story is a complete mess. Take these two lines:
Atheists, he concluded, had the best sex of all. "They can speak with some authority," he said. "They were raised in very secular homes."
All his respondents -- over 18 and all sexual orientations -- had abandoned their churches and described themselves as agnostic or without a religious belief.
So all of his respondents left religious backgrounds but, on the other hand, his atheist respondents were raised in secular homes? One of these two sentences doesn't work in this story.
Anyway, which academic institution sponsored this "online survey"? Who peer reviewed it? What are Ray's academic credentials? Let's see:
Ray, 60, is an independent researcher who has worked as an organizational psychologist in corporations for 30 years. Before that, he spent a decade as a clinical psychologist.
Not just that, but his assistant was a student at University of Kansas. And if that doesn't impress you, the "online survey" that they conducted "drew 2,500 responses an hour"! That metric could not be more meaningless, could it? I mean, yes, you post a link to your "online survey" over on P.Z. Meyer's emporium and Metafilter or whatever, and I'm sure you can get 2,500 hits from atheists in a given hour. So what?
Anyway, we're told that he "only surveyed those who had said they were once religious, not those who today practiced their faith."
Wait, so how do we know that atheists have "better" sex than non-atheists? What controls were put in place? What was the control group? I mean, if you were to believe in completely uncontrolled online surveys, which you are too smart to do, the most we could say is that atheists who were once religious claim that their sex lives improved once they quit their religion. But we couldn't say that atheists have better sex than other people because no other people were surveyed. It's like if I surveyed myself and decided that I have the best sex life in the world.
The story goes on to pretend that there is scientific value in the survey, noting that the "respondents were predominantly highly educated and affluent. They included hetero, same-sex and intersex couples. In the first day, he received 2,500 online responses."
Wait, so there was one hour that he got 2,500 responses and also the first day total was 2,500? Did Rain Man write this? It's just bizarre!
Then we get to this weirdness:
"His results make a lot of sense -- why people who are religious emphasize guilt in sexual behaviors," said Tara Collins, president of a multidisciplinary group of researchers at Kansas University's psychology department, who gave Ray feedback after he presented his survey results.
Collins and others were impressed, but they did express concern about his causal statements and urged him to make some modifications. Ray, she noted, had not looked at the satisfaction level of those who continued to practice their faith.
His research will not be published by an academic institution because it has not been peer-reviewed.
What? What, exactly, were they impressed with? What is there to be impressed with by an online survey of self-selecters? Other than nothing, that is. I had to look Collins up because I couldn't believe any academic would say anything like this. Turns out she's not an academic. She's an undergrad student at KU. Why the story doesn't describe her as a student, but as "president" of a psych department group is just odd. I'm going with the theory that Susan Donaldson James is also an undergrad student at KU who won a contest to put a story up on a big web site like ABC's. Please don't tell me she's paid to write this stuff.
In any case, did you notice the last line of the excerpt? Which comes in the 20th paragraph, by the way? Of course it's not peer reviewed! So why did you spend the previous 19 paragraphs pretending it was real scholarship? It's a freaking online survey. An online survey! Come on, ABC, this isn't 1997 when the interwebs confused us. It's 2011 -- if you're a journalist who doesn't know that online surveys are easily manipulated, hard to control and perhaps completely meaningless, you can turn in your press credentials at the door.
After pointing out that the "online survey" wasn't peer reviewed, the reporter quotes Mark Regnerus, a Ph.D. in sociology who has written two books on sexual behavior. He points out that the survey was not scientific. We're into the twenty-something paragraph at this point, keep in mind:
"It appears that it was a 'fill it out if you want to' kind of survey that is not random, not nationally representative, and relies entirely on self-selection," he said. "In other words, they have data from people who felt like filling out a survey on atheism and sex. As a result, I am not surprised at their findings."
He points to some of his own research on sex and guilt and notes that Ray is not an established researcher at a university and the survey used research methods that don't meet the standards of most published social science.
Ray responds by pointing out that Alfred Kinsey's research was done the same way. He's right. This is the major problem with Kinsey's results and why he thought that, like 48% of all women were prostitutes or whatever. (Only slightly exaggerating here.) In any case, the story goes back to pretending like Ray's research is meaningful and points to other discoveries he made.
This story does a disservice to all believers and nonbelievers in that it presents shoddy research as something of value. But it does even greater harm to statisticians and sociologists. Some journalists may not get religion, we know. But they sure as heck don't get science or statistics either.