NY Mag: The Case of the Amazing Gay-Marriage Data
In 2014 an ambitious political science graduate student, Michael LaCour, co-authored a ground-breaking paper that blew everyone out of the water. The paper, published in the highly respected journal Science, offered concrete evidence that canvassing voters on electoral issues could markedly influence their opinions and sway their vote.
It contradicted nearly every previous study, and therefore surprised everyone in the field. No one apparently dared to ask the sensitive questions, “Were the results valid? Was the study trustworthy?” After all, it was in Science, and the other author of the paper was a highly respected expert in the field.
It took another ambitious graduate student, David Broockman, nearly two years to discover why LaCour’s study was a complete surprise.
It never happened. It was a complete fake.
Part of why LaCour’s results were so noteworthy was that they flew in the face of just about every established tenet of political persuasion. While past research had shown canvassing can be effective at influencing people in certain ways, the sheer magnitude of effect LaCour had found in his study simply doesn’t happen — piles of previous research had shown that, all else being equal, human beings cling dearly to their political beliefs, and even when you can nudge them an inch to the left or right, people’s views are likely to snap back into place shortly after they hear whatever message you’ve so carefully and cleverly crafted. Not so in this case: When LaCour compared the before-and-after views on gay marriage in his study, he found that opinions had shifted about the distance between the average Georgian and the average Massachusettsian, and this effect appeared to have persisted for months.
So when LaCour and Green’s research was eventually published in December 2014 in Science, one of the leading peer-reviewed research publications in the world, it resonated far and wide. “When contact changes minds: an expression of transmission of support for gay equality” garnered attention in the New York Times and a segment on “This American Life” in which a reporter tagged along with canvassers as they told heart-wrenching stories about being gay. It rerouted countless researchers’ agendas, inspired activists to change their approach to voter outreach, generated shifts in grant funding, and launched follow-up experiments.
But back in 2013, the now-26-year-old Broockman, a self-identifying “political science nerd,” was so impressed by LaCour’s study that he wanted to run his own version of it with his own canvassers and his own survey sample. First, the budget-conscious Broockman had to figure out how much such an enterprise might cost. He did some back-of-the-envelope calculations based on what he’d seen on LaCour’s iPad — specifically, that the survey involved about 10,000 respondents who were paid about $100 apiece — and out popped an imposing number: $1 million. That can’t be right, he thought to himself. There’s no way LaCour — no way any grad student, save one who’s independently wealthy and self-funded — could possibly run a study that cost so much. He sent out a Request for Proposal to a bunch of polling firms, describing the survey he wanted to run and asking how much it would cost. Most of them said that they couldn’t pull off that sort of study at all, and definitely not for a cost that fell within a graduate researcher’s budget. It didn’t make sense. What was LaCour’s secret?
Eventually, Broockman’s answer to that question would take LaCour down.
More at New York Magazine
Broockman discovered that the survey firm that LaCour claimed he had used to canvass voters about gay marriage in fact never worked with LaCour. Emails from a “vice president” at the firm had been forged.
The data set in the paper came from an unrelated study, and had been doctored to provide the results LaCour wanted.
No one caught the fraud, partly because there is a built-in reluctance among researchers to accuse their peers of fraudulent studies. As Broockman was told repeatedly, there’s no future in being a whistleblower.
In the end, however, Broockman and his two collaborators exposed the fraud, because the evidence was compelling. Science and the authors had to publish retractions.