I was cued to write this mini-rant by a conversation on Twitter. I don’t really feel that I should have to explain the null hypothesis to people within this community, but in contexts like sexual assault and rape, it seems that all proportional skepticism goes right out the window.
Ami Angelwings (of Escher Girls fame) tweeted about accusations, and how the accusation that “she’s making it all up” is actually itself an accusation, and needs to be vetted out. She went on to say that the null hypothesis in this case is not that “she’s making it up”. Someone I greatly respect in the skeptical community replied to my retweeting that, saying “it is, give evidence”.
Frankly, that’s a load. That isn’t how the null hypothesis works.
In science, when you put together a scenario that you’d like to test and you hypothesize that something will happen as a result of testing it, you are in fact playing a probability game. To give you an example, if you were to fire a loaded gun, the null hypothesis — assuming everything about the gun and bullet is in working order and that I know how to fire a gun — is that a bullet would come out (and subsequently put a hole in the thing I’m pointing it at). An alternative hypothesis would be to suggest that perhaps the bullet might be a dud and nothing happens, and that no bullet would come out. That would be a rarity, and if a pilot study of a single trigger-pull was involved, you might get the wrong idea about the hypothesis based on that one dud and that therefore bullets never come out of guns. But, it is a plausible alternative outcome. If you were to suggest that instead of a bullet, a shark would come out of the gun, that would be an extraordinary claim, and that would require an extraordinary level of evidence.
Now, let’s say the scenario I give you is my claim that I was at a shooting range and fired a gun. This is not a scientific experiment, it’s not something we can duplicate; it’s a claim, and it’s in the past. This might be a perfectly plausible claim to you (because I now live in America, and guns are practically full citizens here). This doesn’t require any significant evidence, because shooting ranges exist here, I have transportation, I’m of age. You have no particular reason to doubt that I have made that suggestion earnestly. However, it’s, for me, an extraordinarily implausible scenario, because I’m pretty anti-gun. I have trouble firing Nerf guns at human targets, and I am frankly a little afraid of holding a directed explosive charge in my hands in general. Too many things could conceivably go wrong, even if those things are grossly implausible, and I’d simply rather not take the risk.
Now, knowing THAT about me, you might think that me going to a gun range is now an implausible scenario, and you have reason to doubt that I did so. The null hypothesis is no longer that you have no reason to doubt that I claimed something absolutely ordinary to other people, because you now know that it’s very out of the ordinary for me. What would normally be the null hypothesis is now suddenly less likely in my specific case than the alternative, that I did not do the things I claimed to do. It would require more explanation — did I have a change of heart? Was I lying originally when I claimed my distaste for fireamrs? Or am I lying now?
What if I had claimed to go to the store? Seems simple enough, common enough. The null hypothesis is that I have no reason to lie, and people (and I) go to the store all the time, so I probably did. Now, if I said I went to the store and picked up a case of dragon feed for dirt-cheap, you’d be hard pressed to believe me because dragons don’t exist and “dragon feed” doesn’t seem to be a euphemism for anything in particular, so you’d need more explanation before believing me in toto. The circumstances of the claim make the claim implausible, not the claim itself. That dragon feed doesn’t exist, doesn’t actually mean I DIDN’T go to the store. Skepticism would require you to investigate the claim about dragon feed, but skepticism against the claim of going to the store is actually hyper-skepticism, skepticism from which is borne “Truthers” of all stripes.
With regard to events like rape or sexual assault, given that they apparently happen all the time, and that only a very small proportion of those are reported, and that only a very small proportion of those that ARE reported are “made up” in the sense that the person who claims to have been assaulted were not actually assaulted, the null hypothesis in the case of “I was raped” is not, in fact, “they made it all up.” That is, in fact, a leap, predicated on some lack of trust of the person in question, often derived from societal biases against the class of the person making the claim.
Now, one’s word is certainly not enough proof to convict a person. That doesn’t mean that when faced with a claim — “that man raped me” — and a counterclaim — “no I didn’t” — that the null hypothesis there is actually “no I didn’t”. The null hypothesis here is actually something more like “we can’t get to the truth of these dueling claims but can take no action in absence of hard evidence”. Yes, it is possible for the “null hypothesis” in a given situation to be misframed. What people believe the null hypothesis to this proposition to be — “she’s making it up” — actually requires that the woman is always lying. It isn’t enough evidence to prove that he DID rape her, but with the burden of proof being set for criminal trials to require evidence beyond a shadow of a doubt, the man will likely not be convicted regardless. But a lack of conviction does not mean a rape did not happen.
Rape, you should know, is distressingly common. Put a guy and a girl in a room together, put them in a scenario where the guy thinks he can get away with it (e.g. the girl is unconscious), and one in three of these guys would have sex with the girl without consent. Mind you, to get those results, you have to not use the word “rape” in the questions, because “everyone” knows rape is wrong. The problem, here, is that people don’t know what rape actually IS.
So, between that, and the fact that we absolutely know that rape is drastically underreported, we can then presume that the majority of rape claims are actually true (or at least true to the extent that the woman was in fact raped), because the number of rape claims is significantly lower than the number of rapes. Because this is a common occurrence, and because there are pressures against admitting that you’re a victim of such a common occurrence, the majority of such claims are true. The null hypothesis when confronted with a woman claiming to have been raped is that yes, she very probably was raped, because it is not an extraordinary claim, and ordinary claims require only ordinary evidence. Testimony absolutely counts as ordinary evidence.
That is not to say that the person claiming to have been raped knows enough of the assailant to accurately make an accusation, or has enough evidence to get that person put in jail, and those cases where the woman was indeed raped but cannot level an accusation that sticks are all often counted against the “false accusations” statistic.
I will further note here that because most individuals are not rapists, any specific claim against a specific person requires extraordinary evidence. It’s not like you’re accusing a person of pooping — most people poop, after all. You’re accusing them of doing something to someone else without their express consent, and the vast majority of people understand that to be wrong (though that number could be buffed significantly and I support every effort toward educating people that violating consent is wrong). That is why one’s word is insufficient evidence to put someone in jail. Hell, even with twenty-plus accusations floating around about Bill Cosby, one cannot simply throw him in jail without hard evidence of one of them, but with that many accusations surfacing, and with the statistics regarding accusations, it seems far more likely that he actually took advantage of people than that there is some conspiracy to ruin him.
But under no circumstances should the null hypothesis in this case be that the woman making a claim is simply making it all up, because it’s actually vanishingly unlikely that that’s the case. That is one of the lower probability outcomes of the scenario in question. Given all the social stigma of making the accusation, given the blowback you get just for saying you were raped (much less directing the accusation at a person, coupling an actor with the passive-voice action), the counterclaim of “she’s making it all up” actually requires evidence in and of itself. In fact, the vast majority of reported rapes go unpunished specifically because the scenario was manipulated so as not to leave evidence, so the perpetrator gets away with it.
That was Ami Angelwings’ point. One cannot simply give the “she’s a lying liar” claim a pass because it plays toward what you incorrectly think the null hypothesis is in the case of the claim that the “liar” charge is countering. That does not follow. That counterclaim is also a claim, and one must treat that claim with the same healthy skepticism as you treat the original claim.
TL;DR: if you’re willing to be skeptical of a claim without evidence, be equally skeptical of a counterclaim without evidence — especially when the counterclaim offers an even less likely scenario.
Skepticsm 101, folks.