On "Obvious" Research Results

There is a tendency in my social circles sometimes to dismiss social science results that seem “obvious” and aligned with our views with, “Well, duh, why didn’t they just ask a [person who experiences that type of marginalization/trauma/adverse situation].”

I’ve seen it happen with studies that show that fat-shaming is counterproductive, and studies that show that sucking up to abusers doesn’t stop abuse, and probably every other study I’ve ever written about here or posted on Facebook.

To be honest, I’m often having to suppress that initial response myself. It is infuriating when we’ve been saying something for years and now Science Proves It. (Of course, science doesn’t really “prove” anything.) It’s especially annoying when some of the some of the same people who deny my experiences when I share them are now posting links to articles about research that says that exact thing, without any apology for disbelieving me.

At the same time, though, I try to separate my frustration from my evaluation of the research. In reality, the fact that a result seems “obvious” or “common sense” doesn’t mean that the study shouldn’t have been conducted; for every result that aligns with common sense, there’s probably at least one that completely goes against it. Considering the fact that negative results have such a hard time getting published in psychology, there are probably a ton of studies sitting around in file drawers showing no correlations between things we assume are correlated.

Moreover, research is important because it helps us understand how prevalent or representative certain experiences are, and listening to individuals share their stories isn’t going to give you that perspective unless you somehow manage to listen to hundreds or thousands of people. (Even then, there will probably be more selection bias than there will be in a typical study, in which the subject pool at least isn’t limited to the researcher’s friends.) I will always believe someone who is telling me about their own experience, but that doesn’t mean that I will assume that everyone who shares a relevant identity with that person has had an identical experience. That would be stereotyping.

So, sure, to me it might be totally obvious that people who make creepy rape jokes are much more likely to actually violate boundaries–because I’ve experienced it enough times–but my experience may not have been representative. It is very much still my experience, and it is very much still valid and I have the right to avoid people who make creepy rape jokes since they make me uncomfortable, but it isn’t necessarily indicative of a broader trend. (Of course, now I know that it probably is, because multiple studies have strongly suggested it.)

The weirdest thing by far about the “Why didn’t they just ask a [person who experiences that type of marginalization/trauma/adverse situation]” response is that, well, they did. That’s literally what they’re doing when they conduct research on that topic. Sure, research is a more formal and systematic way of asking people about their experiences, but it’s still a way.

And while researchers do tend to have all kinds of privilege relative to the people who participate in their studies, many researchers are also pushed to study certain kinds of oppression and marginalization because they’ve experienced it themselves. While I never did end up applying to a doctoral program, I did have a whole list of topics I wanted to study if I ever got there and many of them were informed directly by my own life. The reason researchers study “obvious” questions like “does fat-shaming hurt people” isn’t necessarily because they truly don’t know, but because 1) their personal anecdotal opinion isn’t exactly going to sway the scientific establishment and 2) establishing these basic facts in research allows them to build a foundation for future work and receive grant funding for that work. In my experience, researchers often strongly suspect that their hypothesis is true before they even begin conducting the study; if they didn’t, they might not even conduct it.

That’s why studies that investigate “obvious” social science questions are a good sign, not a bad one. They’re not a sign that clueless researchers have no idea about these basic things and can’t be bothered to ask a Real Marginalized Person; they’re a sign that researchers strongly suspect that these effects are happening but want to be able to make an even stronger case by including as many Real Marginalized People in the study as financially/logistically possible.

As I said, I do completely empathize with the frustration of feeling like nobody takes our experiences seriously until they are officially Proven By Science. I also wish that people didn’t need research citations before they are willing to accommodate an individual’s preferences for the sake of inclusivity or just not being an asshole. (For instance, if I ask you to stop shaming me for my weight, you should stop doing it whether or not you have seen Scientific Proof that fat-shaming is harmful, because I have set a boundary with you.)

However, if we take individual experiences as necessarily indicative of broader trends, we would be forced to conclude that, for instance, there is an epidemic of false rape accusations or that Christian children are overwhelmingly bullied in the United States for their religious beliefs. Certainly both things happen. Certainly both things happen very visibly sometimes. Both are awful things that should never happen, but it is, in fact, important to keep in perspective what’s a tragic fluke and what’s a tragic pattern, because flukes and patterns require different prevention strategies.

I’ll admit that a part of my discomfort with “well duh that’s obvious why’d they even study that” is because I don’t want the causes I care about to become publicly aligned with ignoring, ridiculing, or minimizing science. We should study “obvious” things. We should study non-“obvious” things. We should study basically everything as long as we do it ethically. We should do it while preparing ourselves for the possibility that studies will not confirm what we believe to be true, in which case we dig deeper and design better studies and/or develop better opinions. I find Eliezer Yudkowsky’s Litany of Tarski to be helpful here:

If the box contains a diamond,
I desire to believe that the box contains a diamond;
If the box does not contain a diamond,
I desire to believe that the box does not contain a diamond;
Let me not become attached to beliefs I may not want.

Even if your experiences turn out to be statistically atypical, they are still valid. Even if it turns out that fat-shaming is an effective way to get people to lose weight, guess what! We still get to argue that it’s hurtful and wrong, and that it’s none of our business how much other people weigh. Knowing what the science actually says at this point is the first step to an effective argument. Knowing what the possibly-faulty science is currently saying is the first step to making better science.

{advertisement}
On "Obvious" Research Results
{advertisement}
The Orbit is (STILL!) a defendant in a SLAPP suit! Help defend freedom of speech, click here to find out more and donate!

11 thoughts on “On "Obvious" Research Results

  1. 1

    On ‘obvious.’ One reason I feel that it’s good to research topics which provide evidence for the obvious is what is ‘obvious’ or ‘common sense’ is reliant on bias and privilege. Not all things believed by large numbers of people to be ‘obvious’ is really true.

    I mean, I’ve had people say that it’s *obvious* that Black people commit more crimes, or that wearing certain clothes make you more likely to get raped. Both of those are false, but widely believed. I also know people who for years knew that it was obvious that the police were racist, corrupt and violent. They knew that negative stereotypes, even when you don’t believe them, can still have a negative impact. People know what they believe is obvious, but not all things believed to be obvious are true. And people rely on their own experiences, and those of the people around them, whether those are really typical or not. I recall meeting a guy my first year in college. He thought the median income in the USA was 80,000 dollars. Not a dull guy, but he’d obviously never really been around poor people, and somehow the people around him (those working jobs at fast food, grocery stores, retail) who make far less just must have never really registered to him as a significant population.

    I’m also with you that listening to people and doing research aren’t two separate things. It’s one thing for me to listen to African American friends I have talk about their experiences with police. But research can provide an extra level of confirmation, one that even denialists have to deal with. In some ways, research is kind of like asking hundreds or thousands of people about their experiences, or looking it up somehow.

  2. 2

    There is also the point that studying even the most obvious stuff has the benefit of quantifying whatever it is, allowing more statistical examinations than simply “trust us, it exists.” The people that would dismiss it as mere anecdata now have no such dismissal readily available, unless they want to dismiss the entire field of social studies.

    Which some people ctually do, but it’s wasy to laugh at them.

  3. 3

    Yeah, obviously (duh) not everybody believes the same obvious things. Many people firmly believe in shaming, be it fat shaming or slut shaming or shaming a child who performed badly in a test. Because obviously your shaming makes them feel bad and because nobody wants to feel bad they’ll change their behaviour so people will no longer make them feel bad. That’s logic, people! Stands to reason!
    If you experienced that particular thing, you can say “you know, for myself this didn’t work. When people shame me for being fat I become sad and that drains me of all the energy I would need to change something. Instead I find comfort in food. You might still be the odd one out, the one person in millions who actually dies from vaccines, so real data is needed. This can confirm your experience or tell you that really, you’re the odd one out.

  4. 4

    We still get to argue that […] it’s none of our business how much other people weigh.

    In the context of the kind of society where everyone pays for their own healthcare, that’s fair enough. In the context of a society like the UK, where everyone pays into a national health service, it’s very much our business how much other people weigh. https://www.nice.org.uk/advice/lgb9/chapter/economic-impact . If you’re lucky enough to live in a society which takes care of its ill people for free, it’s not unreasonable to regard smoking tobacco, drinking alcohol to excess and being overweight or obese as fundamentally anti-social behaviour.

  5. 5

    the correlation between health and weight is tenuous at best

    And yet the UK government’s Department of Health feel able to accurately quantify the economic costs of people being overweight, and report this cost without any caveats. It’s as though they consider the link “obvious”. Which is what I thought this thread was about.

  6. 6

    #2 Holms

    There is also the point that studying even the most obvious stuff has the benefit of quantifying whatever it is, allowing more statistical examinations than simply “trust us, it exists.” The people that would dismiss it as mere anecdata now have no such dismissal readily available, unless they want to dismiss the entire field of social studies.

    Not only that. Studying that stuff can let you find non-obvious ways to improve the situation, too.

  7. 7

    I love bringing up this quote when someone plays the “it’s obvious” card.

    “Tell me,” [Ludwig] Wittgenstein’s asked a friend, “why do people always say, it was natural for man to assume that the sun went round the earth rather than that the earth was rotating?” His friend replied, “Well, obviously because it just looks as though the Sun is going round the Earth.” Wittgenstein replied, “Well, what would it have looked like if it had looked as though the Earth was rotating?”

    “Obvious” can change radically based on prior experience. It’s obvious to me that the room light will go on when I flip the lightswitch, because I grew up in a country with a reliable power grid. Not everyone does, though, and so my “obvious” isn’t universal.

    But I’m repeating your own points. Have you spotted Neuroskeptic’s latest blog post, which details how an “obvious” scientific finding probably doesn’t exist?

    The paper is called Romance, Risk, and Replication and it examines the question of whether subtle reminders of ‘mating motives’ (i.e. sex) can make people more willing to spend money and take risks. In ‘romantic priming’ experiments, participants are first ‘primed’ e.g. by reading a story about meeting an attractive member of the opposite sex. Then, they are asked to do an ostensibly unrelated test, e.g. being asked to say how much money they would be willing to spend on a new watch.

    There have been many published studies of romantic priming (43 experiments across 15 papers, according to Shanks et al.) and the vast majority have found statistically significant effects. The effect would appear to be reproducible! But in the new paper, Shanks et al. report that they tried to replicate these effects in eight experiments, with a total of over 1600 participants, and they came up with nothing. Romantic priming had no effect.

  8. 8

    We should do it while preparing ourselves for the possibility that studies will not confirm what we believe to be true, in which case we dig deeper and design better studies and/or develop better opinions.

    … Even if it turns out that fat-shaming is an effective way to get people to lose weight, guess what! We still get to argue that it’s hurtful and wrong, and that it’s none of our business how much other people weigh.

    Oh boy, do I feel that way about the bucketloads of social psychology research saying that rapport and liking with someone require a sense of similarity with them.
    I do not like those results at all, because to me they provide a justification to remain prejudiced and narrow-minded. Because if “the research” keeps on saying we feel better around people who resemble ourselves, then by being a follower of peer pressure, an ineffective bystander to social injustice, or even an out-and-out racist/sexist/homophobe/transphobe/etc., we keep on believing that we’re just looking out for our own psychological comfort. That the studies support that point of view, and so there’s no reason for humans to ever change.

    Yet I still argue that sticking to our own kind is bad for us in the long run. That being comfortable at all costs makes us feel happy and calm, but still sows the seeds of our destruction. That it may be human nature to be conformist and comfort-seeking, but for the sake of humanity and the best of ourselves, we have to make that wrong. We have to defy human nature, and our own instincts.

    (Which means, on that latter point, that we have to make a distinction between the kind of instincts that warn us about unsafe people, and the kind of instincts that misguidedly push away those too different from us.)

  9. 10

    However, if we take individual experiences as necessarily indicative of broader trends, we would be forced to conclude that, for instance, there is an epidemic of false rape accusations or that Christian children are overwhelmingly bullied in the United States for their religious beliefs.

    I just really wanted to reiterate this point. A huge problem right now is that we live in a world driven by convenient narratives instead of facts. There have been some high-profile false rape allegations in the last few years, and they tend to get a lot more attention than all of the other true rape allegations or unreported rape allegations. Without looking at the real facts (i.e., false rape allegations are very rare and rape is very under-reported) it would be easy to convince yourself that there is an epidemic of false rape allegations and more should be done to protect the rights of the accused and critically examine accusers and their accusations. However, in the world of objective facts the rights of accused are well protected and accusers already face a huge set of obstacles to overcome leading most rapes to go unreported and most of those that are reported to not lead to a conviction. False rape allegations do happen, and they’re tragic, but without objective research we can’t know the real scope of the problem to see what if anything can or should be done about it.

    The same narrative driven “obvious truth” is seen in beliefs on crime and violent crime generally. The facts tell us that the US is safer than its been since the 1960s, but the “obvious” truth is that we are overrun with crime. It’s a big country and an even bigger world; there will always be enough horrible crimes to fill a news broadcast.

    We can’t live in a world that’s driven purely by personal narratives. Anecdotes and testimonies can be great for hypothesis development and setting personal boundaries. But to drive policy and societal change we need research.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.