by Brian D. Earp / (@briandavidearp)

* Note: this article was originally published at the Huffington Post.

Introduction

As someone who has worked on college campuses to educate men and women about sexual assault and consent, I have seen the barriers to raising awareness and changing attitudes. Chief among them, in my experience, is a sense of skepticism–especially among college-aged men–that sexual assault is even all that dire of a problem to begin with.

“1 in 4? 1 in 5? Come on, it can’t be that high. That’s just feminist propaganda!”

A lot of the statistics that get thrown around in this area (they seem to think) have more to do with politics and ideology than with careful, dispassionate science. So they often wave away the issue of sexual assault–and won’t engage on issues like affirmative consent.

In my view, these are the men we really need to reach.

A new statistic

So enter the headline from last week’s New York Times coverage of the latest college campus sexual assault survey:

“1 in 4 Women Experience Sex Assault on Campus.”

But that’s not what the survey showed. And you don’t have to read all 288 pages of the published report to figure this out (although I did that today just to be sure). The executive summary is all you need.

Here is what the authors of the survey–prepared on behalf of the Association of American Universities (AAU)–had to say in their introductory remarks:

[E]stimates such as “1 in 5″ or “1 in 4″ as a global rate [are] oversimplistic, if not misleading. None of the studies which generate estimates for specific IHEs [institutes of higher education] are nationally representative.

They go on to highlight that only 19.3 percent of students who were contacted actually responded to the survey, despite incentives–a low response rate for these kinds of surveys–and that even they were not likely to be representative of the student body within their own schools.

Specifically: “An analysis of … non-response bias found [that] estimates may be too high because non-victims may have been less likely to participate” (see Appendix 4 of the AAU report for an in-depth discussion of the analyses used).

None of this is buried in the fine print. In fact, the authors of the report (still in the executive summary) explicitly chastise news organizations for their misleading coverage of previous surveys:

[M]any news stories are focused on figures like “1 in 5″ in reporting victimization. As the researchers who generated this number have repeatedly said, the 1 in 5 number is for a few IHEs and is not representative of anything outside of this frame. The wide variation of rates across IHEs in the present study emphasizes the significance of this caveat.

Another caveat has to do with definitions. “Sexual assault”–an incredibly loaded term–can mean a lot of different things in different contexts. In this survey, it means “nonconsensual sexual contact involving [either] sexual penetration [or] sexual touching.”

“Sexual touching” includes “kissing” as well as “rubbing against the other in a sexual way, even if the touching is over the other’s clothes.” (I’ll say more about this wording later on.)

What about “nonconsensual”? This means either that the act was physically forced, or that the person’s consent could not be obtained because they were “passed out, asleep or incapacitated due to drugs or alcohol.” But no definition of “incapacitated” is given, so it’s not clear how drunk (to use the example of alcohol) you have to be to meet this particular condition.

Interpretations could range pretty widely.

Are you a victim?

Finally, consider that the survey’s developers “specifically avoided” using the words ‘rape’ or ‘sexual assault’ in the questions they administered to the students. This was so that “respondents would use a set of uniform definitions when reporting on the types of events that were of interest.”

There are some sound methodological reasons for doing this. But as Ashe Schow has pointed out, it can also have the effect of eliciting much higher responses than would otherwise be obtainable, due to avoiding such harsh-sounding words. As a result, “many students who don’t [actually] view themselves as victims and [who] don’t believe they were sexually assaulted” could end up being counted as “victims of assault.”

There’s an obvious response here. “Well,” you might say, “they were sexually assaulted, and they are victims, even if that’s not how they feel.”

But let’s try a little perspective. Technical definitions on a questionnaire may not, in fact, line up with someone’s lived experience, and people should be free to reach their own self-understandings when it comes their status as victims (or non-victims). And some students may simply disagree with the definitions. Given, however, that the students were forced to choose between “yes” and “no” in response to the various questions–and were given no chance to explain or elaborate–these are not just trivial details.

A more accurate headline?

Now, much more could be said about caveats, but using just the information we have so far, we can see that a more accurate headline would look something like this:

Approximately 1 in 4 of 19% of a Non-Representative Sample of Women Who Responded to a Non-Representative Survey of 27 Colleges (Out of Roughly 5,000) Reported Experiencing Sexual Assault, Where “Sexual Assault” is Taken to Mean Anything from Being on the Receiving End of an Unsolicited Kiss to Forcible Penetration at Gunpoint, Regardless of the Particular Context 

Obviously, that’s too long to be practical – so let’s see if we can think of something shorter:

Latest Campus Sexual Assault Survey Paints Complicated Picture: Report Stresses Methodological Limitations 

That’s about right in terms of accuracy … but it wouldn’t sell very many papers.

Faced with this dilemma, the New York Times, a discourse-shaping publication with a massive readership, elected to go ahead with a headline that the authors of the very report they’re covering went out of their way to–repeatedly–emphasize was “misleading.”

And that’s putting it nicely. If you take into consideration the likelihood of inflated estimates due to non-response bias, a controversial definition of assault, and a relatively small, self-selecting sample, then the headline is simply false. (The actual content of the article isn’t much better. The New York Times author waits six paragraphs before mentioning the 19.3% response rate, and even then only in passing.)

Why this matters

So here is my problem with the New York Times coverage. In my view, even one assault is one assault too many, and what this survey does show is harrowing enough. There is no need to exaggerate the findings. Also sexual assault is serious. It isn’t something to be dumbed down into a simple factoid or turned into fodder for a superficial headline.

Even more to the point, there is a risk that “overselling” these kinds of findings will actually backfire in the long run, casting a shadow of cynicism over the efforts of those who are struggling–against not inconsiderable odds–to establish a culture (but hopefully not a bureaucracy!) of affirmative consent on college campuses.

The last thing our society needs is one more excuse to take rape statistics lightly.

One more thought

I said I’d come back to the issue of kissing. Remember that this counts, in the AAU survey, as “sexual touching” – and therefore (possibly) a form of sexual assault.

I expect that many people will be tempted to roll their eyes. Kissing? Sure, it might be unpleasant if you weren’t expecting it, but is it really in the same “category” as rape? Aren’t these broad definitions just being used to “inflate” college sexual assault statistics, when what we really care about is something more violent?

You might think that I’d agree with this view (based on my qualifications about the survey’s methodology, above). But I don’t think it’s actually that simple. Context matters. If the person who kisses you against your will–or after one too many drinks–is your professor, or someone who’s been harassing you all semester long, or a friend who’s now violating your trust, the emotional consequences could be pretty severe. It also depends a lot on who you are. Some people experience even “extreme” forms of assault and yet somehow manage to recover and move on with their lives. Others may be emotionally handicapped for decades.

So violating another person’s sexual autonomy, even if it feels like “only a little bit” to you, is ultimately a moral non-starter. It isn’t worth the risk.

Improving campus culture and sexual assault discourse

This doesn’t mean that every drunken pass at a party should be treated like a horrible crime. But it does mean that people of all genders, including men, women, trans people, queer people, and other gender-nonconformers (the latter should be highlighted because they are the group that reported the highest rates of sexual assault in the survey) need to come together in a spirit of good faith–really listen to each other–and try to promote a culture of basic respect when it comes to sexual relationships.

The goal, in other words, should be to massively reduce the incidence of sexual harm–not only on college campuses, but everywhere. But in order to do this, we need to be serious about how we estimate the prevalence of assault, frank about the limitations of what we know and don’t know, thoughtful about how we define our terms (including the range of behaviors we feel comfortable collapsing under one label), and painstakingly careful about how we communicate our findings to the public.

I know far too many people who have experienced sexual assault, running the gamutof offensive behaviors and contexts. I have seen first-hand the enormous toll that these kinds of violations can take on people’s lives. These individuals–and all of us–deserve a sober conversation about sexual harm, and the way it actually plays out on college campuses. We can do better than misleading sexual assault statistics turned into click-bait by the New York Times.

Reference

Cantor, D., Fisher, B. Chibnall, S. et al. (2015). Report on the AAU Campus Climate Survey on Sexual Assault and Sexual Misconduct. The Association of American Universities. Available here.

* Note that in this version of this article, compared to the original at the Huffington Post, a link to the AAU report has been added in reference to Appendix 4 (where non-response bias was estimated), so that readers may see those analyses for themselves.

Follow Brian D. Earp on Twitter by clicking here.