Counter-Science before Data Science

By Os Keyes

This is a cross-post of the piece titled “Can Data Do Good?”, now out in Issue 18 of Logic Magazine

Data has eaten, is eating, will eat, the world. What do we do about it? There are about as many answers as there are people asking (probably more). But if we set aside those who advocate rolling over with a smile on your face, responses tend to come in two particular flavors.

The first—let’s call it “data abolition”—disputes the entire question, particularly the “is.” It disputes the inevitability of datafication, in other words, and suggests pushing back, hard, against the practices that are advancing data’s eating-of-the-world and the narratives making that consumption seem so inevitable and complete. The second—let’s call it “data co-option”—argues for datafication as a source of hope. In particular, it maintains that datafication might be co-opted to challenge the very dynamics causing negative outcomes, thereby creating a kind of data resistance, or “data counterpower.” If data is a new register of power, we should work to twist it back on itself.

This second approach underpins what is increasingly referred to as data activism: using data to address injustice. For instance, we can look at the (in many ways) fantastic book Data Feminism by Catherine D’Ignazio and Lauren F. Klein. As the title suggests, Data Feminism is about how data collection and mobilization can be of use in achieving feminist political goals. Data, the authors argue, can be a tool of resistance. A quintessential example of such data activism would be Mapping Police Violence, a tool built not only to track murders by law enforcement in the US, but also to highlight how underreported and patchy the data on such killings is.

There’s nothing wrong with data activism, and no inherent contradiction between co-option and abolition: many people hold both views, including myself. You can simultaneously be skeptical about the inevitability of data and pragmatically willing to use it where it appears. The difficulty is doing both in parallel: to deploy data only when it is strategic to do so. Overbalancing—skewing to data—risks having data co-option campaigns being co-opted themselves, naturalizing datafication as beneficial or inevitable. Further, such efforts might not bear fruit. Being strategic, then, requires knowing about data’s usefulness. Where does data activism work, under what conditions, and for whom?

A lot of writing advocating such activism is quiet on these questions. In Data Feminism, the authors note explicitly that “data alone do not always lead to change—especially when that change also requires dominant groups to share their resources and their power.” But despite this caveat, there’s little discussion of where data does, or does not, lead to change: not only its potential, but the situations where it doesn’t work. Where does the reliance on data risk the harm of naturalizing datafication, without the benefits of addressing injustice? What are the factors involved in data activism being effective?

These are the questions that I want to try to answer. Not through looking at the now, of data activism in the era of data science, but at the past: at efforts to do data activism before data science was a term. In particular, the history of trans people’s use of data to fight for access to healthcare holds lessons for how to strike the right balance between data abolition and data co-option. What happens when we look not forward, to the hoped-for (or feared) future, but backwards, at the ancestors of data counterpower itself?

An Embarrassment of Data

But: why focus on trans healthcare? The answer has to do with how it became institutionalized (so much as it is) within medicine. Without digging too much into the history, the important thing to understand is that when trans medicine became recognized in the 1960s, practitioners partly got involved for research reasons. That is: they were interested in understanding the outcomes of treatment, not only through providing it but through making patients the object of study.

One consequence of this was the model that treatment has often taken, that of the dedicated Gender Identity Clinic (GIC): a one-stop-shop for evaluation, therapy, hormone prescription, and surgical access. GICs, often based at university hospitals, enabled doctors to control the entire process (and the patients) in a way amenable to research. And while this model has changed to some degree, it is the GIC-based clinicians—with their interest in research, and the accompanying claim to scientific authority—who have tended to end up running the show. In some respects, then, trans healthcare could be seen as an ideal site for data-based change: a putatively scientifically-driven space in which many of the most powerful figures are professional researchers.

One nation that follows the GIC-based model is the UK, and my first story starts there, in the 1990s—some thirty-odd years after the clinics were established. Under the National Health Service (NHS), anyone seeking treatment with their local doctor is funneled to one of a (small) number of dedicated, NHS-run clinics around the country. Along with their selves and their medical records, patients bring a promise of funding—a promise that their local branch of the NHS, rather than the clinic, will foot the bill if certain conditions are met.

The key word is “if.” For starters, NHS branches had (and have) wildly inconsistent policies for what they will fund for trans therapeutics. Some only funded psychiatry; some funded both psychiatric support and surgical treatment; others funded both, but for a limited number of patients a year. To make matters worse, even if funding was theoretically available, trans people ran into the fact that GICs’ low priority to the NHS—and, more broadly, its highly conservative and cautious approaches to trans care—makes for long waiting lists for even initial appointments, let alone treatment. At one point a patient seeking surgery through the Charing Cross GIC in London was held on the waiting list, and then in therapy, for fourteen years. All of this, but particularly the funding deficiencies, was often justified by the NHS with claims that treatment was not important enough to be a high priority, and that care—particularly surgery— did not have a strong enough evidence base showing it improved patients’ quality of life to be funded.

Into this stepped Press for Change (PfC), a long-running trans campaigning group in the UK that has fought for, among other things, increased access to medical treatment. Robust data showing positive outcomes for patients would seem to challenge both of the NHS’s excuses for not resourcing it: not only would this data provide certainty on the question of improved patient outcomes from surgery, but it would demonstrate the urgency and importance of providing funding in the first place. So in the mid-1990s, PfC sought to gather just such data. They launched a survey of patient experiences, both online and on paper. In other words, they used data to gather and surface marginalized people’s experiences and desires in order to challenge public policy—a pretty canonical example of what we might now call data activism.

So what happened? When I sent that question to Claire Eastwood (who organized the survey) through PfC co-founder Christine Burns, the answer came back: nothing. The data was collected, but it was never analyzed. Why? In a word, “capacity.” PfC was, in Claire’s words, “massively overloaded as the [overall] campaign expanded, and… sadly this task got overtaken by other urgent and vital work, and never made it back up the priority list.” As Burns put it, the problem wasn’t an absence of data—they had an “embarrassment of data”—but rather that PfC was “always on the edge of biting off more than we could chew,” and the survey pushed them over that edge. In theory, the data could have been used to make a stronger argument for the efficacy of surgical care. But in practice, making that argument through an analysis of the data was more than PfC had the resources to do.

The point here is that the work of data activism is, well, work: work to collect data, to analyze it, to publish, and to use it. If movements have one thing in common, it’s there are always fifteen tasks for every hour of the day—and the more pressured a movement is (the more vulnerable their members, the more urgent their tasks), the greater the cost of taking on new tasks. What this means is, practically speaking, an inverse relationship between the cost of data activism and the need to address the problem. The more pressing the injustices that activism confronts, the more pressed activists are for resources—and so the less likely there is to be capacity for data collection, analysis, and reporting.

Who Gets Counted

Of course, claims about care efficacy aren’t the only argument used to deny resourcing for trans health and concerns. Another is population, the topic of our next example. Or to phrase it as a question: how many people want gender-affirming medical care?

The authoritative answer, for the longest time, was “between 1:30,000 and 1:100,000 people”—and by authoritative I mean that it even made its way into the American Psychiatric Association’s (APA) official diagnostic texts. The reliability of this answer matters quite a bit because, to paraphrase D’Ignazio & Klein, who gets counted counts. A low number means an argument for low funding for care, a high number for increased funding—and, if politicians are cynical, an increased need to at least pay lip service to a population’s needs; people, after all, vote.

It turns out the official estimate is not very reliable at all—in fact, it’s almost certainly a dramatic undercount. The reason is its source: it comes from a range of studies done by GICs in various countries in the 1970s. This creates two problems. One: the 1970s was a very different time (mostly in bad ways) for the shape of trans healthcare, and for articulations of trans identity and desire. Two: the GICs doing this work at the time were, to varying degrees, awful.

With few exceptions, GICs across the world have been known (particularly historically) for their onerous diagnostic and selection criteria. Some excluded married trans people; trans people who had ever had children; trans people who were gay, lesbian, or bisexual; or trans men altogether without consideration—and that consideration included literal years of psychological tess, psychiatric interviews, and required life changes, including (at some) a demand that the patient quit their job and transfer to a more “gender-appropriate” one, regardless of the impact this might have on the stability of their life. Paradoxically, many even refused treatment to patients for the sin of having successfully sought treatment through other routes.

What’s more, trans people—particularly those with access to community spaces—knew the environments were unpleasant, and often sought care almost anywhere else if possible. The result is that the estimates from GICs were less estimates of how many people wanted treatment, and more of how many people, having exhausted every other option, successfully passed through highly rigid clinical processes. Extrapolating population-level data from those accepted into the clinics is like extrapolating how many people buy cars from purchases at a Ferrari dealership; there’s a lot of filtering going on before someone even gets to signing the paperwork.

But the answer to “how many people?” matters, and one person who saw that was University of Michigan professor Lynn Conway. A trans woman and renowned computer scientist, Conway decided to make her own estimate, based not on forty-year-old data from the GICs but—thanks to her contacts in and knowledge of North American trans communities—the far larger number of private practitioners. With a wider and more representative sample of treatment pathways, her 2001 research came up with a very different number: not 1 in 30,000 people seeking gender-affirming care, but 1 in 250. And in retrospect, these numbers are (quite clearly) closer to the truth. Conway didn’t simply publish them online—she presented them at the biannual conference for researchers and practitioners around trans healthcare, and even published them in a gender studies journal.

So when the APA’s Task Force on Gender Identity and Gender Variance met in 2008 to write a report updating its approach to questions of gender, one would expect them to take these challenges seriously. And they did address them: in a footnote, after reasserting the validity of the 1970s numbers. Not only that, but the footnote in question mentioned Conway’s work only in order to dismiss it because it “seems to represent a minority position among researchers, although transgender activists tend to endorse the study.” No methodological challenge; no claim the study was incorrect. Simply wholesale dismissal. Why? Because if the answer were true, the medical researchers would have spotted it before Conway did. Because if trans people agreed with the estimate, it was automatically suspect.

The reason for this response is fairly obvious: power. Professionals in trans healthcare—particularly at GICs—get a lot of their official power and authority from the perception that they are singular experts in all things trans. The Clarke Institute in Canada, for example, was for the longest time the Canadian government’s sole source of expertise on (among other things) how trans people should be treated in prison. And if those experts admit they can’t even be trusted to count—well, what can they be trusted on? Accurate or not, published or not, Conway’s study challenged clinicians’ authority. It’s interesting to note that in the newest version of the Diagnostic and Statistical Manual (psychiatry’s bible), the APA did provide a higher prevalence (and a caveat that this was likely an undercount), but credited GIC-based researchers with this discovery.

What this highlights is that what counts is not only conditional on what gets counted, but also who is doing the counting. On the surface, we’re talking about a prime site for data activism: research-oriented, quantitative questions in the domain of professional scientists. But when the answers come from people whose claim to authority undermines the credibility and the ego of those same scientists, they’re not taken seriously at all. Data alone works only if people are willing to listen: the rest of the time, you need not only data, but a bigger, more fundamental shift in the shape of how decisions are made, and by whom.

Power Routing

These examples might suggest I think data is overplayed as a source of resistance—at least, data alone. And, well: I do. But that’s not the same as saying it’s useless. Sometimes data can make a real difference: not by confronting power but by routing around it. That’s exactly what it does in my third example, which is about a far more personal question: where does one go for surgery?

Gender-affirming surgery in the US operated for most of the 1980s and 1990s largely through a small number of independent private surgeons. This created an information asymmetry, extreme even in comparison to the default power dynamic between a patient and a doctor. Surgeons tended to show off the best-case scenarios, and didn’t necessarily have direct access to patient experiences even if they wanted to. Precisely because of the small number of providers, most people had to travel hundreds or thousands of miles for care (and still do)—meaning they usually returned home as soon as possible, cutting into doctors’ awareness of post-operative experiences. Even with the most honest surgeon, their promises around healing time, cost, and outcome were ultimately a guess, and one that patients had to take at face value. If you were lucky enough to have access to community spaces with people who had received surgery, you might be able to get some anecdotal information about some surgeons, but that’s about it.

So in 1996, a trans woman named Michelle Wallace launched the Post Operative Transsexual Survey (POTS). Hosted on her website, which was part of the very 1990s webrings-and-Geocities-account network that made up a big portion of the early trans internet, the survey ran for around a year. In that time, it got 164 responses—more than most academic followup studies. And Wallace posted the aggregated results online, for anyone to see and use.

Unlike surveys by surgeons, which might ask vaguely and abstractly about quality of life, Wallace was positioned to ask more concretely about what concerned prospective patients, having been one herself. She asked obvious-seeming questions that were anything but obvious to academic researchers. Were the nurses caring? How quickly could you go back to work? If you had a question for the surgeon, did they make themselves available? The result was a community resource that people who were thinking about surgery could use to get a sense of what their options were, and make their choice based on what mattered most to them.

This guerilla data activism worked because it didn’t directly challenge the formal structures of power. Instead, it routed around it—it worked not against, but athwart. Which brings us back to the question of how data activism and abolitionism relate, and what we can reasonably expect of the former.

Vitally, the routing-around approach to data activism avoids a common tension between activism and abolition, in which activism tacitly reinforces the grounding of our understandings of rights, power, and injustice in data-based evidence, while abolition seeks to undermine it. After all: to argue that you should be taken seriously because of your data is to argue that data should be taken seriously. But a central component of POTS was who Michelle Wallace was addressing, and about what. She wasn’t addressing the state or doctors; she wasn’t seeking to ground claims for recognition in data. Instead, data served to inform people within the community it was about. As a result, one of the main tensions between activism and abolition was, if not avoided, then at least drastically reduced.

Obviously, in-community change isn’t enough: there needs to be some form of address outside it. But these stories suggest that, taken in isolation or in the abstract, data itself won’t provide that. Certainly data is a potential tool and technique in enacting social change. But like any other tool, it comes with costs, and is dependent on broader projects. In the absence of those broader projects, its main strength is in pooling community knowledge, not challenging structural power directly. If the goal is to mount a direct challenge, data activism might be a component of activism writ large. But it will only be effective if paired with the same humdrum, menial, and vital tactics—political campaigning, mass movements, media work—that were the bread and butter of social activism long before capital-d-and-s Data Science came along.