Unsafe at This Particular Speed
Some of the ways that coronavirus science—doing it, interpreting it, acting on it—is remarkably complicated.
Thanks for reading. Black Lives Matter.
Gonna go ahead and break some ground here by declaring that science, generally, is hard. Science during a fast-moving pandemic that is focused on trying to understand and stop that pandemic has proven itself to be really hard.
Because there has never been a time when every new study gets so highly and immediately publicized, often by writers and television personalities who probably don’t spend much time poring over the methods section of papers, it seems worth a look at some of the reasons why almost every aspect of coronavirus research is, well, complicated. Here are a few:
Masks work
In recent days, several studies have emerged suggesting that mask mandates are extremely effective at slowing the virus’s spread. These use a few different methods: some involve computer modeling where assumptions like how much “exhaled virus inoculum” is actually captured are required; others, though, compare various locations infection rates based on when a mask mandate was implemented.
Take one study conducted in Germany: researchers analyzed different regions based on when they implemented a requirement to wear a mask, and found that “face masks reduce the daily growth rate of reported infections by around 40%.” Pretty good! But I would argue that what they actually mean is that a mask mandate results in that reduction. My main quibble is that the implementation of such a requirement might have ancillary effects on behavior: maybe hearing that the government is requiring mask use helps the direness of the situation sink in, or maybe people on the street unconsciously keep even more distance between each other when the mandate is in effect, and so on. These are almost impossible to control for.
The Germany study’s control group is another area of the country that had a similar rate of infection to the main area (the city of Jena) before mask requirements, and then did not implement the mandate when Jena did. A reasonable method, but human behavior has so many variables to it that making hard-number claims about the results of something like this seems remarkably shaky to me.
All that said, every study—modeling, retrospective, and others—has found that masks do help bring the virus’s transmission rate down dramatically. The accumulated evidence is clear—wear a mask—but the details are probably a lot less firm than they might sound.
Risk factor studies
You might have heard some of the conflicting reports about risk factors—is obesity the biggest factor for severe COVID cases, or is it just age, or do things like diabetes and other underlying conditions play a role? This stuff can be really hard to tease out.
A fascinating study conducted in Italy and Spain found that people with type-A blood had an almost 50 percent higher risk of a severe case of COVID-19 requiring ventilation or oxygen supplementation, while those with type-O blood had a much lower risk. The study used what’s known as a case-control design, where people with the virus were compared against a group from the general population, to look for genetic variations that were associated with such severe cases. The gene for type-A blood was the most stark finding.
But who exactly are the “controls” in this study? They were taken from a pool of voluntary blood donors in both countries. Sounds reasonable, especially when trying to conduct studies like this at the pace that the pandemic requires—blood is readily available and you don’t need to recruit anyone else to get the study going. But the authors don’t seem to account for what’s known as the Healthy Donor Effect: essentially, the concept that people who donate blood tend to be healthier than the general population.
The HDE has been found in multiple studies. In one, blood donors had better self-reported health, saw both general practitioners and specialist doctors less often, and had healthier lifetsyles than the general population. In another, blood donors who stopped donating were more likely to have had a disease diagnosis, increasing prescription medication use, and to have consulted a doctor. This was the case even with simple adjustments for advancing age, among other things.
This sort of thing induces what’s known as selection bias into studies involving blood donors—the “general population” isn’t actually reflective of the general population. With the blood type study, this isn’t necessarily a problem—though if the thing you’re looking at is severe cases of an illness that other research suggests is worse in less healthy people, it might play a role. It definitely muddies the waters a bit, and at very least calls the final number (a 45 percent increase in risk of respiratory failure with the gene for type-A blood) into question.
Vaccines in a Disappearing Pandemic
There are currently more than 135 potential coronavirus vaccines in development. Because we’d all really like to be able to sit down at a restaurant and not spend the entire meal more or less terrified, it would be nice if one of those finished big phase III trials testing its efficacy and started rolling out to the public very soon. But we might not have enough pandemic to test them all that well.
This is, in a certain way, a really good problem to have. The places where vaccine research is farthest ahead—the U.K., the U.S., and China—may start seeing the effect of lockdowns and social distancing (and masks!) in a way that makes testing a vaccine difficult. You need enough sick people in a given area to be able to tease out whether those who got the vaccine were less likely to become infected; if no one (more or less) is becoming infected, than you have no idea what the vaccine actually did. One researcher at Oxford, where a team is farther ahead on vaccine work than anyone else, said they are “in a race against the virus disappearing.” Weird race!
Of course, big chunks of the U.S. have graciously decided to try and fix this problem by opening too early and restarting high transmission levels, so maybe this issue will just sort of work itself out.
An Assumption of Good Faith
There have always been some missteps in scientific and medical publishing. But generally, the slow pace of that world does a reasonable job of catching them; papers are submitted to journals, peer-reviewed by relevant experts, and eventually published. But over the last few months, far more studies are being released and heavily reported on that haven’t really gone through that process, or at least have undergone a heavily accelerated version—we see “preprints” of studies, “accepted manuscripts” that haven’t been fully reviewed, and so on.
Which gives us fuck-ups like Surgisphere.
Remember in late May, when a big study from The Lancet reported significantly increased rates of death and heart problems in COVID-19 patients who received hydroxychloroquine, the president’s favorite anti-malarial? Well, that was based on data from Surgisphere, an American company “whose handful of employees appear to include a science fiction writer and an adult-content model.” They supposedly had data from almost 700 hospitals around the world, encompassing almost 100,000 patients. They, uh, didn’t really have that data, and before long both The Lancet and the New England Journal of Medicine had retracted studies based on Surgisphere data, and other journals were examining previous papers and their processes more generally.
(Worth noting: the New England Journal recently published a completely separate, non-Surgisphere-related study that found hydorxychloroquine did shit-all. So.)
To be clear, Surgisphere seems to have been perpetuating its little slice of fraud before the coronavirus crisis, but the sheer speed of this process—from study initiation to publication to retraction—lays bare how difficult conducting complicated epidemiological and biomedical research is when your timeline is condensed into a pandemic-appropriate window.
Some of these difficulties with COVID-related research are about science in general, and some are about this particular virus being kind of an asshole. Its incubation period makes transmission easy and studying it hard; its fatality rate will probably stay unknown for a long time still, and that’s before we get into all the ways it is killing people via side-eye.
The scientists working on this are more or less rebuilding a well-established world on the fly. And to be clear, they’re mostly doing an incredible job—we know an astonishing amount about the virus over a very short period of time, and a period that included mandates for pretty much everyone to stay at home. We might well have a vaccine within around a year from when those efforts began—the previous record, for Ebola, was about five years, and on average vaccine development has taken ten years. I’m not criticizing, not really; but not every study says what we want it to say.
[random bits]
Find me: