On March 18, the Centers for Disease Control and Prevention put out early data suggesting Covid-19 hits younger people more often than had been previously believed. Deep in the report it became clear the data were incomplete and researchers didn’t know the ages of 42% of those infected.

Early Studies, Big Impact
And then there was the worldwide firestorm over hydroxychloroquine, fanned by President Donald Trump’s tweets and public backing. It was ignited, in large part, by a small French study whose methods, including the lack of a good control group, have since been widely criticized by the medical and research community. The publisher and medical society that run the International Journal of Antimicrobial Agents have commissioned an independent review to determine whether concerns about the study have merit, according to an April 11 statement.

“There are hundreds of articles out there, but the quality is quite poor” in many, said Lauren Westafer, an emergency-room doctor and assistant professor at the University of Massachusetts Medical School in Baystate, speaking broadly about falling standards during the pandemic. “It’s problematic to make conclusions based on that kind of data,” added Westafer, who has a podcast about dissecting medical science. “The methods and quality are so deeply flawed.”

The acceleration of medical data is partly thanks to the rise of preprint services such as bioRxiv and medRxiv. The two sites often post drafts that haven’t been reviewed by other scientists or published in medical journals.

MedRxiv, launched less than a year ago, has been particularly inundated with coronavirus reports. While 225 papers were posted in January, when the virus was just emerging in China, the April total hit 1,500 — 1,000 of which are related to the virus, said John Inglis, executive director of Cold Spring Harbor Laboratory Press, which started and manages the two archives.

The websites were developed originally to help researchers share their preliminary work and help refine it before publication. But since the coronavirus, the general public is increasingly digesting the information, often after it is tweeted out on social media or highlighted in media reports.

“We are acutely aware of the fact that there are many kinds of audiences now for what is appearing on our preprint services, and that is a new phenomenon,” Inglis said. “Most of the time, the audience for this preprint is a professional one, but the epidemic has opened them up to the public gaze.”

The organization has tried to emphasize that the findings are preliminary and shouldn’t be considered a guide to any type of medical care. Still, it’s impossible to know what impact they are having.

“What we haven’t yet developed is an infrastructure that is equipped to deal with this volume,” Inglis said. “Here is this massive information, and people need guidance on what is important and reliable and what is well done and what isn’t well done,” he said. “Everyone is struggling to handle the volume.”

Attracting Critics
Even when research is careful and well vetted, the frenzy to get out new results can trip up researchers. Before it ran in the Journal of the American Medical Association, Northwell Health’s patient study went through multiple revisions prior to publication. One goal of the study was to report a series of unusual symptoms that could help other hospitals better identity Covid-19 cases. The Northwell researchers worked tirelessly, in some cases holding conference calls at 2 a.m, the only time overworked doctors were available. One key finding: Only 31% of its Covid-19 patients had a fever at the time they were admitted.