Does the Vaccine Matter? by Shannon Brownlee and Jeanne Lenzer

Health November 2009

Whether this season’s swine flu turns out to be deadly or mild, most experts agree that it’s only a matter of time before we’re hit by a truly devastating flu pandemic—one that might kill more people worldwide than have died of the plague and aids combined. In the U.S., the main lines of defense are pharmaceutical—vaccines and antiviral drugs to limit the spread of flu and prevent people from dying from it. Yet now some flu experts are challenging the medical orthodoxy and arguing that for those most in need of protection, flu shots and antiviral drugs may provide little to none. So where does that leave us if a bad pandemic strikes?

Does the Vaccine Matter?

by Shannon Brownlee and Jeanne Lenzer

http://www.theatlantic.com/doc/200911/brownlee-h1n1

Drive too fast along Red Lion Road, beside Philadelphia’s Northeast Airport, and you will miss the low-rise cement building where the biotech company MedImmune has been quietly pumping out swine flu vaccine at about a million doses a week. Through the summer and fall, workers wearing protective gear that covered them from head to toe brewed up batches of live, genetically modified flu virus. Robots then injected tiny doses of virus-laden fluid into glass vials, which were mounted into nasal spritzers, labeled, and readied for shipment at the direction of the Centers for Disease Control and Prevention, in Atlanta, which is helping to coordinate the nation’s pandemic-preparedness plan. In the most ambitious vaccination program the nation has mounted since the anti-polio campaign in the 1950s, the federal government has commissioned MedImmune and four other companies to produce enough vaccine to cover the entire U.S. population.

Vaccination is central to the government’s plan for preventing deaths from swine flu. The CDC has recommended that some 159 million adults and children receive either a swine flu shot or a dose of MedImmune’s nasal vaccine this year. Shots are offered in doctors’ offices, hospitals, airports, pharmacies, schools, polling places, shopping malls, and big-box stores like Wal-Mart. In August, New York state required all health-care workers to get both seasonal and swine flu shots. To further protect the populace, the federal government has spent upwards of $3billion stockpiling millions of doses of antiviral drugs like Tamiflu—which are being used both to prevent swine flu and to treat those who fall ill.

But what if everything we think we know about fighting influenza is wrong? What if flu vaccines do not protect people from dying—particularly the elderly, who account for 90 percent of deaths from seasonal flu? And what if the expensive antiviral drugs that the government has stockpiled over the past few years also have little, if any, power to reduce the number of people who die or are hospitalized? The U.S. government—with the support of leaders in the public-health and medical communities—has put its faith in the power of vaccines and antiviral drugs to limit the spread and lethality of swine flu. Other plans to contain the pandemic seem anemic by comparison. Yet some top flu researchers are deeply skeptical of both flu vaccines and antivirals. Like the engineers who warned for years about the levees of New Orleans, these experts caution that our defenses may be flawed, and quite possibly useless against a truly lethal flu. And that unless we are willing to ask fundamental questions about the science behind flu vaccines and antiviral drugs, we could find ourselves, in a bad epidemic, as helpless as the citizens of New Orleans during Hurricane Katrina.

The term influenza, which dates back to the Middle Ages, is taken from the Italian word for occult or astral influence. Then as now, flu seemed to appear out of nowhere each winter, debilitating or killing large numbers of people, only to vanish in the spring. Today, seasonal flu is estimated to kill about 36,000 people in the United States each year, and half a million worldwide.

Yet the flu, in many important respects, remains mysterious. Determining how many deaths it really causes, or even who has it, is no simple matter. We think we have the flu anytime we fall ill with an ailment that brings on headache, malaise, fever, coughing, sneezing, and that achy feeling as if we’ve been sleeping on a bed of rocks, but researchers have found that at most half, and perhaps as few as 7 or 8 percent, of such cases are actually caused by an influenza virus in any given year. More than 200 known viruses and other pathogens can cause the suite of symptoms known as “influenza-like illness”; respiratory syncytial virus, bocavirus, coronavirus, and rhinovirus are just a few of the bugs that can make a person feel rotten. And depending on the season, in up to two-thirds of the cases of flu-like illness, no cause at all can be found.

Nobody knows precisely why we are much more likely to catch the flu in the winter months than at other times of the year. Perhaps it’s because flu viruses flourish in cool temperatures and are killed by exposure to sunlight. Or maybe it’s because in winter, people spend more time indoors, where a sneeze or a cough can more easily spread a virus to others. What is certain is that influenza viruses mutate with amazing speed, so each flu season sees slightly different genetic versions of the viruses that infected people the year before. Every year, the World Health Organization and the Centers for Disease Control and Prevention collect data from 94 nations on the flu viruses that circulated the previous year, and then make an educated guess about which viruses are likely to circulate in the coming fall. Based on that information, the U.S. Food and Drug Administration issues orders to manufacturers in February for a vaccine that includes the three most likely strains.

Every once in a while, however, a very different bug pops up and infects far more people than the normal seasonal flu variants do. It is these novel viruses that are responsible for pandemics, defined by the World Health Organization as events that occur when “a new influenza virus appears against which the human population has no immunity” and which can sweep around the world in a very short time. The worst flu pandemic in recorded history was the “Spanish flu” of 1918–19, at the end of World WarI. A third of the world’s population was infected, with at least 40million and perhaps as many as 100million people dying—more than were killed in World Wars I and II combined. (Some scholars suggest that one reason World WarI ended was that so many soldiers were sick or dying from flu.) Since then, two other flu pandemics have occurred, in 1957 and 1968, neither of which was particularly lethal.

In August, the President’s Council of Advisors on Science and Technology projected that this fall and winter, the swine flu, H1N1, could infect anywhere between one-third and one-half of the U.S. population and could kill as many as 90,000 Americans, two and a half times the number killed in a typical flu season. But precisely how deadly, or even how infectious, this year’s H1N1 pandemic will turn out to be won’t be known until it’s over. Most reports coming from the Southern Hemisphere in late August (the end of winter there) suggested that the swine flu is highly infectious, but not particularly lethal. For example, Australian officials estimated they would finish winter with under 1,000 swine flu deaths—fewer than the usual 1,500 to 3,000 from seasonal flu. Among those who have died in the U.S., about 70 percent were already suffering from congenital conditions like cerebral palsy or underlying illnesses such as cancer, asthma, or AIDS, which make people more vulnerable.

Public-health officials consider vaccine their most formidable defense against the pandemic—indeed, against any flu—and on the surface, their faith seems justified. Vaccines developed over the course of the 20th century slashed the death rates of nearly a dozen infectious diseases, such as smallpox and polio, and vaccination became one of medicine’s most potent weapons. Influenza virus was first identified in the 1930s, and by the mid-1940s, researchers had produced a vaccine that was given to soldiers in World WarII. The U.S. government got serious about promoting flu vaccine after the 1957 flu pandemic brought home influenza’s continuing potential to cause widespread illness and death. Today, flu vaccine is a staple of public-health policy; in a normal year, some 100 million Americans get vaccinated.

But while vaccines for, say, whooping cough and polio clearly and dramatically reduced death rates from those diseases, the impact of flu vaccine has been harder to determine. Flu comes and goes with the seasons, and often it does not kill people directly, but rather contributes to death by making the body more susceptible to secondary infections like pneumonia or bronchitis. For this reason, researchers studying the impact of flu vaccination typically look at deaths from all causes during flu season, and compare the vaccinated and unvaccinated populations.

Such comparisons have shown a dramatic difference in mortality between these two groups: study after study has found that people who get a flu shot in the fall are about half as likely to die that winter—from any cause—as people who do not. Get your flu shot each year, the literature suggests, and you will dramatically reduce your chance of dying during flu season.

Yet in the view of several vaccine skeptics, this claim is suspicious on its face. Influenza causes only a small minority of all deaths in the U.S., even among senior citizens, and even after adding in the deaths to which flu might have contributed indirectly. When researchers from the National Institute of Allergy and Infectious Diseases included all deaths from illnesses that flu aggravates, like lung disease or chronic heart failure, they found that flu accounts for, at most, 10 percent of winter deaths among the elderly. So how could flu vaccine possibly reduce total deaths by half? Tom Jefferson, a physician based in Rome and the head of the Vaccines Field at the Cochrane Collaboration, a highly respected international network of researchers who appraise medical evidence, says: “For a vaccine to reduce mortality by 50 percent and up to 90 percent in some studies means it has to prevent deaths not just from influenza, but also from falls, fires, heart disease, strokes, and car accidents. That’s not a vaccine, that’s a miracle.”

The estimate of 50 percent mortality reduction is based on “cohort studies,” which compare death rates in large groups, or cohorts, of people who choose to be vaccinated, against death rates in groups who don’t. But people who choose to be vaccinated may differ in many important respects from people who go unvaccinated—and those differences can influence the chance of death during flu season. Education, lifestyle, income, and many other “confounding” factors can come into play, and as a result, cohort studies are notoriously prone to bias. When researchers crunch the numbers, they typically try to factor out variables that could bias the results, but, as Jefferson remarks, “you can adjust for the confounders you know about, not for the ones you don’t,” and researchers can’t always anticipate what factors are likely to be important to whether a patient dies from flu. There is always the chance that they might miss some critical confounder that renders their results entirely wrong.

When Lisa Jackson, a physician and senior investigator with the Group Health Research Center, in Seattle, began wondering aloud to colleagues if maybe something was amiss with the estimate of 50 percent mortality reduction for people who get flu vaccine, the response she got sounded more like doctrine than science. “People told me, ‘No good can come of [asking] this,’” she says. “‘Potentially a lot of bad could happen’ for me professionally by raising any criticism that might dissuade people from getting vaccinated, because of course, ‘We know that vaccine works.’ This was the prevailing wisdom.”

Nonetheless, in 2004, Jackson and three colleagues set out to determine whether the mortality difference between the vaccinated and the unvaccinated might be caused by a phenomenon known as the “healthy user effect.” They hypothesized that on average, people who get vaccinated are simply healthier than those who don’t, and thus less liable to die over the short term. People who don’t get vaccinated may be bedridden or otherwise too sick to go get a shot. They may also be more likely to succumb to flu or any other illness, because they are generally older and sicker. To test their thesis, Jackson and her colleagues combed through eight years of medical data on more than 72,000 people 65 and older. They looked at who got flu shots and who didn’t. Then they examined which group’s members were more likely to die of any cause when it was not flu season.

Jackson’s findings showed that outside of flu season, the baseline risk of death among people who did not get vaccinated was approximately 60 percent higher than among those who did, lending support to the hypothesis that on average, healthy people chose to get the vaccine, while the “frail elderly” didn’t or couldn’t. In fact, the healthy-user effect explained the entire benefit that other researchers were attributing to flu vaccine, suggesting that the vaccine itself might not reduce mortality at all. Jackson’s papers “are beautiful,” says Lone Simonsen, who is a professor of global health at George Washington University, in Washington, D.C., and an internationally recognized expert in influenza and vaccine epidemiology. “They are classic studies in epidemiology, they are so carefully done.”

The results were also so unexpected that many experts simply refused to believe them. Jackson’s papers were turned down for publication in the top-ranked medical journals. One flu expert who reviewed her studies for the Journal of the American Medical Association wrote, “To accept these results would be to say that the earth is flat!” When the papers were finally published in 2006, in the less prominent International Journal of Epidemiology, they were largely ignored by doctors and public-health officials. “The answer I got,” says Jackson, “was not the right answer.”

The history of flu vaccination suggests other reasons to doubt claims that it dramatically reduces mortality. In 2004, for example, vaccine production fell behind, causing a 40 percent drop in immunization rates. Yet mortality did not rise. In addition, vaccine “mismatches” occurred in 1968 and 1997: in both years, the vaccine that had been produced in the summer protected against one set of viruses, but come winter, a different set was circulating. In effect, nobody was vaccinated. Yet death rates from all causes, including flu and the various illnesses it can exacerbate, did not budge. Sumit Majumdar, a physician and researcher at the University of Alberta, in Canada, offers another historical observation: rising rates of vaccination of the elderly over the past two decades have not coincided with a lower overall mortality rate. In 1989, only 15 percent of people over age 65 in the U.S. and Canada were vaccinated against flu. Today, more than 65 percent are immunized. Yet death rates among the elderly during flu season have increased rather than decreased.

Vaccine proponents call Majumdar’s last observation an “ecological fallacy,” because he fails, in their view, to consider changes in the larger environment that could have boosted death rates over the years—even as rising vaccination rates were doing their part to keep mortality in check. The proponents suggest, for instance, that influenza viruses may have become more contagious over time, and thus are infecting greater numbers of elderly people, including some who have been vaccinated. Or maybe the viruses are becoming more lethal. Or maybe the elderly have less immunity to flu than they once did because, say, their diets have changed.

Or maybe vaccine just doesn’t prevent deaths in the elderly. Of course, that’s the one possibility that vaccine adherents won’t consider. Nancy Cox, the CDC’s influenza division chief, says flatly, “The flu vaccine is the best way to protect against flu.” Anthony Fauci, a physician and the director of the National Institute of Allergy and Infectious Diseases at the NIH, where much of the basic science of flu vaccine has been worked out, says, “I have no doubt that it is effective in conferring some degree of protection. To say otherwise is a minority view.”

Majumdar says, “We keep coming up against the belief that we’ve reduced mortality by 50 percent,” and when researchers poke holes in the evidence, “people pound the pulpit.”

The most vocal—and undoubtedly most vexing—critic of the gospel of flu vaccine is the Cochrane Collaboration’s Jefferson, who’s also an epidemiologist trained at the famed London School of Tropical Hygiene, and who, in Lisa Jackson’s view, makes other skeptics seem “moderate by comparison.” Among his fellow flu researchers, Jefferson’s outspokenness has made him something of a pariah. At a 2007 meeting on pandemic preparedness at a hotel in Bethesda, Maryland, Jefferson, who’d been invited to speak at the conference, was not greeted by any of the colleagues milling about the lobby. He ate his meals in the hotel restaurant alone, surrounded by scientists chatting amiably at other tables. He shrugs off such treatment. As a medical officer working for the United Nations in 1992, during the siege of Sarajevo, he and other peacekeepers were captured and held for more than a month by militiamen brandishing AK-47s and reeking of alcohol. Professional shunning seems trivial by comparison, he says.

“Tom Jefferson has taken a lot of heat just for saying, ‘Here’s the evidence: it’s not very good,’” says Majumdar. “The reaction has been so dogmatic and even hysterical that you’d think he was advocating stealing babies.” Yet while other flu researchers may not like what Jefferson has to say, they cannot ignore the fact that he knows the flu-vaccine literature better than anyone else on the planet. He leads an international team of researchers who have combed through hundreds of flu-vaccine studies. The vast majority of the studies were deeply flawed, says Jefferson. “Rubbish is not a scientific term, but I think it’s the term that applies.” Only four studies were properly designed to pin down the effectiveness of flu vaccine, he says, and two of those showed that it might be effective in certain groups of patients, such as school-age children with no underlying health issues like asthma. The other two showed equivocal results or no benefit.

Flu researchers have been fooled into thinking vaccine is more effective than the data suggest, in part, says Jefferson, by the imprecision of the statistics. The only way to know if someone has the flu—as opposed to influenza-like illness—is by putting a Q-tip into the patient’s throat or nose and running a test, which simply isn’t done that often. Likewise, nobody really has a handle on how many of the deaths that are blamed on flu were actually caused by a flu virus, because few are confirmed by a laboratory. “I used to be a family physician,” says Jefferson. “I’ve never seen a patient come to my office with H1N1 written on his forehead. When an old person dies of respiratory failure after an influenza-like illness, they nearly always get coded as influenza.”

There’s one other way flu researchers may be fooled into thinking flu vaccine is effective, Jefferson says. All vaccines work by delivering a dose of killed or weakened virus or bacteria, which provokes the immune system into producing antibodies. When the person is subsequently exposed to the real thing, the body is already prepared to repel the bug completely or to get rid of it after a mild illness. Flu researchers often use antibody response as a way of gauging the effectiveness of vaccine, on the assumption that levels of antibodies in the blood of people who have been vaccinated are a good predictor—although an imperfect one—of how well they can ward off the infection.

There’s some merit to this reasoning. Unfortunately, the very people who most need protection from the flu also have immune systems that are least likely to respond to vaccine. Studies show that young, healthy people mount a glorious immune response to seasonal flu vaccine, and their response reduces their chances of getting the flu and may lessen the severity of symptoms if they do get it. But they aren’t the people who die from seasonal flu. By contrast, the elderly, particularly those over age70, don’t have a good immune response to vaccine—and they’re the ones who account for most flu deaths. (Infants with severe disabilities, such as leukemia and congenital lung disease, and people who are immune-compromised—from AIDS, or diabetes, or cancer treatment—make up the rest. As of August8, only 36 deaths from swine flu had been confirmed among children in the U.S., and the overwhelming majority of those children had multiple, severe health disorders.)

In Jefferson’s view, this raises a troubling conundrum: Is vaccine necessary for those in whom it is effective, namely the young and healthy? Conversely, is it effective in those for whom it seems to be necessary, namely the old, the very young, and the infirm? These questions have led to the most controversial aspect of Jefferson’s work: his call for placebo-controlled trials, studies that would randomly give half the test subjects vaccine and the other half a dummy shot, or placebo. Only such large, well-constructed, randomized trials can show with any precision how effective vaccine really is, and for whom.

In the flu-vaccine world, Jefferson’s call for placebo-controlled studies is considered so radical that even some of his fellow skeptics oppose it. Majumdar, the Ottawa researcher, says he believes that evidence of a benefit among children is established and that public-health officials should try to protect seniors by immunizing children, health-care workers, and other people around them, and thus reduce the spread of the flu. Lone Simonsen explains the prevailing view: “It is considered unethical to do trials in populations that are recommended to have vaccine,” a stance that is shared by everybody from the CDC’s Nancy Cox to Anthony Fauci at the NIH. They feel strongly that vaccine has been shown to be effective and that a sham vaccine would put test subjects at unnecessary risk of getting a serious case of the flu. In a phone interview, Fauci at first voiced the opinion that a placebo trial in the elderly might be acceptable, but he called back later to retract his comment, saying that such a trial “would be unethical.” Jefferson finds this view almost exactly backward: “What do you do when you have uncertainty? You test,” he says. “We have built huge, population-based policies on the flimsiest of scientific evidence. The most unethical thing to do is to carry on business as usual.”

Just after 6 p.m. on a warm Friday evening in July, Dr. David Newman is only minutes into a 10-hour shift in the emergency room of New York City’s St. Luke’s Hospital, and already he has assumed responsibility for 11 patients. The young Italian tourist sitting on the bed in front of the doctor has meningitis, and through an interpreter, Newman tells him he almost certainly has the viral form of the disease, which will do nothing more than make him feel ill for a few days. There is a tiny chance, says Newman, that the illness is caused by a bacterium, which can be deadly, but he is almost positive that’s not what the tourist has. He says to his patient, “I can’t tell you with 100 percent certainty that you don’t have it, but if you do, you’ll begin to feel worse and you’ll need to come back.” The tourist, on learning that he might be infected with a potentially lethal disease, looks down at his feet and confesses that he is much more worried about another illness: swine flu. Newman smiles patiently. “It would be nice if you had swine flu,” he says. “Compared to bacterial meningitis, swine flu is safe.”

Late last spring, as headlines and airwaves warned of a possible pandemic, patients like Newman’s began clogging emergency rooms across the country, a sneezing, coughing, infectious tide of humanity more worried than truly sick, but whose mere presence in the emergency room has endangered the lives of others. “Studies show that when there is ER crowding, mortality goes up, because patients who need immediate attention don’t get it,” says Newman, the director of clinical research in the Department of Emergency Medicine at the hospital, which is affiliated with Columbia University. In an average year the ER at St. Luke’s, a sprawling 1,076-bed hospital on 113th Street, takes in 110,000 patients, some 300 a day. At the height of the summer swine flu outbreak, that number doubled. The vast majority of panicky patients who came in the door at St. Luke’s and other emergency departments didn’t actually have the virus, and of those who did, most were not sick enough to need hospitalization. Even so, says Newman, when patients with even mild flu symptoms show up in the hospital, they vastly increase the spread of the virus, simply because they inevitably sneeze and cough in rooms that are jammed with other people.

Many of the worried sick come to St. Luke’s and other hospitals in search of antiviral drugs. The CDC recommends the use of two drugs against H1N1: oseltamivir and zanamivir, better known by their brand names, Tamiflu and Relenza, which together form the second pillar of the government’s anti-pandemic-flu strategy. Public-health officials at the state and local levels are also recommending the drugs. Guidelines issued by the New York City Department of Health, says Newman, “encourage us to give a prescription to just about every patient with the sniffles,” a practice that some experts worry will quickly lead to resistant strains of the virus.

Indeed, that’s already happening. Daniel Janies, an associate professor of biomedical informatics at Ohio State University, tracks the genetic mutations that allow flu virus to develop resistance to drugs. Flu can become resistant to Tamiflu in a matter of days, he says. Handing out the drug early in the pandemic, when H1N1 poses only a minimal threat to the vast majority of patients, strikes him as “shortsighted.” Indeed, samples of resistant H1N1 were cropping up by midsummer, increasing the likelihood that come late fall, many people will be infected with a resistant strain of swine flu. Alarmed at that prospect, the World Health Organization issued an alert on August 21, recommending that Tamiflu and Relenza be used only in severe cases and in patients who are at high risk of serious complications. By mid-August, two U.S. swine flu patients had developed Tamiflu-resistant strains.

The U.S. first began stockpiling Tamiflu and Relenza back in 2005, in the wake of concern that an outbreak in Southeast Asia of bird flu, a far more deadly form of the disease, might go global. On November 1, 2005, President George W.Bush pronounced pandemic flu a “danger to our homeland,” and he asked Congress to approve legislation that included $1billion for the production and stockpiling of antivirals. This was after Congress had already approved $1.8billion to stockpile Tamiflu for the military, a decision that was made during the tenure of Defense Secretary Donald Rumsfeld. (Before joining the Bush Cabinet, Rumsfeld was chairman for four years of Gilead Sciences, the company that holds the patent on Tamiflu, and he held millions of dollars’ worth of stock in the company. According to Roll Call, an online newspaper covering events on Capitol Hill, Rumsfeld says he recused himself from all government decisions involving Tamiflu. Gilead’s stock price rose more than 50 percent in 2005, when the government’s plan was announced.)

As with vaccines, the scientific evidence for Tamiflu and Relenza is thin at best. In its general-information section, the CDC’s Web site tells readers that antiviral drugs can “make you feel better faster.” True, but not by much. On average, Tamiflu (which accounts for 85 to 90 percent of the flu antiviral-drug market) cuts the duration of flu symptoms by 24hours in otherwise healthy people. In exchange for a slightly shorter bout of illness, as many as one in five people taking Tamiflu will experience nausea and vomiting. About one in five children will have neuropsychiatric side effects, possibly including anxiety and suicidal behavior. In Japan, where Tamiflu is liberally prescribed, the drug may have been responsible for 50 deaths from cardiopulmonary arrest, from 2001 to 2007, according to Rokuro Hama, the chair of the Japan Institute of Pharmacovigilance.

Such side effects might be worth risking if the antivirals prevented serious complications of flu, such as pneumonia, hospitalization, and death. Roche Laboratories, the company licensed to manufacture and market Tamiflu, says its drug does just that. In two September2006 press releases, the company announced, “Tamiflu significantly reduces the risk of death from influenza: New data shows treatment was associated with more than a two third reduction in deaths,” and “Children with influenza [are] 53 percent less likely to contract pneumonia when treated with Tamiflu.” Once again cohort studies (the same kind of potentially biased research that led to the conclusion that flu vaccine cuts mortality by 50 percent) are behind these claims. Tamiflu costs $10 a pill. It is possible that people who take it are more likely to be insured and affluent, or at least middle-class, than those who do not, and a large body of evidence shows that the well-off nearly always fare better than the poor when stricken with an infectious disease, including flu. In both 2003 and 2009, reviews of randomized placebo-controlled studies found that the study populations simply weren’t large enough to answer the question: Does Tamiflu prevent pneumonia?

As late as this August, the company’s own Web site contained the following statement, which was written under the direction of the FDA: “Tamiflu has not been proven to have a positive impact on the potential consequences (such as hospitalizations, mortality, or economic impact) of seasonal, avian, or pandemic influenza.” An FDA spokesperson said recently that the agency is unaware of any data submitted by Roche that would support the claims in the company’s September 2006 news release about the drug’s reducing flu deaths.

Why, then, has the federal government stockpiled millions of doses of antivirals, at a cost of several billion dollars? And why are physicians being encouraged to hand out prescriptions to large numbers of people, without sound evidence that the drugs will help? The short answer may be that public-health officials feel they must offer something, and these drugs are the only possible remedies at hand. “I have to agree with the critics the antiviral question is not cut-and-dried,” says Fauci. “But [these drugs are] the best we have.” The CDC’s Nancy Cox also acknowledges that the science is not as sound as she might like, but the government still recommends their use. And as with vaccines, she considers additional randomized placebo-controlled trials of the antiviral drugs to be “unethical” and thus out of the question.

This is the curious state of debate about the government’s two main weapons in the fight against pandemic flu. At first, government officials declare that both vaccines and drugs are effective. When faced with contrary evidence, the adherents acknowledge that the science is not as crisp as they might wish. Then, in response to calls for placebo-controlled trials, which would provide clear results one way or the other, the proponents say such studies would deprive patients of vaccines and drugs that have already been deemed effective. “We can’t just let people die,” says Cox.

Students of U.S. medical history will find this circular logic familiar: it is a long-recurring theme in American medicine, and one that has, on occasion, had deadly consequences. In 1925, Sinclair Lewis caricatured a medical culture that allowed belief—and profits—to distort science in his Pulitzer Prize–winning book, Arrowsmith. Based on the lives of the real-life microbiologists Paul de Kruif and Jacques Loeb, Lewis tells the story of Martin Arrowsmith, a physician who invents a new vaccine during a deadly outbreak of bubonic plague. But his efforts to test the vaccine’s efficacy are frustrated by an angry community that desperately wants to believe the vaccine works, and a profit-hungry institute that rushes the vaccine into use prematurely—forever preempting the proper studies that are needed.

The annals of medicine are littered with treatments and tests that became medical doctrine on the slimmest of evidence, and were then declared sacrosanct and beyond scientific investigation. In the 1980s and ’90s, for example, cancer specialists were convinced that high-dose chemotherapy followed by a bone-marrow transplant was the best hope for women with advanced breast cancer, and many refused to enroll their patients in randomized clinical trials that were designed to test transplants against the standard—and far less toxic—therapy. The trials, they said, were unethical, because they knew transplants worked. When the studies were concluded, in 1999 and 2000, it turned out that bone-marrow transplants were killing patients. Another recent example involves drugs related to the analgesic lidocaine. In the 1970s, doctors noticed that the drugs seemed to make the heart beat rhythmically, and they began prescribing them to patients suffering from irregular heartbeats, assuming that restoring a proper rhythm would reduce the patient’s risk of dying. Prominent cardiologists for years opposed clinical trials of the drugs, saying it would be medical malpractice to withhold them from patients in a control group. The drugs were widely used for two decades, until a government-sponsored study showed in 1989 that patients who were prescribed the medicine were three and a half times as likely to die as those given a placebo.

Demonstrating the efficacy (or lack thereof) of vaccine and antivirals during flu season would not be hard to do, given the proper resources. Take a group of people who are at risk of getting the flu, and randomly assign half to get vaccine and the other half a dummy shot. Then count the people in each group who come down with flu, suffer serious illness, or die. (A similarly designed trial would suffice for the antivirals.) It might sound coldhearted, but it is the only way to know for certain whether, and for whom, current remedies actually work. It would also be useful to know whether vaccinating healthy people—who can mount an immune response on their own—protects the more vulnerable people around them. For example, immunizing nursing-home staff and healthy children is thought to reduce the spread of flu to the elderly and the immune-compromised. Pinning down the effectiveness of this strategy would be a bit more complex, but not impossible.

In the absence of such evidence, we are left with two possibilities. One is that flu vaccine is in fact highly beneficial, or at least helpful. Solid evidence to that effect would encourage more citizens—and particularly more health professionals—to get their shots and prevent the flu’s spread. As it stands, more than 50 percent of health-care workers say they do not intend to get vaccinated for swine flu and don’t routinely get their shots for seasonal flu, in part because many of them doubt the vaccines’ efficacy. The other possibility, of course, is that we’re relying heavily on vaccines and antivirals that simply don’t work, or don’t work as well as we believe. And as a result, we may be neglecting other, proven measures that could minimize the death rate during pandemics.

“Vaccines give us a false sense of security,” says Sumit Majumdar. “When you have a strategy that [everybody thinks] reduces death by 50 percent, it’s pretty hard to invest resources to come up with better remedies.” For instance, health departments in every state are responsible for submitting plans to the CDC for educating the public, in the event of a serious pandemic, about hand-washing and “social distancing” (voluntary quarantines, school closings, and even enforcement of mandatory quarantines to keep infected people in their homes). Putting these plans into action will require considerable coordination among government officials, the media, and health-care workers—and widespread buy-in from the public. Yet little discussion has appeared in the press to help people understand the measures they can take to best protect themselves during a flu outbreak—other than vaccination and antivirals.

“Launched early enough and continued long enough, social distancing can blunt the impact of a pandemic,” says Howard Markel, a pediatrician and historian of medicine at the University of Michigan. Washing hands diligently, avoiding public places during an outbreak, and having a supply of canned goods and water on hand are sound defenses, he says. Such steps could be highly effective in helping to slow the spread of the virus. In Mexico, for instance, where the first swine flu cases were identified in March, the government launched an aggressive program to get people to wash their hands and exhorted those who were sick to stay home and effectively quarantine themselves. In the United Kingdom, the national health department is promoting a “buddy” program, encouraging citizens to find a friend or neighbor willing to deliver food and medicine so people who fall ill can stay home.

In the U.S., by contrast, our reliance on vaccination may have the opposite effect: breeding feelings of invulnerability, and leading some people to ignore simple measures like better-than-normal hygiene, staying away from those who are sick, and staying home when they feel ill. Likewise, our encouragement of early treatment with antiviral drugs will likely lead many people to show up at the hospital at first sniffle. “There’s no worse place to go than the hospital during flu season,” says Majumdar. Those who don’t have the flu are more likely to catch it there, and those who do will spread it around, he says. “But we don’t tell people this.”

All of which leaves open the question of what people should do when faced with a decision about whether to get themselves and their families vaccinated. There is little immediate danger from getting a seasonal flu shot, aside from a sore arm and mild flu-like symptoms. The safety of the swine flu vaccine remains to be seen. In the absence of better evidence, vaccines and antivirals must be viewed as only partial and uncertain defenses against the flu. And they may be mere talismans. By being afraid to do the proper studies now, we may be condemning ourselves to using treatments based on illusion and faith rather than sound science.