"Compared to what?"

This post, I need to say right off the bat, could not have been written, nay conceived, without the expertise of the CDC's medical epidemiologist Dr. Bill Atkinson, who as you might know, has been a force for public health for over 20 years. Recently, Bill, Maria Walters, and the Skepchicks launched a campaign to vaccinate folks against pertussis at Dragon*Con, and we fully expect that with more lead time next year, their efforts to vaccine the hell out of Nerdlanta will be victorious and complete! I met Bill at the vaccination clinic, and he generously agreed to come to my classes and talk about alternative medicine, especially the anti-vaccine movement and how to evaluate a clinical trial.

What he revealed was a stunning look at how vaccines are tested.

Bill discussed a clinical trial for the shingles vaccine. Shingles is caused by a nasty virus, herpes zoster, that hides out in the nerves of those of you who have had chickenpox. When it comes out to play, it can result in painful, persistent blistering. (Warning, this is gross.)

Thus Spake Zoster.

I don't want this. This disease, as far as I can tell, dogs the elderly, but can also appear in young people. I remember a friend in grade school (8th grade) who was kept home in misery because of shingles under his arm. It occurs to me that he could still be having outbreaks of this. Yikes!

In the last couple of years, a vaccine for shingles has been developed, and a clinical study...no, let's call it a "science party"... was thrown where two types of party favors were given out. The first was a placebo, the second, experimental vaccine. This was one hell of a party, including 38,546 patients over the age of 60, all of them "doing shots", as it were. The "science party" raged continuously between November 1998 and September 2001 at 22 different academic centers around the country.

It was all fun and games, however, until Bill gave us a sobering number. Of the 19,270 people who received the vaccine, 793 of the patients died.

Shingles vaccine! Run away!

Before you personally wet yourself over this (WARNING: Do NOT follow link), Brave Sir Robin, Bill asked, what else would you like to know?

"What did they die from?"

"Were they sick ahead of time?"

"What about the placebo group?"

Bang. Bill brought up a slide. 795 people in the placebo group died versus 793 in the vaccine group. No difference.

This clever reveal made a crucial, fundamental point not only about assessing vaccine efficacy, but also about understanding randomized studies in general:

When you say, "My grandpa had the experimental shingles vaccine and then he died," it tells you nothing about the vaccine.

When you document 793 people dying after receiving an experimental vaccine, it tells you nothing about the vaccine.

If you tell me that 1,000,000 children showed signs of autism in a global trial after getting a vaccine, you have told me nothing about the vaccine. When you say it outside of a clinical trial, you have told me...even less.

Your numbers, no matter how high or big or scary, are not only meaningless, but, to the layman like me, potentially dangerously misleading. Any meaningful interpretation of adverse outcomes in these types of trials depends on the researcher, layman, or journalist asking, "Compared to what?"

In fact, the shingles vaccine trial is a damned good trial, and should probably get its own book in the Bible of Medical Science ("...A reading from the Letter of the Shingles Prevention Study Group to the FDA").

You can check out the abstract of the study here. A flowchart that describes the study and its subsequent adverse events (different from "adverse reactions," mind you) study can be found here.

HJ