Vaccination: a critique (sure to be uncontroversial)

http://healthfreedoms.org/2011/10/14/big-study-vaccinated-kids-2-5-more-diseases-than-unvaccinated/

This little gem popped up in my facebook a little while ago. I have found that people are frequently taken in by stories like this and so I'm going to offer a critique of this "study" below with the hope that in doing so can shed light some of the ways in which deeply flawed research can be presented as valid.

There are a few things that come to mind respect to this study (which is actually just a presentation of survey results and not a study in any scientifically valid sense). I believe that the linked article has understated the number of caveats required to place it in proper perspective. But first, let me preface by saying that I don't care if you vaccinate your children. I ain't judging anyone. The status quo being what it is, my children are more at risk from auto accidents than they will likely ever be from dying from an illness transmitted from an unvaccinated child (though one of my kids did contract whooping cough from an unvaccinated friend). I don't think we have to hate each other over something like this.

First thing, the sampling process in this study is flawed. By sampling process, I mean the way in which they went about getting their data. The data from from the unvaccinated group comes from an online survey from an antivax website that anyone can fill out as many times as they like. I filled one out, feel free to do the same. When you do, recognize that you are now part of the sample. In the context of political polls, these types of things are usually footnoted with the phrase "not scientific, for entertainment purposes only" for good reason: there aren't really rigid controls in place to validate the data. In science, unvalidatable data is frequently worse than worthless, it's a waste of time, money, and effort.

Second, if you set aside data validity problems for a moment, consider the fact that this isn't a randomized study. If it were randomized, my next point would be irrelevant. Lack of randomization isn't always a problem. You can adjust your statistical models to account for the types of bias that a lack of randomization can induce. However, these results aren't adjusted at all.

How can a lack of randomization induce bias? Imagine you have two groups of people and you want to use them to study the effects of a certain supplement on nose hair growth (or, you know, something else completely unrelated to asthma). In the first group 90% of members have asthma. In the second group, 10% do. Now, say you give the first group the supplement and the second group gets a nothing. Regardless of any effects on nose hair growth, any unadjusted analysis of your groups will find an association between the supplement and asthma, even though there is no association. You will look at your results and say "gee, I don't know if nose hair growth is all that important here, because Fuck! Asthma!" Your failure to account for the discrepancy between asthma rates in your two groups has led you to a false conclusion. Randomization, if properly implemented, would avoid this problem because you would very likely end up with approximately equal proportions of people with asthma in each group. You could also use various analytical techniques to adjust for the effects of differing characteristics between your groups, which is how things are typically done with observational studies where randomization isn't feasible.

Why does that matter here? Because there doesn't seem to be any effort made in the interpretation of these results (beyond, perhaps, a dismissive remark) to account for the differing rates of underlying disease in vaccinated vs unvaccinated populations. Now, it could be that there is no difference. I'm open to that. But I think that it is more likely that the types of people who don't vaccinate aren't a representative sample of the general population. I don't have the information, so this next statement is all me, but it seems like non-vaccinating groups tend to skew middle class, college or self educated, and exceedingly health conscious. All of these things are likely negatively correlated with disease, so that if you belong to any of these groups, you (and your children) are less likely to get sick than if you didn't.

The sample is very likely biased and no efforts were made to account for disease incidence in underlying populations. With the huge caveat that the sampled data is valid, it is entirely possible that that these results show differences in disease rates between one group, the subpopulation of people who both choose not to vaccinate and frequent antivax websites, and another group, the general population, but that this difference has absolutely nothing to do with vaccination. Which is unfortunate, because if this study was done right (maybe it has been elsewhere) the results could be really interesting (though I suspect maybe not something you'd ever see endorsed on an antivax website, but that's my personal bias).

How do you feel about vaccination? Are you vaccinated? Are your kids? My kids are vaccinated, as am I. I think that there is likely more risk in the act of driving a child to a doctor's office than there is in having that child vaccinated. This doesn't mean that I think it impossible that some children might be predisposed to have bad reactions to certain vaccines. On the other hand, I also recognize that the current state of modern medicine leaves a bit to be desired with respect to where money is spent (R&D vs marketing), the regulatory process (for instance, the FDA getting a large amount of funding via fees from the companies whose products they regulate), and the fine line public health officials must draw between motivational honesty and scaremongering. In other words, I think that it is appropriate to view all health information with a bit of skepticism, however, I think that antivaccination positions are generally reflexive, overly reliant on appeals to emotion and driven in large part by people who have an economic interest in selling alternative medical treatments. Thoughts?

 

Yes, absolutely yes. It's asinine.

 

I think that it's puzzling too, but then again, I think vaccines are useful and have saved a ton of lives and improved the quality of life of a lot of people too.

I think that the idea that disease is more prevalent in the vaccinated plays to the general notion amongst these folks that vaccinations are more harmful than good. I think many antivax folks discarded the notion that vaccines can be beneficial a long time ago (any antivax people around feel free to correct me on this).

As a side note, I don't want this thread to become contaminated with contempt. I don't think that it has, but this is the type of subject where both sides tend to let passions override mutual respect so I just want to reiterate that I ain't hating on anyone and I'd appreciate it if no one else partook in hateration.

 

Funny story about that. A recent US measles outbreak was caused by an unvaccinated kid bringing it back from an African trip with his/her parents.

 

I love my two little vaccinations! I'm referring to the two smallpox vaccinations I have. Even as a kid I thought they looked cool; sort of like two little meteorite craters on my upper arm. I was born in Europe and came to the U.S. in early 1950's and I believe entry was not allowed unless you had smallpox vaccinations. Later in life all the kids got polio vaccines, and then measles and I think mumps and chickenpox, too.

My three children also got all the vaccinations recommended by mainstream, conventional medical advice.

As I got older I came to view medical "science" as partly an art with plenty of unknowns and differences of opinion even between respected doctors and scientists. So that lead to my healthy skepticism about medical advice...I find it sort of like my approach to the "news": I find out about things from multiple sources and perspectives before forming my own opinion or deciding what to do.

I find that most of what I read about the antivax positions sound closed-minded, fear-driven to me. On a related note, I think that the way pharmaceutical companies manipulate "the system" for their profit motives seriously compromises their credibility.

 

This pretty much sums up what I think about people who are antivax. I have a friend whose brother is an engineer and their child? (it might be children) are not vaccinated - anecdotal evidence from my friend suggests their child gets sick a *lot*. This is obviously not a large statistical base , but it is amazing how quickly we jump to conclusions based on the tiny sample size that we have access to. My friend keeps saying to me that her brother "should know better" - she think that he has swallowed the antivax paranoia.

As Baraka said, the diseases like Small pox, Tuberculosis etc. that have almost been completely eradicated as a big tick in my book for vaccinations.

Put me firmly into the "my kids are vaccinated" camp.

 

I no longer vaccinate my dogs on veterinary advice. Mulan girl became ill after her first and had her first fit. There is something about her breed that tend to make them more likely to have adverse reactions, vet called it a bit of a sensitive liver and said its best to do no more, she is covered anyway. Its a bit different with dogs - you wouldnt go back ever year for a smallpox jab - overkill, but insurance companies are strict on yearly vaccinations. Talking to the owner of the practice - our fave vet - he said there has been one proven death of a dog from vaccine - and by proven he meant with a full post mortem. It was the carrying agent that caused the animals death. Knowing what he knows, he does vaccinate his children.
Of course collected stats are often subject to manipulation, just as proper studies can be subjected to being buried - Dr David Kelley springs to mind. When are the next elections....?

 

I agree on both points. There are likely people who work at pharmaceutical companies who care about improving the general welfare of humanity, but I suspect that they aren't allowed to make financial decisions. This is because they'd probably be a whole lot less willing to spend ridiculously large amounts of money trying to get doctors to prescribe pills that may or may not be much better than generic alternatives or trying to get congress to rejigger laws for the sole benefit of pharmaceutical companies.

 

I am vaccinated and will be checking with my doctor at my upcoming physical about making sure I'm up to date on everything recommended by the CDC plus Gardasil and the pneumonia vaccine. Everyone should be vaccinated as recommended unless they have an allergy or a condition that contrainidcates vaccination. All children who are able to be safely vaccinated should be, no exceptions. Loss of herd immunity is coming back to bite even first world countries in the ass.

 

As much as I enjoy TFP's standard of mutual respect at some point a line must be drawn where educated people stand up and say "No, that does not mean your ignorance is just as good as my facts". I cannot in good conscience pay even lipservice to the idea that vaccinations are the true source of Autism/AIDS/Ingrown Hairs, and even if it WERE so I think so far we're doing quite better than when we had smallpox and polio tearing around the populace.

But playing devil's advocate here for a moment, lets look at that last line and do some grisly arithmetic: Are we better off with the antivaxer's list of side effects, or with unchecked versions of the previously vaccinated diseases going around? Especially when you consider the speed of modern travel and increasingly crowded cities.

 

I think that the point in maintaining mutual respect is that you can't convince someone of anything if you engage with them via mutual animosity. I know vaccine skeptics and I know capital S Skeptics. I'm pretty sure that the people who are genuinely concerned about vaccine safety (at least, the ones who aren't trying to derive emotional or physical currency off of vaccine skepticism) are just people with a healthy, but possibly misdirected, mistrust of the medical establishment who really just want what's best for their children. As a parent, I can relate to this; they just want to make the best decision with regards to their child's health.

If I didn't know a thing about public health research, if I had had bad experiences with doctors with arrogance disproportionate to their abilities (I have) I would probably question the usefulness of vaccines too.