The following table provides a summary of the H1N1 vaccine forecasts and actual availability:
|Date||Forecast (million doses)||Actual|
|7/30/09||120 by October.|
|9/12/09||50 by Oct. 15,|
|9/26/09||40 by mid October|
|10/17/09||28-30 by the end of October.||11.4|
|10/26/09||30 by the end of October|
Why the error in their forecasts? My favorite quote (NY Times Oct 25, 2009) is
Dr. Thomas R. Frieden, the director of the Centers for Disease Control and Prevention, “We really thought that having five different manufacturers would buy us some insurance, that they wouldn’t all have problems.”
The implicit assumption in this quote is that yields would be independent across flu manufacturers. This may be true if yield primarily depends on manufacturing choices that could differ across the 5 manufacturers. But the manufacturers all used the same seed stock (the virus injected into eggs that is suppose to replicate within the eggs to make the vaccine). Consequently, if the seed stock isn’t very good, it doesn’t matter if you have one manufacturer or 100! If you are going to diversity your risk by choosing multiple suppliers, then you should make sure that their yields are uncorrelated. (On a related point, there was no shortage of nasal spray vaccine, because they had a much higher yield … and used a different seed stock.)
There are some other reasons for the errors. In July the forecast was based on an assumption of a high yield. However, the actual yield would not be known until the first batches were completed in August. Therefore, it seemed premature to put any faith in the July forecast without better information.
So why were the September forecasts wrong? The government basically asked the manufacturers’ for their forecasts, and not surprisingly, they got optimistic numbers. I am not saying the manufacturers lied. Instead, they probably thought it was possible that they could improve yields quickly enough to make the numbers. For example Sanofi Pasteur’s initial yield was 1.5 doses per egg and they did manage to increase it to 3 doses per egg.
Some of the shortfall was due to shortages in parts. For example, MedImmune, maker of the nasal vaccine, had more vaccine than it could put into nasal sprayers (their yield was apparently 5 times higher than expected), despite having their supplier of nasal sprayers work 3 shifts, 7 days a week.
Finally, one of the manufacturers, from Australia, satisfied Australian demand first.
The lesson from all of this, dare I say, is “don’t count your chickens before they have hatched”! Production yields are uncertain, and the government could have benefited from learning more about what determines those yields, when information would be learned about those yields and relying less on manufacturers’ forecasts.