The Institute for Health Metrics and Evaluation (IHME), based at the University of Washington in Seattle, caused considerable alarm on 7 April when it released projections of Covid-19 deaths which predicted total deaths for the UK would be the highest in Europe at 66,314, and higher than their projected total deaths of 60,415 for the USA. According to the results on their webpage at https://covid19.healthdata.org/, daily deaths for USA would peak at 2200 in the next few days and start declining from 12 April. In contrast, the UK daily deaths continue to rise almost linearly for the next 12 days from 623 per day to 2900 per day. The curve then flattens at around 3000 deaths per day for a while before declining back to zero in June, giving total deaths of 66,314.

According to the Guardian newspaper: “The 66,000 figure was disputed by scientists whose modelling of the likely shape of the UK epidemic is relied on by the government. Prof Neil Ferguson, of Imperial College London, said last week when the prediction was published that the IHME figures were twice as high as they should have been.”

Three days later, IHME revised the UK projection downwards to around 37,000 deaths by end of July. Despite this lower figure, the UK would still have the highest death toll in Europe. The IHME website says this revision is due to the inclusion of four more days of data as input to their projection model. However, the very different projections for the UK from those for the USA and other European countries did not seem plausible to me, or explicable as due to different social distancing policies (the only predictive variable included in the IHME model).

So I have tested their projections over a short time period of days against subsequent reality. On April 11 I recorded their projections from the last data point for 9 April through to April 18. And today, I downloaded cumulative deaths from the  Johns Hopkins Covid-19 site and calculated deaths per day for Italy, Switzerland, UK and USA. The graph below shows the reported deaths for these countries as solid lines, and the IHME projected deaths from 9 April as dashed lines. I have to conclude their projection model is producing seriously bizarre results.

Reported deaths to 13 April are shown as solid curves. The IHME projected deaths from 9 April to 18 April are shown as dotted curves.

Today I also took another look at their latest projections on their website and they have changed quite substantially again. Now the UK deaths peak yesterday 13 April and start declining from now on, leading to an eventual total deaths of 23,791. The Swiss deaths per day, which have been plateaued for about a week with some signs they may be starting to decline, are projected to start rising to more than double the current number and then start declining from May 7. This is despite Switzerland implementing social distancing rules earlier than the UK and USA.

The following plot compares the government policy responses to COVID-19 for these four countries using the OxCGRT Stringency Index. The IHME also uses an index based on four policy indicators as a predictive variable in its model, and assumes that all countries reach maximum stringency one week after the last input data point. So I can’t see how this variable would create such large differences in projections.

The OxCGRT Stringency Index combines information of nine indicators of government response (school closures, travel bans, shop closures, etc) into a single index on a scale of 0 to 100 (maximum stringency).

The IHME projection model is based on fitting a curve to the cumulative deaths time series with the form shown in the figure below, which results in a symmetrical curve for daily deaths. This means that the fitted curves will tend to have faster declines for countries with faster rising death rates. I can see no reason to think that is what happens in reality.

The IHME projection model is based on fitting a curve to the cumulative deaths time series with the form shown on the left. The daily deaths are a symmetrical curve with shape d = exp(-α*t*t), where t=0 at the peak of the curve.

I checked the projections on the website today, and indeed for three of the countries, the number of days between the peak and one third of the peak deaths is similar for before and after the peak: Switzerland (before 26 days,  after 24 day),  UK (13,13) and Italy (13, 17). The USA is quite different with 14 days to peak and 29 days to reach 1/3 of peak afterwards. I conclude that despite the sophisticated Bayesian curve fitting used, the model appears to be fundamentally inappropriate for Covid-19 projections.