TietoEVRY's corona epidemic prediction accuracy is top notch

When the Finnish government decided on exceptional circumstances in March, TietoEVRY's advanced healthcare cloud analysis team began to develop scenarios on the subject. The aim was to make a realistic forecast of the duration of the epidemic and the number of infections.

Now, more than two months have passed since the scenarios prepared in March were created, it is a good time to compare the different scenarios and reality that has unfolded. In Finland, the corona epidemic has developed and been contained more positively than expected.

The following time scales are included in the table:

1. The three forecasts prepared at the end of March by TietoEVRY and made available to the company's own and cloud analytics customers (1.4. onwards)
2. Three estimates of current infections in Finland: 2x (“Lower limit”), 5x (“Intermediate”) and 10x (“Upper limit”) based on the number of detected infections
   1. At present, it is not known exactly what the actual infection number is, but it has been estimated that the infection number (FI) would be about 2-10 times the number of confirmed infections.
3. The three scenarios defined in the memorandum guiding the Finnish Government's decision on exceptional circumstances created 16 March 2020 (FI) and the two scenarios outlined in the memorandum and informing the decision made on 25 March 2020 (FI) on the closure of Uusimaa.

These projections include the effects of emergency slowdown measures. In the table, we see how the predictions determined by TietoEVRY's logarithmic scale follow quite closely the current estimates of the total number of infections.

How accurate were the forecasts?

It has been said that making an accurate prediction is simple, but not easy; achieving good forecast accuracy is rarely based on a superior forecast model.

Therefore, to manage uncertainty and to understand the phenomenon, it is essential to understand exactly what assumptions are being entered into the model and if these need to be changed during the forecast period. It is often the case that poor forecast accuracy can be attributed to inaccurate assumptions being entered into the model at the early stages and not accounting for changes over the course of the forecast period.

When analyzing the forecasting errors in the forecast models used in the Finnish government’s memorandum, it has been noted that these are not due to the epidemic models themselves, which were updated at different points during the spring, but precisely because of the assumptions themselves. In practice, people’s behaviour changed more than the scenarios had assumed which accounts for the discrepancy between predicted and detected infections.

Numerous ready-made models for modelling epidemics already exist and we can even speak of model libraries. For those without extensive experience in epidemiology or education in epidemic modelling, it would be questionable to try and develop a new way to model the epidemic itself.

For the first modelling of the epidemic, TietoEVRY followed the well-known SEIR model based on fairly simple differential equations. Because the model includes an interest-to-interest type exponential function, it must be carefully assessed whether the assumptions of that basic reproduction number last for the duration of the simulation period. At the same time, the co-efficients of the differential equation are no longer constant.

Do the assumptions last?

The determining factor in predicting accuracy in this case was not the use of the SEIR model over another model, but whether the parameters of the model used were applicable and realistic over time. To achieve high prediction accuracy, the most important element is to describe the assumptions correctly, that is, to predict the parameters of the model over time.

In practical epidemic situations, accuracy is determined by correctly predicting the infection rate over time. A change in infection rate can be attributed to a combination of many things, including the impact of policy measures and changes in people’s behaviour. Data from both of these measures was available through both scientific publications and a variety of data sources – from service mobile service provider data reports to individual laboratory results. The accuracy of the forecast was determined by how well the most reliable estimates of the infection rate could be derived from different data sources and accounted for in the Finnish epidemic model.

Government forecasts assumed that over a period of 100 days, people would reduce their social contacts by a third or a quarter. TietoEVRY used a more multi-dimensional analysis to determine the infection rate and this is the reason why good predictive accuracy was achieved. This proves that the most complex model or assumptions are not essential and is backed up by Occam's razor theory which recommends choosing the simplest of two theories to explain a phenomenon.

Even in situations where in-depth mathematics underpins an approach, it is worth trying to describe through simple assumptions, a way to manage uncertainty over a period of time. When the assumptions made can be described as detached and comprehensible entities, it is easy to discuss the uncertainties of the outcome with clarity and science, not mysticism.