noun_Email_707352 noun_917542_cc noun_Globe_1168332 Map point Play Untitled Retweet

TietoEVRY's corona epidemic prediction accuracy is top notch

Tapio Pitkäranta / June 04, 2020

This spring, the role of different forecasting models and scenarios in the coronavirus epidemic has sparked much debate.

When the Finnish government decided on exceptional circumstances in March, TietoEVRY's advanced healthcare cloud analysis team began to develop scenarios on the subject. The aim was to make a realistic forecast of the duration of the epidemic and the number of infections.

Now, more than two months have passed since the scenarios prepared in March were created, it is a good time to compare the different scenarios and reality that has unfolded. In Finland, the corona epidemic has developed and been contained more positively than expected.

 

The following time scales are included in the table:

  1. The three forecasts prepared at the end of March by TietoEVRY and made available to the company's own and cloud analytics customers (1.4. onwards)
  2. Three estimates of current infections in Finland: 2x (“Lower limit”), 5x (“Intermediate”) and 10x (“Upper limit”) based on the number of detected infections
    1. At present, it is not known exactly what the actual infection number is, but it has been estimated that the infection number (FI) would be about 2-10 times the number of confirmed infections.
  3. The three scenarios defined in the memorandum guiding the Finnish Government's decision on exceptional circumstances created 16 March 2020 (FI) and the two scenarios outlined in the memorandum and informing the decision made on 25 March 2020 (FI) on the closure of Uusimaa.

These projections include the effects of emergency slowdown measures. In the table, we see how the predictions determined by TietoEVRY's logarithmic scale follow quite closely the current estimates of the total number of infections.

How accurate were the forecasts?

It has been said that making an accurate prediction is simple, but not easy; achieving good forecast accuracy is rarely based on a superior forecast model.

Therefore, to manage uncertainty and to understand the phenomenon, it is essential to understand exactly what assumptions are being entered into the model and if these need to be changed during the forecast period. It is often the case that poor forecast accuracy can be attributed to inaccurate assumptions being entered into the model at the early stages and not accounting for changes over the course of the forecast period.

When analyzing the forecasting errors in the forecast models used in the Finnish government’s memorandum, it has been noted that these are not due to the epidemic models themselves, which were updated at different points during the spring, but precisely because of the assumptions themselves. In practice, people’s behaviour changed more than the scenarios had assumed which accounts for the discrepancy between predicted and detected infections.

Numerous ready-made models for modelling epidemics already exist and we can even speak of model libraries. For those without extensive experience in epidemiology or education in epidemic modelling, it would be questionable to try and develop a new way to model the epidemic itself.

For the first modelling of the epidemic, TietoEVRY followed the well-known SEIR model based on fairly simple differential equations. Because the model includes an interest-to-interest type exponential function, it must be carefully assessed whether the assumptions of that basic reproduction number last for the duration of the simulation period. At the same time, the co-efficients of the differential equation are no longer constant.

Do the assumptions last?

The determining factor in predicting accuracy in this case was not the use of the SEIR model over another model, but whether the parameters of the model used were applicable and realistic over time. To achieve high prediction accuracy, the most important element is to describe the assumptions correctly, that is, to predict the parameters of the model over time.

In practical epidemic situations, accuracy is determined by correctly predicting the infection rate over time. A change in infection rate can be attributed to a combination of many things, including the impact of policy measures and changes in people’s behaviour. Data from both of these measures was available through both scientific publications and a variety of data sources – from service mobile service provider data reports to individual laboratory results. The accuracy of the forecast was determined by how well the most reliable estimates of the infection rate could be derived from different data sources and accounted for in the Finnish epidemic model.

Government forecasts assumed that over a period of 100 days, people would reduce their social contacts by a third or a quarter. TietoEVRY used a more multi-dimensional analysis to determine the infection rate and this is the reason why good predictive accuracy was achieved. This proves that the most complex model or assumptions are not essential and is backed up by Occam's razor theory which recommends choosing the simplest of two theories to explain a phenomenon.

Even in situations where in-depth mathematics underpins an approach, it is worth trying to describe through simple assumptions, a way to manage uncertainty over a period of time. When the assumptions made can be described as detached and comprehensible entities, it is easy to discuss the uncertainties of the outcome with clarity and science, not mysticism.

Interested in joining our projects that make a difference? Check out our open positions!
Tapio Pitkäranta
TietoEVRY alumn

At TietoEVRY Tapio was working with Nordic healthcare customers to utilize cloud technology. Example partner has been Helsinki University Hospital that has implemented one of the world’s biggest data lake implementations with cloud applications together with TietoEVRY. Tapio believes that value based healthcare is achieved by embedding technology and data driven research to patient care development.

Author

Tapio Pitkäranta

TietoEVRY alumn

Share on Facebook Tweet Share on LinkedIn