India's Objections To WHO COVID-19 Mortality Estimates Are Misleading, Experts Say

In times like the pandemic, when it might not always be possible to confirm that a death was from Covid-19, health agencies rely on estimates of excess mortality--the gap between expected and observed mortality from a disease--to arrive at better estimates of the true toll of Covid-19.

The WHO is expected to release global and country-level estimates of excess mortality this week, Steve MacFeely, director of data and analytics at the WHO, said during an online discussion on March 21. Excess mortality estimates are particularly important as countries around the world, including India, are expected to have missed registering a significant number of deaths from Covid-19 for a variety of reasons, ranging from insufficient testing to active suppression of the diseases' real toll, experts say.

The WHO's process, which involved discussion with its 194 member states, began in February 2021, but the Indian government has since at least December 2021 been stalling the exercise, objecting to the WHO's methodology and promising to share data that it has yet to share, internal documents and communications between the government and the WHO, viewed by IndiaSpend show. India is the only country that has these objections, MacFeely said, and the WHO's estimates will be released this week with a note of India's objections.

The WHO is expected to place estimated mortality from Covid-19 at 4 million as of the end of 2021, about eight times India's officially reported Covid-19 deaths of 480,000.

In response to questions from IndiaSpend, a health ministry spokesperson directed IndiaSpend to a recent press release put out by the ministry, following an article by the New York Times on the wrangling between the WHO and the government. However, that press release makes claims that experts associated with the WHO's process, as well as external experts, have called out as wrong or misleading.

At the heart of India's objections to the WHO estimate lie data--their presence or absence, availability of that data, and their applicability. When it became clear to journalists and researchers that not all deaths from Covid-19 were being counted in India, media organisations, including IndiaSpend, began to access state-level Civil Registration System (CRS) portals that count all deaths in a district for every month. In states like Madhya Pradesh and Andhra Pradesh in particular, the CRS data showed large gaps between CRS-registered deaths for previous years and deaths for pandemic months, as well as a large gap between reported Covid-19 deaths and observed mortality.

Indian journalists made CRS-based monthly mortality data for 2020 and 2021 (as well as previous years, where available) for 18 states available to researchers for use, and these data now form the basis of the WHO's estimates. However, the Indian government disputes the use of these data, despite them being from a government source; communications between the WHO and the Indian government show that the government offered to make monthly mortality estimates available to the WHO, but did not subsequently do so.

This places India in "Tier II" countries as classified by the WHO in terms of data, a classification the Indian government has objected to. "[I]nclusion of a country like Iraq which is undergoing an extended complex emergency under Tier I countries raises doubts on WHO's assessment in categorization of countries as Tier I/II and its assertion on quality of mortality reporting from these countries," the government's press release said. However, the WHO has said that the classification represents both the quality of data in a country and its willingness to share them; "Some countries say they have good data but have chosen not to make it available. That's their prerogative, but that's why they are classified accordingly," MacFeely told IndiaSpend.

While India has good mortality data, its refusal to share them has resulted in its classification as Tier II, said a WHO official who asked not to be named as the discussions with India are ongoing and sensitive. As a result, the WHO uses data from the 18 states and has made extrapolations for the rest of the country on the basis of the typical share of deaths between different states, Ariel Karlinsky, economist and statistician at the Hebrew University in Jerusalem, and a member of the Technical Advisory Group (TAG) of the WHO's COVID-19 Mortality Assessment Group, explained.

Unlike claims made by the Indian government in its press release, the WHO's estimates do not rely on models that include factors like seasonal variations in temperature, Jon Wakefield, professor of statistics and biostatistics at the University of Washington, who had led the modelling for India as a member of the TAG, explained. Those models are only required for countries which do not have monthly mortality data, while in the case of India, the data were available through the work of journalists, the WHO official explained.

Another point of contention is what the baseline for deaths in a year should be. What would mortality have been without the pandemic, would be the first step towards estimating what the excess during the pandemic then was. The WHO has relied on its Global Health Estimates (GHE 2019) dataset for 2019 for baseline mortality for India, which the government objects to in its press release, stating that "India has a robust system of data collection and management".

However, the most recent national mortality estimates for India are for 2019 via the Sample Registration System (SRS 2019) and came out only in October 2021. Additionally, the Indian government has used GHE 2019 data in its own published research and other work, Prabhat Jha, founding director of the Toronto-based Centre for Global Health Research, and a leading global expert on mortality, who is also a member of the TAG, told IndiaSpend. "Moreover, the SRS 2019 produces lower counts of deaths than the GHE 2019, meaning that if anything, the excess mortality during the pandemic is higher than the WHO's estimates."

"The death reporting system needs repair and transparency and the WHO and other stakeholders can collaborate with Indian policymakers to chart a plan to get better data in the future by crowdsourcing, adding a question to the census, household surveys," Bhramar Mukherjee, chair of biostatistics and professor of epidemiology at the University of Michigan, who is also a member of the TAG, told IndiaSpend. She added that tracking deaths and its causes is important for public health and public policy."

Other estimates in academic papers have produced similar estimates, with Jha's own work producing an estimate of 3.2 million deaths between June 2020 and July 2021.

"Multiple models with multiple data sources have come up with similar estimates...By all methods and models three to four million excess deaths reflect the true toll of the pandemic," said Mukherjee. "[E]ven with this excess mortality estimate of total deaths, deaths per million in India are quite low and the country dealt with the third wave much better by speeding up its vaccination efforts."

The WHO has explained its methodology to Indian officials multiple times since December 2021, the WHO official, who wanted to remain anonymous, said. "Ultimately, excess mortality estimates are essential for preparedness for the next pandemic," MacFeely said. "These are reliable and established methods used for the estimates. We know that India is not happy, and the final publication will mention that. But for all countries, we need to put these numbers out now."

Note: This story has been updated with comments from Bhramar Mukherjee, chair of biostatistics and professor of epidemiology at the University of Michigan.

We welcome feedback. Please write to respond@indiaspend.org. We reserve the right to edit responses for language and grammar.