NITI Aayog, National Institution for Transforming India, Government of India

FAQ

The Index includes 23 indicators which are used to calculate the composite index score and generate overall performance ranks and incremental ranks. A detailed list of indicators is provided in Table 2.3 of the report.
An additional indicator on ‘average out of pocket expenditure per delivery in public health facility (in INR)’ was included for calculation of an independent reference year index, since data was only available for 2015-16, which is provided in Annexure 3 of the report.
The final list of indicators included in the Index was developed through an iterative process of development, feedback and field testing in two States, taking into account importance and availability (at least annually) of reliable data. 28 indicators were shortlisted in the original Health Index (Annexure 2). Once data collection and initial validation was completed, the availability and quality of data for all States was reviewed in a meeting chaired by Member, NITI Aayog. Based on the observations shared by MoHFW, the World Bank, and the Independent Validation Agency (IVA), as well as inputs from States and experts, 23 indicators (see table 2.3 of the report) were retained.
The Index covers three domains- Health Outcomes (sub-domains: Key outcomes & Intermediate outcomes; Governance and Information (sub-domains: Health monitoring and data integrity & Governance); Key inputs/Processes (sub-domain: Health systems/ Service delivery).
The composite index score is calculated for a Base Year, which largely covers the 2014-15 period and a Reference year, which largely covers the 2015-16 period. The Incremental ranks are a measure of the difference in performance between these two periods. For some indicators, data from published sources is provided for calendar years instead of financial year while for others it is available as a pooled estimate, or with a wider reference period. The exact period to which data for each indicator applies is provided in Table 2.3 of the report.
Since the focus of the exercise is to measure annual incremental change in performance of States and UTs, two time periods, i.e., base year and reference year, are considered.. The Base Year largely covers the States’ performance during 2014-15 and the Reference Year covers the performance during 2015-16.

The Overall performance for base and reference years is based on the composite index scores obtained by States and UTs for the Base Year and Reference Year respectively. Accordingly, Base Year and Reference Year ranks are generated to indicate the absolute levels of achievements of States, relative to one another.

The focus of this exercise however, is to measure annual incremental change in performance, to nudge States towards obtaining optimal levels of performance. The Incremental scores are therefore generated as the difference between the composite index score in the Base Year and Reference Year and the States /UTs are ranked based on this incremental score.

Overall rankings are based on composite index scores and signify absolute levels of performance of each State, relative to one another. These are therefore a reflection of the cumulative effects of health systems development, socio-economic determinants of Health and other associated factors over several decades. Incremental rankings are based on the difference between composite index scores in the base and reference year, and are a function of the absolute level at which States are, since States and UTs that start at lower levels of the Health Index (lower levels of development of their health systems) are generally at an advantage in notching up incremental progress over States with high Health Index score due to diminishing marginal returns in outcomes for similar effort levels. It is a challenge for States at high levels of the Index score even to maintain their performance levels. For example, Kerala ranks on top in terms of overall performance and at the bottom in terms of incremental progress mainly as it had already achieved a low level of Neonatal Mortality Rate (NMR) and Under-five Mortality Rate (U5MR) and replacement level fertility, leaving limited space for any further improvements.

Nonetheless, based on the highest overall Index scores of the top ranked State/UT in each category of States/UTs, clearly there is room for improvement in all States and UTs.

Yes, ranks have been generated for all States and UTs. However, in order to ensure comparison among similar entities, to the extent feasible and based on data availability, the States and UTs are ranked in three separate categories- Larger States (21), Smaller States (8) and Union Territories (7).  (See section 2.4.2 of the report for details)
No, all States and UTs are not ranked together. In order to ensure comparison among similar entities, to the extent feasible and based on data availability, the States and UTs are ranked in three separate categories- Larger States (21), Smaller States (8) and Union Territories (7).  (See section 2.4.2 of the report for details)
No, all the 23 indicators are applicable only to the ‘Larger States’ category, which includes 21 States for which data on all these indicators is available. For the ‘Smaller States’ a sub-set of 19 indicators are applicable while for the UTs 18 indicators are applicable. The exact number and list of indicators included for each category of States is provided in the Table 2.2 and Table 2.3 of the report.
As the Index is a weighted composite Index comprising of indicators in three domains, each domain has been assigned weights based on the expert opinion and its importance. Given the focus on measuring performance, health outcomes have been assigned the maximum weightage. Within a domain or sub–domain, the weight has been equally distributed among the indicators in that domain or sub-domain.  Table 2.2 of the report provides the details of the domains and their weightages for each category of States/UTs.
Broadly the data sources used for the index are – Sample Registration System (SRS), Health Management Information System (HMIS), Central MoHFW data, State Report, National Family and Health Survey (NFHS), Civil Registration System (CRS), The details on indicator wise data source and period is provided in table 2.3 of the report.
Each indicator value was scaled, based on the nature of the indicator. For positive indicators, where higher the value, better the performance (e.g. service coverage indicators), the scaled value (Si) for the ith indicator, with data value as Xi. was calculated as follows:
text 
Similarly, for negative indicators where lower the value, better the performance (e.g.  NMR, U5MR, human resource vacancies), the scaled value was calculated as follows:
text 
The minimum and maximum values of each indicator were ascertained based on the values for that indicator across States within the grouping of States (Larger States, Smaller States, and UTs) for that year.
The minimum and maximum values of each indicator were ascertained based on the values for that indicator across States within the grouping of States (Larger States, Smaller States, and UTs) for that year.

After validation of data by the Independent Validation Agency, data submitted by the States and pre-entered from established sources was used for the Health Index score calculations on the portal.

Each indicator value was scaled, based on the nature of the indicator.

For positive indicators, where higher the value, better the performance (e.g. service coverage indicators), the scaled value (Si) for the ith indicator, with data value as Xi. was calculated as follows:
text 
Similarly, for negative indicators where lower the value, better the performance (e.g.  NMR, U5MR, human resource vacancies), the scaled value was calculated as follows:
text 
The minimum and maximum values of each indicator were ascertained based on the values for that indicator across States within the grouping of States (Larger States, Smaller States, and UTs) for that year.

Based on the above scaled values (Si), a composite Index score was then calculated for the base year and reference year after application of the weights using the following formula:
text 
where Wi is the weight for ith indicator.

The ranks were generated based on the composite index scores as follows:

  • The composite Index score provides the overall performance and domain-wise performance for each State and UT and has been used for generating Overall Ranks by ordering the States / UTs in descending order of index scores.
  • The difference between the composite Index score of reference and base years was used to compute the annual incremental performance and ranks.

All calculations were automated on the portal and verified by the Independent Validation Agency.

The dedicated interactive web portal (social.niti.gov.in) includes functions for submission of data. Data was entered in the portal by the States and UTs, as per the indicator definitions and data sources provided in the Guidebook (Performance on Health Outcomes, A Reference Guidebook, NITI Aayog, December 2016), except for some designated indicators/fields pre-entered on the basis of data source identified at the outset.

The final submission of data on the portal was done by the designated State-level competent authority.

Mentors were assigned to most States to facilitate data collection and submission on the portal. The Empowered Action Group (EAG) States and North-Eastern States were provided dedicated mentor support while other States received the same on request. The mentor agencies assigned to various States are listed in Table 3.4 of the report.

An Independent Validation Agency (IVA), namely, IPE Global, was hired by NITI Aayog through a competitive selection process to review and validate the Health Index data and the State rankings.

The data submitted on the portal was validated by the IVA and the validation steps are given in the figure below.
text 
Field visits were conducted to carry physical validation of the data in Assam, Chhattisgarh, Rajasthan, Kerala, Himachal Pradesh, Bihar and Jharkhand. A regional workshop was also held to cover the seven North-Eastern States. The detailed note on discrepancies in data submitted and their resolution is provided in Annexure 1 of the report.

The data was finalized by the IVA after resolution of all discrepancies in consultation with State and Central governments, who, after thorough review of the data and supporting documentation, identified gaps and data discrepancies which were then discussed with state nodal officers (SNOs) and State-level authorities. A State-specific validation report was prepared and shared with the Principal Secretaries, Mission Directors and SNOs highlighting the results of the validation exercise. The States were requested to review the validation report and provide feedback. Subsequently, the IVA also presented the validation results through five video conferences held during August 16-18, 2017, with groups of 7-8 States to share the findings and discuss discrepancies, data gaps, variations and deviations.

HMIS data entered by the States was examined by the Independent Validation Agency against supporting documentation of records submitted by States and compared with the data downloaded/provided by Ministry of Health and Family Welfare at the Centre from the HMIS portal. In cases where there was a conflict, outliers, unusual increase/decrease in data identified by the IVA, these were discussed with the State Nodal Officer and MoHFW officer, and the most reliable value as ascertained by the IVA was considered as the final value. Field verification of HMIS data could not be carried out by the IVA due to lack of feasibility for carrying out independent field surveys.

The HMIS is a large administrative database that has been developed and improved over the years through the efforts of a large workforce across the Centre and the States. Despite its known limitations, since the exercise seeks to measure annual change in performance, for several indicators, the HMIS is the only annually available data source for many of the indicators for all States/UTs. The validation component also would serve to help improve this database going forward, while encouraging States to update and improve the quality of data uploaded in the HMIS in a timely manner.

An indicator titled ‘Data Integrity measure’ with a weight of 70, measures the quantum of deviation of HMIS data from NFHS 4 data, has been used in the index, thereby also holding States accountable for the quality of their HMIS data.

The lack of acceptable quality data on annual basis presented challenges:

    • Critical areas such as infectious diseases, non-communicable diseases, mental health, governance, and financial risk protection could not be fully captured
    • For several indicators, the data was limited to service delivery in public facilities
    • Weights had to be determined based on expert opinion
    • For SRS related key outcome indicators, data was available only for Larger States. Hence, the Health Index scores and ranks for Smaller States and UTs were calculated excluding these indicators
    • Field verification of HMIS and program data by IVA due to lack of feasibility of conducting independent annual field surveys

Detailed notes on the limitations of the index are provided in Section 2.5 of the report.

Annual reliable outcome data on non-communicable diseases across States could not be obtained during the indicator development process, which included field testing in two States for data availability. In order to at least provide an indication of availability of service delivery for NCDs, an indicator on ‘Proportion of districts with functional Cardiac Care Units (CCUs)’ in the government sector was included in the Index. This limitation will be attempted to be addressed in the next round of the Index with the initiation of NCD screening programmes, the State level disease burden estimates published by Department of Health Research & Institute of Health Metrics and Evaluation, as well as other relevant inputs.

The Health Index report acknowledges that States and UTs that start at lower levels of the Health Index (lower levels of development of their health systems) are generally at an advantage in notching up incremental progress over States with high Health Index scores, due to diminishing marginal returns in outcomes for similar effort levels. However, the focus of this exercise is to nudge historically poor-performing States (particularly the Empowered Action Group (EAG) States-) to transform their health systems and help India achieve SDG 3: the Sustainable Development Goal for health which strives to ensure healthy lives and promote wellbeing for all at all ages

Nonetheless, it may also be noted that Kerala’s composite index scores for the Base Year and Reference Year are 80.00 and 76.55 respectively.  This shows that there is still scope for improvement in the performance of the State, as well as, the challenge to maintain high performance levels. The incremental ranks promote competition for a State against itself, and its prior achievements.

Overall, the incremental performance does not appear to be associated with the overall Index score. Among Larger States, some of the better-performing States have made negative incremental progress. Three of the top five Larger States (Kerala, Gujarat, and Himachal Pradesh) recorded negative changes in the overall performance Index score between base and reference years. However, this association is not consistent in the group of Smaller States and UTs.
The Health Index provides States’ a composite measure of their performance on a range of indicators over two time periods, as well as the annual change in performance over these two time periods. The ranks highlight to States where they stand relative to one another, and provide them a benchmark of the level of achievement they must strive for. The Report provides States with details on indicator-wise performance, where performance has improved, stagnated or declined, thereby highlighting the specific areas in which States are required to invest additional efforts (Annexure 4).
The next round of the Health index is envisaged to be completed by the end of 2018, and will include overall performance up to 2016-17, and incremental performance from 2015-16 to 2016-17.
The Index was developed by NITI Aayog with technical assistance from the World Bank through an iterative process in consultation with the Ministry of Health and Family Welfare (MoHFW), States and UTs, domestic and international sector experts and other development partners. The details on key roles and responsibilities of each stakeholder is provided in table 3.1 of the report.
The interactive web-portal for index is available at social.niti.gov.in. The web-portal provides all the details of the index – about index, guidebook on performance on health outcomes, state/UT category wise overall, incremental and domain-wise results (in form of tables, graphs etc.), indicator wise data (options available for download of data and results) and the health index report.