A new WASH index: improving accountability and equity

Published on: 27/06/2018

The new JMP WASHdata website offers a simple tool for visualising data on country-level achievement towards the Sustainable Development Goals (SDGs) for water and sanitation. The targets for water, sanitation and hygiene are broken down into service-level ladders with each rung representing the proportion of the population receiving that level of service. Evaluating these rungs for each country is useful in judging progress on a small scale, but it can be difficult to compare these ladders across countries or regions. Most often, we see comparisons between countries based on the proportion of the population receiving a “safely-managed” or “at least basic” service. This type of evaluation aids the sector in understanding how far a country lies from achieving 100% coverage, as advocated for in the SDG targets, but fails to shed light on where the rest of the population falls on that ladder—i.e. the population with no or very limited services. 

In striving to achieve the SDGs for water and sanitation, should we place more emphasis on lifting those with basic services up to safely managed? Or should we instead focus on that portion of the population that is underserved or wholly unserved? Using an index that includes a weighting scale is beneficial, because it allows us to account for both, and to show whether progress is being made in either.

With this issue in mind, Stuart Kempster and Andres Hueso, in their co-authored draft WaterAid discussion paper, put forth a potential method to compare achievement levels in sanitation across countries and time. This new index, the Sanitation Progress Index (SPI), considers members of the population who fall at the lower rungs of the ladder. It does so by placing a weight at each rung, with “safely managed services”, the gold standard, assigned a weight of 1 and “open defecation”, or the lack of service, a weight of 0. By simply multiplying the proportion of the population at each rung by the weight assigned and summing these components, the SPI for each country/region is easily calculated. This score provides a more holistic picture of where a country lies in achieving the SDGs and makes it possible to acknowledge progress made even when it is not reaching the higher benchmarks of safely managed or basic services (for example, a country which has improved access via shared toilets or construction of traditional latrines in place of open defecation).

Using the method proposed by Kempster and Hueso, we calculated an SPI for each of the IRC focus countries over the 15-year time period from 2000-2015. We followed a weighting scale prioritizing pulling those practicing open defecation up to at least ‘limited’ services (Open Defecation: 0, Unimproved: 0.1, Limited: 0.6, Basic: 0.8, Safely Managed: see Note). While there is no truly objective method for weighting each rung, the bias inherent in different weighting scales can be used for positive political purposes, such as advocacy for reaching at least limited services for all.¹ We then compared these index scores for each country to the proportion of the population receiving ‘at least basic’ services—the more traditional method of comparison. Presented below are the SPI scores for each country, as well as comparisons to the more commonly used ‘at least basic’ indicator. The SPI score is shown adjusted to a scale from 0-100 to provide for a clearer comparison between the two scores.   

Graph 1. SPI scores compared to the proportion of the population receiving ‘at least basic’ services for four of the six IRC focus countries.

Using the same weighting system from the Sanitation Progress Index, we then calculated a “Water Progress Index” (WPI) for each of the IRC focus countries from 2000-2015. The results of this comparison are shown below, along with a comparison to ‘at least basic’ service levels for the focus countries.   

Graph 2. WPI scores compared to the proportion of the population receiving ‘at least basic’ services for the IRC focus countries.

Comparing across countries within each index, as well as across indices, provides some interesting insight into the progress that each of our focus countries has made over the 15-year period.

• Honduras performs at a similar level in both water and sanitation and shows improvement even starting from a relatively high level of coverage in the country. For both water and sanitation, the ‘at least basic’ service level indicator portrays this success at a higher level than with the Progress Index. Despite the high portion of the population with ‘at least basic’, the index score is pulled down due to the large portion of the population falling in the completely unserved rung.
• India shows a similar trend to Honduras with the unserved population for water bringing down the index score relative to the traditional indicator. For sanitation, India is performing at about the same level in both types of indicators, echoing both the commitment to get the entire population up to at least the first rung of the ladder and making simultaneous improvements at higher rungs.
• In Ghana, the difference between the two sanitation indicators is quite large. Most of the population in Ghana falls on the ‘limited services’ rung (shared services) rendering the score for the SPI much higher than the proportion of the population with ‘at least basic services’.
• Ethiopia shows significant improvements during the 15 years, echoing the trend discussed by Stef Smits that improvement is more rapid for countries at lower rates of coverage. SPI and WPI scores are increasing at a faster rate than the ‘at least basic’ indicator, reflecting efforts to lift the unserved to minimum services, and showing relatively slower progress in getting people to higher rungs of the service ladder.

 Using these progress indices, as opposed to comparing on one service level, allows for a clearer preliminary study on the trends of progress in water and sanitation. These comparisons can aid researchers and monitoring professionals in determining which trends warrant further investigation, and they can be useful in advocacy and agenda-setting. Data and presentation of data can have a strong influence on public opinion, and an alternative index can bring a new perspective to the conversation that may have implications on where to focus and allocate resources. The index highlights the importance of reaching un- and underserved population segments, rather than just improving service quality toward safely managed services at the higher rungs. Through this method of data presentation, we can work to incentivise pro-poor policies and emphasis on reaching everyone.

Note

Another caveat of using this method is that many countries do not have data available in the JMP database for safely managed services, therefore bringing down their index score and perhaps unfairly representing their progress. Thus, the limitation of one method is reflected in another. One possible way to deal with this issue is to calculate the index scores for countries without safely managed data by weighting “at least basic” as 1 and moving down from there. Kempster and Hueso do address this issue in their paper and explain that the index is used to provide comparison with the end goal of “safely managed” services for all. Therefore, the reduction in the SPI score resulting from the lack of data offers an incentive for countries to improve their monitoring systems.

For more information on the Sanitation Progress Index (SPI) please contact WaterAid's Policy Analyst for Monitoring and Accountability, Stuart Kempster at StuartKempster@wateraid.org

Update

Since Kempster and Hueso developed the SPI, they have updated their methodology to calculate what they call a 'total service gap'. They describe this measure as a way to "combine data on each rung of the [sanitation] service ladder to quantify how far away each country is from universal safely managed services" based on a "sensitivity analysis" that assesses "the validity of using uniform service level weights". Read more in their November 2018 article "Moving up the ladder : assessing sanitation progress through a total service gap" listed below under Resources.