The tricky business of what is considered “evidence” in WASH

Published on: 23/05/2013

DFID, the British bilateral cooperation agency has published the latest "Water, Sanitation and Hygiene – evidence paper". I agree generally with the broad lines of the conclusions about what we still need to know to do our jobs in the sector better, but it was with utter disbelief that I read what is considered "evidence" in the report and what is discarded as not being "sufficient evidence".

The DFID report bases its conclusions on "evidence" which is extrapolated from assumptions and a small number of data points, but published in academic journals, while extensive data collection and analysis done by NGOs and governments and freely available to anyone with the ability to use Google - is largely ignored as "evidence".

Two conclusions and areas with which I am most familiar, had findings based on peculiar and narrow choices of materials:

• "There is suggestive evidence that sustainability is a major challenge for WASH services with facilities falling into disuse or disrepair over time" (p.10)
• "There is limited systematic data on the costs of providing and sustaining WASH services" (p.11)

On the first point on evidence of sustainability, over the last years, there has been consistent and overwhelming evidence and many reports of rates of failure and non-functionality both for water and sanitation. Many of these references and reports, published by NGOs and Governments are being compiled by Improve International. Most of them are not published in an academic journal – but many are based on several data points with a sufficiently solid research methodology behind them.

Some examples of strong evidence on the lack of sustainability of WASH services include the joint performance report in Uganda which annually reports on non-functionality, the water point mapping done in Liberia and Sierra Leone, both countries with strong DFID aid support to the sector. The evaluation report of the Dutch aid to the water and sanitation sector, the EU evaluation of development assistance to the sector provide further evidence on the lack of sustainability of WASH services. More recently, the latest research report by IRC presents the results of a baseline assessment of the status of service levels, service providers and support functions in three districts in Ghana. The study shows a high level of non-compliance with the country set norms and standards.

On the second point, for the conclusions related with costs, it is indeed correct that limited studies have been done on the costs of service provision. For that reason, WASHCost has been publishing cost data and its methodology (LCCA - the life-cycle cost approach) for the last three years. The authors of the DFID report have however, chosen to reproduce a table from 2007 with data from 2000 (p. 86) which is based mostly entirely on assumptions and not real data collection (p. 84) and which shows data which has now proven to be grossly underestimated.

The WASHCost benchmarks for water and sanitation are based on an extensive research programme with large data sets; please see the links below for more information. The results are limited in geographic scope (4 countries) and we are now very happy that a large number of organisations and governments in different countries are applying the life-cycle cost approach (LCCA) not only for water and sanitation, but also for hygiene and school sanitation programmes.

In conclusion, we expect that in a few years, the sector can have a larger collective evidence base on costs and sustainability of services (including research to be funded by DFID). But probably, only a selection of these will ever be published in scientific or academic journals and with appropriate validation we encourage DFID to start using and referring to these other valuable studies as well.

Catarina Fonseca is Project Director WASHCost and Lead International Programme at the IRC International Water and Sanitation Centre,@FonsecaCatarina