Part 1 of this series, in posing the question “Does measuring social impact matter?”, concluded that it is not whether to measure social impact (you should!), but how you can do it effectively and efficiently.
To that end, we attended The Community Development Resource Association’s Planning, Monitoring, Evaluation & Learning course late last year, a week-long immersion into all things MERL, MEAL, M&E and PMEL. Facilitated by the CDRA’s Marianne Brittijn and Dzvinka Kachur, the subtext of the course – “When ticking boxes no longer makes sense” – sent out an explicit challenge to delegates: the development world is changing rapidly, and you had better be prepared to change with it.
The development world is changing rapidly, and you had better be prepared to change with it.
“Monitoring and evaluation, when done well, can assist organisations in improving their practice and effectiveness,” said Brittjn. “But often, M&E is set up to fail, unintentionally.”
“As we become more technically sophisticated at capturing lots of data, we can easily lose sight of what really matters. The pressure to show ‘results’ ends up undermining our ability to take a step back and look at our work with a fresh perspective, asking new and better questions.”
As we become more technically sophisticated at capturing lots of data, we can easily lose sight of what really matters.
Marianne Brittijn, CDRA
Brittjin posed some probing questions:
“Does our PM&E framework represent what we are really trying to achieve? Are we bringing learning into practice adequately? Are our partners learning and growing with us? Or are we just ‘ticking the boxes’ with a growing sense of unease, losing track of the deeper questions?”
Through these questions, she emphasised a return to getting the basics right, so that organisations can comply with the demand for quantitative data, whilst at the same time providing them with a more developmental way of doing PM&E that is focused on continuous reflection and learning in order to improve future practice and achieve more sustainable impact.
Which, after all, is what we all seek to do.
Investing in monitoring, evaluation and learning
How much should organisations be investing in Monitoring, Evaluation and Learning? In answering this question, Brittjin pointed us towards a study commissioned by Comic Relief, DFID, Big Lottery Fund, NIDOS and Bond, that addresses the lack of evidence available to support NGOs working in international development in deciding what resources to commit to monitoring, evaluation and learning (MEL).
The study, “Investing in Monitoring, Evaluation and Learning”, focused on understanding the full investment that NGOs are making in MEL, the kinds of MEL systems they have, and how NGOs use and value their MEL systems.
NGOs take MEL seriously
The study found that a significant number of NGOs are incredibly serious about MEL, and so make notable investments in it because they believe it’s a way to improve not just their own work, but also that of their partners. It goes on to note that, “for most NGOs, their MEL systems are reported to support them in making day-to-day project management decisions with many saying MEL also supports strategic management and learning.”
The study also found that the most useful MEL systems, according to the NGOs using them, combine long-term relationships with partner NGOs with the ability to do data analysis close to the ground and MEL capacity building of local partners.
“They (the effective systems) also have deep integration of MEL within an NGO head office, ensuring it is perceived as a collective responsibility, and a focus on improving work with beneficiaries and partners, rather than proving effectiveness to donors or external stakeholders.”
Important aspects in resourcing and maintaining MEL systems
The study showed that three aspects are important in resourcing and maintaining MEL systems:
- Strategic investment and funding
- Adequate staff capacity
1) Funding MEL
- The amount NGOs spend on MEL varies enormously. Whilst some report they spend very little, others are spending significant proportions of their overall budgets.
- NGOs fund MEL through project and unrestricted funds on an ongoing basis.
- MEL activities are normally separated out within project budgets, but these budgets frequently do not reflect the full cost of MEL.
- In fact, NGOs generally spend more on MEL activities than they budget for or report on. In part, this is due to costs normally located in core costs (e.g. salaries) and overheads (e.g. database maintenance) not being considered valid MEL costs. It can also be due to the perception that some funders will not accept the full cost of MEL being included in project budgets, or NGOs themselves not realising the extent of the true costs.
- Whilst most donors will fund at least some of the costs of project MEL, finding funds to develop bespoke systems can be challenging.
NGOs generally spend more on MEL activities than they budget for or report on.
Significant expenditure on MEL requires leadership buy-in and support. The study suggests that it’s necessary to have leadership that:
- is committed to having a MEL system that supports the needs and aims of the organisation and is prepared to make it a strategic priority;
- has determined what an appropriate MEL system looks like for their NGO; and,
- is clear why MEL is important for the organisation.
- In most NGOs, MEL takes up a considerable proportion of staff time at all levels. This is often under-recognised in project budgets, though it does tend to be clearly articulated either across many job descriptions or within dedicated MEL roles.
- NGOs find it challenging to ensure adequate MEL capacity at the field level.
This (staff time) is often under-recognised in project budgets…
The path to achieving more sustainable impact
To be clear, none of these should be issues given the trends that are underpinning dramatic change in the development world, and a shift from the social era to the social impact era (see Part 1 of this series).
What we know now:
- MEL simply must be appropriately budgeted for and funded.
- Leaders must make a strategic commitment to MEL systems which scale, are secure and cost-effective, and which serve to support the needs of the organisation.
- Staff must become proficient with technology that reduces their administrative workload, improves data quality, and encourages transparency and collaboration.
It’s this transformation which will go a long way to ensure our organisations can satisfy the demand for quantitative data, whilst at the same time providing them with a more developmental way of doing PM&E in order to improve future practice and achieve more sustainable impact, as Brittijn and Kachur implore.
Part 3 of this series looks at some of the issues for NGOs to consider when investing in monitoring, evaluation and learning, and their implications.