Press enter to search
Blog

Podcast: I Think You’re On Mute: Measuring What Matters

Listen now on: Spotify | SoundCloud | Apple | YouTube

The humanitarian sector has made significant strides to support more localised action with various frameworks and tools, but there’s still a gap in understanding how these approaches impact affected communities.

Over the past three years, the Humanitarian Advisory Group has been exploring ways to measure the impact of localisation at the community level. A Pathway to Localisation Impact: Laying the Foundations, introduced a model to support humanitarian actors to better define and understand the impact of their localisation practices on crisis-affected populations.

This model was tested with organisations in Pakistan, Bangladesh, and the Philippines. In this episode of I Think You’re On Mute, host Beth Eggleston talks to Aaftab Ullah and Zaki Ullah, members of the partnering research team that tested the model in Pakistan.

More podcast episodes here

 

Podcast host and guests

Beth Eggleston

Beth is the Director of the Humanitarian Advisory Group (HAG) and co-founded the organisation in 2012. She has worked in the humanitarian sector specialising in civil-military coordination and humanitarian reform for the last two decades and has field experience in Afghanistan, Liberia, Tonga, Costa Rica, Laos PDR, Timor-Leste, and Vietnam.

Aaftab Ullah

Aaftab has worked as Manager M&E with GLOW Consultants. He has over 10 years of professional experience. He has successfully contributed to a number of research publications.

Zaki Ullah

Zaki is the CEO of GLOW Consultants, a research firm based in Pakistan. He has over 15 years of professional experience working both in development and humanitarian settings. He specialises in monitoring, evaluation and research work.

 

Podcast research and links

Podcast transcript

Beth Eggleston: Before we begin, I’d like to acknowledge the Traditional Custodians and their ancestors of the unceded lands and waters on which we live, work and depend. We recognise all First Nations peoples around the world and celebrate their enduring connections to country, and pay our respects to elders past, present, and emerging.

The last decade has really seen a focus on locally led response to disasters. Our research has charted the course of the localisation agenda, and now we have reached the point where we seek to understand, is it working? Are we meeting the needs of communities in line with their priorities, as intended? More to the point, are we shifting the power to communities and local actors affected by crises?

[Soundbite: Locally-led humanitarian aid: Peter Walker at TEDxLakeGeneva

Yet everything we know about making aid work in complex, highly rapidly changing environments tells us you need feedback all the time. You need context-specific knowledge. You need people on the ground willing to take leadership and risk. And if you want to do anything other than just keep people alive in a crisis, locally-led has got to be part of the bigger picture.]

Beth Eggleston: While the humanitarian sector has made significant strides with various frameworks and tools to support more localised action, there’s still a gap in understanding how these approaches are impacting affected communities. Without a defined pathway and a clear endpoint, the sector is unlikely to make the important shift to more evidence-based localisation practice. Above all, this means understanding impacts at community level – where humanitarian action aims to make a difference. So, how can we gather the evidence to change this?  Over the past three years, the Humanitarian Advisory Group has been exploring ways to measure the impact of localisation at the community level. Our first paper, A Pathway to Localisation Impact: Laying the Foundations, introduced a model to support humanitarian actors to better define and understand the impact of their localisation practices on crisis-affected populations. This was then tested by partnering with organisations implementing humanitarian programmes in Pakistan, Bangladesh and the Philippines, with the research team walking alongside to define and measure changes that took place at the community level because of localised practices. In this episode we’ll chat with members of our partnering research team involved in testing the model in Pakistan. I’m Beth Eggleston, and this is I Think You’re On Mute.

I am delighted to have Aaftab and Zaki Ullah join us all the way from Pakistan! Aaftab and Zaki are close friends of HAG and have been our partners since the first iteration of Humanitarian Horizons. Welcome to you both. Can you please introduce yourselves to our listeners?

Aaftab Ullah: Thank you Beth. I am Aaftab and I’m working with GLOW as theses last three years and in collaboration with HAG I was part of more than 12 assignments. So thank you for inviting us to this broadcast today.

Beth Eggleston: You’re welcome, Aaftab – thank you. And Zaki, over to you.

Zaki Ullah: Thanks, Beth and, first of all, thank you very much for having me on this podcast today. My name is Zaki Ullah, I’m heading an organisation based in Pakistan, GLOW Consultants, which is the research arm, and we are partnering with HAG for the last eight years now and it’s a fantastic journey throughout.

Beth Eggleston: Thank you so much, Zaki. So my first question is for you, Zaki, actually, can you tell us a little bit more about what the framework hopes to do and why impact at the community level really matters?

Zaki Ullah: Yeah thanks Beth for asking this question, a very important one. So let me try to responded comprehensively, so there is a discussion, a lot of discussion, on a buzzword localisation for a while now, there’s a lot of donor funding going there, other organisations working on it, eventually building the capacity of the local actors, local organisation putting them into leadership for distribution. But then at the same time, this a little bit, other discussion that whether is this stage in sector sort of, angle to achieve from this localisation, or do we want to be having something else, for example, a more robust, better response for the affected population? So whether the localisation in the organisation level is an end itself, or is it a meaningful broader objective or impact at the community level? So that was the sort of a background with all this the work we did on the on the framework and eventually we came up with this framework, because previously there was none sort of discussion or a framework how to measure the impact of localisation on the on the community level, everything on the discussions with staff at the, organisation level, the local actor level. So the framework that we developed is in line with the sort of international guidelines standard that humanitarian actors attend to. For example, global humanitarian standards pretty much aligned to what is there. And then it is looking into the localised practices, measuring the impact of having localised practices and its impact on the communities, but also looking into how those localised practices were adopted by the local partners or improved based on the input or the assistance or help provided by the international actors. So that’s the sort of background and the importance of this framework, which is kind of unique.

Beth Eggleston: Absolutely. I agree with you that it’s unique. So Aaftab, we worked together with GLOW on the development of the model and the approach in Pakistan, can you share what some of the key takeaways were from that process?

Aaftab Ullah: Thank you, Beth. As working in collaboration with HAG, we at GLOW for the localisation impact measurement framework, which we checked with the local organisation SHIFA and so that provided us with the renewable insights. Some of the key takeaways or findings, I’d like to mention. The first one that the shelter design and technical skills from an international organisation and then that was more localised in traditional building practices. Like one of the standout examples I would mention here that was in the shelter design process, like SHIFA, they’re working with an INGO, they captured the technical skills from that INGO related to the construction standards and some other safety standards. These skills were then localised with the traditional building practices and materials that were more suitable and familiar to the community. Second one, now I’d mentioned that is the community feedback mechanism. SHIFA with the support from an international partner, they developed a specialised software to systematically capture, track and address the community feedback. So this allowed SHIFA to refine their shelter design based on the real time input from the community and they brought changes after receiving the feedback on ventilation issues in the initial design. They were able to quickly adopt the plan to improve airflow, which was crucial in that hot weather and that specific area. So thank you. These were the initial findings from the framework.

Beth Eggleston: Thank you so much, Aaftab. And I think these examples that you’ve given have really allowed us to see how useful this framework can really be. And Zaki, how do you think we can get this framework that we’ve developed together into the hands of those who will find it the most useful?

Zaki Ullah: Yeah again very important aspect because what we have seen that organisations come up with sort of frameworks or I mean, the research comes up with a very good solution for some problems, but then those solutions were not adopted. And one of the key reasons is the dissemination. Many people, many stakeholders who are actually interested in some sort of solutions, that they are unaware of it, so dissemination – reaching out to the to the ones, to the stakeholders who would be interested. This is something we are doing today as well with this podcast, hopefully more stakeholders, whether they’re the doners who are interested in localisation, the UN agencies, even the local actors themselves, if they want to see and assist in localised practices. So dissemination is the key.

Beth Eggleston: Yeah. Thank you so much, Zaki. And indeed we will be including the link to the framework on the show notes and it’s there and open for everyone to use on our website. Aaftab, I’d love to reflect a little bit more on some of what you were saying. You’ve really being able to paint a picture for us around how useful this has been in a very practical sense. I’m keen to understand, what are your future hopes for this framework and the insights that it will hopefully deliver?

Aaftab Ullah: Our hopes from this framework are there to expanding the evidence-based practices. We hope that the framework continues to provide detailed evidence that supports the effectiveness of the localised approaches in future. Second one that is, is strengthening the local leadership. Like previously, as I give the examples. So, I hope that this framework will empower the local organisations like SHIFA, which were empowered by one of the INGO so that they should empower these organisations to take the lead in the humanitarian response by showcasing their success in enhancing community engagement and accountability. So the framework can help shift power dynamics toward more locally-led initiative.

Beth Eggleston: Thank you. Aaftab and I love what you’re saying, because it’s all about shifting power, that’s absolutely what we’re talking about. And, Zaki, you mentioned that the framework is easy to use. I think this is such a key point. We want our research to, you know, turning to tools and guidance that’s very accessible and easy to use for all different types of organisations. So throughout our partnership, we’ve been able to, co-design this this framework and we’re really looking forward to socialising this with you and really hope that we are able to learn from more insights in the future as we see the framework use in practice. Thank you so much for your insights today. It’s been great to hear from you and to hear what’s been happening with the framework in Pakistan. Thank you so much.

Aaftab & Zaki Ullah Thanks Beth. Thank you. Thank you.

Beth Eggleston: Building on these insights from Zaki and Aftab, we found there were five main lessons when testing the model in Pakistan. One: The approach can be linked to other levels of localisation measurement initiatives to ensure that it aligns with the broader localisation context and to avoid approaches being siloed. Two: There is a need to consider the scale implications and considerations of the framework. The existing domains in the model were helpful in guiding initial discussions and assessments, providing a structured framework for the research team’s consideration of potential areas of impact and influence. Three: The accompaniment process was a useful approach, not only to define outcomes, but to support strengthened capacity and shared understanding of critical monitoring and evaluation approaches. Four: Structured processes and timelines are not central to building strong relationships and engagement with the communities. And five: Incorporate learning from other localisation initiatives to ensure mutual learning and reinforce best practice approaches to localisation measurement.

[Soundbite: Locally-led humanitarian aid: Peter Walker at TEDxLakeGeneva

If we can ally the heavy lifting power and science of the international agencies onto the local knowledge and commitment of the community groups and put a local group in the driving seat, then and only then, will we have an aid system capable of addressing the coming challenges of our more globalised and chaotic world. Locally led aid and true partnerships with international agencies will increase the appropriateness of aid. It will increase the connectedness to the at-risk communities. And ultimately it will increase aid effectiveness. This can happen. It is happening.]

Beth Eggleston: So, with all of this we need to ask ourselves those questions from the beginning of this episode: Are we meeting the needs of communities in line with their priorities, as intended? More to the point, are we shifting the power to communities and local actors affected by crises? More evidence is needed but in the model tested in Pakistan, Bangladesh and the Philippines – yes, there are changes at the community level because of localised practices, the shift is happening. In the next episode, we’ll be reflecting on all we’ve learned through our Humanitarian Horizons programme and sharing our main takeaways from six years of research. I hope you can join me. I’m Beth Eggleston, and this is, I Think You’re On Mute.

 

This podcast, I Think You’re On Mute, is supported by the Australian government through the Department of Foreign Affairs and Trade. The views expressed through this podcast are the presenters alone and are not necessarily the views of the Australian Government.

This podcast was produced and recorded by Room3, a production company that works with not-for-profits and social enterprises, and supported by Green Letter Communications.