Emergency Response Meta Evaluation 2017 - 2019
This report details the findings of the Emergency Response Meta Evaluation carried out in 2020 that covered Concern's emergency responses from 2017-2019.
Where we work
Our annual report
The final evaluation of Concern's Emergency Protection and Resilience Building for Refugees in Southeast Turkey project was conducted in October 2020.
Concern was funded by ECHO to implement the 12 month ‘Emergency Protection and Resilience Building for Refugees in Southeast Turkey’ programme, which started on 01/09/2019 and ended on 31/08/2020.
The programme aimed to reduce the exposure to Protection risks of the most vulnerable refugees in Southeast Turkey by enhancing their capacity to access basic and protection services and building their personal resilience. The programme consisted of four components: Outreach; Case Management (CM) and Individual Protection Assistance (IPA); Psychosocial Support (PSS); and COVID-19 response.
The internal, remote evaluation conducted in October 2020 and scored the programme according to the Development Assistance Committee (DAC) criteria as outlined below where: 4 = outstanding performance (above expectations), 3 = performance in line with what would be expected of a well-functioning organisation, 2 = generally acceptable performance but with some clear, and documented, shortcomings, 1 = barely acceptable performance with some major shortcomings and reservations, 0 = totally unacceptable performance or insufficient data to make an assessment.
The evaluation assessed the programme to be relevant and appropriate to the context and the needs. Outreach, information and awareness raising sessions were essential to empower refugees to know and claim their rights. CM and IPA provided tailored protection solutions, aiming to reduce protection risks and meet specific protection needs. PSS and caregiver engagement activities addressed the psychosocial needs of children and contributed to improving their wellbeing.
The adaptations made to the programme in response to the COVID-19 outbreak were timely, relevant and based on a rapid needs assessment that highlighted that people were struggling to meet their basic needs. Distribution of in-kind and cash assistance to vulnerable refugee and Turkish families was therefore deemed a relevant response.
Most programme objectives and results were met; therefore, the programme was assessed to be effective with some documented areas for improvement, particularly in the outreach component. Programme management, Monitoring, Evaluation, Accountability and Learning (MEAL) processes, the information management system, the case management processes were overall considered effective. Issues around data protection were also detected by evaluators. While this did not impact the effectiveness in terms of meeting programme objectives, it does have implications for the quality of the information management system.
The assessment of efficiency showed the programme had procedures and systems in place to facilitate the efficient use of resources. A budget realignment was undertaken, and resources reallocated as a consequence of COVID-19, although the budget was not fully spent at the end of the programme.
The extent to which the programme generated significant or higher-level effects (impact) was assessed through the changes identified by informants. These changes certainly speak for a protection outcome being achieved by the programme in terms of reduction of protection risks for vulnerable individuals and families and in terms of increased psychosocial wellbeing of children targeted by the PSS intervention.
There is strong coherence of the programme within the framework of internal policies, global standards of protection work, national strategic objectives in the framework of the regional response plan as well as standard operating procedures developed by the protection sector at national level. A few minor discrepancies between SOPs and practice were observed.
The assessment of coverage found the programme provided the affected population with assistance and protection proportionate to their needs. The nature of the intervention enabled Concern to address the Protection needs of Syrian and Non-Syrian refugees as well as vulnerable Turkish nationals with COVID-19 adaptions. At the end of the programme, the total number of people supported was 50,604 with outreach activities, COVID-19 adaptions, Caregiver Engagement, Psychosocial Assistance and Individual Protection Assistance / Case Management programming.
The programme was found to be well connected to the broader Protection environment in Turkey and the programme coordination strengthened synergies between sector responses, with strong engagement in response-wide coordination mechanisms, facilitating referrals between departments and promoting an approach geared towards providing holistic support to refugees and other asylum seekers.