Final Impact Evaluation Consultancy

17/12/2024 www.akhtaboot.com
Location :

Amman, Jordan

Job Category :

others

Job requirements

  • 10 - 12

  • Senior (Team Leader/Staff Supervisor)

Description

Job Description and Requirements

Background:

Mercy Corps is a leading global organization powered by the belief that a better world is possible. In disaster, in hardship, in more than 40 countries around the world, we partner to put bold solutions into action — helping people triumph over adversity and build stronger communities from within.

 

Inclusion Security and Resilience Youth Project is a 4 year project that aims to foster resilience, social cohesion, and inclusive security through the three components, focusing on addressing the psychosocial and civic engagement needs of youth, women, and local communities in Jordan.

 

Purpose / Project Description:

The project includes two phases: the initial phase (1st March 2022 to 29th February 2024 referred to by Phase 1) and an extension phase (1st March 2024 to 31st March 2025 referred to by Phase 2). An extensive external mid-line evaluation was conducted for Phase 1, and this evaluation will build on the findings from the mid-line evaluation and assess the sustainability of Phase 1 outcomes while focusing on the Phase 2 interventions.

 

Component 1 (Phase 1 & 2) : Building Resilience Through Mental Health and Psychosocial Support (MHPSS)

This component was designed to enhance the resilience of at-risk youth through transversal MHPSS interventions that addressed their mental, emotional, and social well-being. Key initiatives included the Civic Engagement Program, Nature Club Program, Champions Program, and Support Groups, which aimed to improve youth psychosocial well-being, foster meaningful interactions within their social networks, and cultivate civic engagement skills. These efforts helped place youth on positive pathways toward becoming non-violent, productive, and contributing members of society.

 

To complement youth-focused interventions, this component equipped parents with essential skills to support their children’s social, emotional, and cognitive development, thereby strengthening family units. Programs were implemented by trained community personnel recruited from Community Action Hubs (CAHs) located in Zarqa, Jerash, Sahab, and Karak. This locally grounded approach ensured sustainability, cost-efficiency, and community acceptance. Additionally, Mercy Corps upgraded the infrastructure of the CAHs, transforming them into sustainable, fully operational hubs where youth felt a sense of ownership, belonging, and support. The hubs were enhanced to ensure they were accessible, environmentally friendly, and visually appealing, making them safe and welcoming spaces for youth and community engagement.

 

Furthermore, specialized support was provided to female survivors of gender-based violence (GBV) housed in Dar Amneh protection center in Amman. A trauma-informed therapy intervention focused on enhancing their self-perception, emotional expression, and relationship with their bodies. The second phase provided a holistic support to the female survivors by addressing their psychological, emotional, and physical well-being. The initiative combined personalized therapy, self-care activities, and gentle physical exercises like yoga to enhance self-esteem, foster emotional resilience, and promote overall mental and physical health. By creating a safe and nurturing environment, the intervention aimed to empower survivors, improve their self-perception, and support their journey toward recovery and self-confidence.

 

In collaboration with the Ministry of Youth (MoY), this component included a Holistic Health Programme for youth and parents across 13 youth centers, focusing during the first phase on psychological first aid, sexual and reproductive health, preventing cybercrime, and addressing online bullying and its relationship with stress. These sessions were later extended to 13 additional youth centers across Jordan’s 12 governorates under the ISRY contract extension, significantly expanding the project’s reach, with the exclusion of sexual and reproductive health sessions at the request of the MoY.

 

Component 2 (Phase 1 & 2) : Increased Trust and Gender Equal Partnership between Public Security Department (PSD) and Communities

 

This component focuses on fostering trust and promoting gender equality in partnerships between the Public Security Directorate (PSD) and communities. It achieves this by increasing the PSD's capabilities to engage with communities, improving the effectiveness and inclusivity of dialogue mechanisms, and supporting tangible results through PSD-community partnerships. Component 2 is implemented by a Mercy Corps partner, TAG International.

 

Component 3 (Phase 1): Advancing Youth and Women’s Participation in National Peace and Security Initiatives

 

Implemented during Phase 1, this component focused on improving national-level participation of youth and women in peace and security efforts. In collaboration with the Ministry of Youth, the project supported the development of Jordan’s first National Action Plan (JoNAP) for the implementation of United Nations Security Council Resolution 2250 on Youth, Peace, and Security (YPS). Institutional capacity support was also provided to the newly established YPS Unit at the MoY to ensure alignment with its unit mandate. Due to some constraints, this component had to be discontinued.

 

 

Consultant Objectives:

In line with Mercy Corps' commitment to quality programming and learning, this evaluation aims to build on the findings from the mid-line evaluation of Phase 1, and to assess its’ sustainability while evaluating the interventions and outcomes of Phase 2. The purpose of the evaluation can be summarised as follows:

 

● Process evaluation: The evaluation will assess whether ISRYP’s Phase 2 activities have been implemented as intended. It will provide insights into how ISRYP component 1 in this phase have been most effective, the circumstances and local contexts under which they worked best, and opportunities for improvement in future interventions. Additionally, it will evaluate the sustainability of Phase 1 outcomes for participants, offering valuable reflections on the long-term impact of earlier efforts.

● Outcome evaluation: The evaluation aims to measure the project’s impact on the target population by assessing progress against established indicators. This analysis will enhance accountability and learning, enabling ISRYP and its donor to leverage evidence to evaluate the project's success and identify any shortcomings. Particular attention will be given to understanding how external contextual factors may have influenced the outcomes, ensuring that lessons learned inform strategic planning and the design of future interventions.

 

 

Consultant Activities:

The Consultant will:

● Develop a final methodology for the project final evaluation, including the research approach, data collection tools, and sample size for the different stakeholder groups, and request support from ISRYP in providing contact information for these stakeholders.

● Schedule, organise, and conduct all key data collection activities including Key Informant Interviews (KIIs), in-depth interviews, and/or focus group discussions.

● Provide human resources for all data collection tasks.

● Provide logistical arrangements and bear the financial costs for all data collection tasks.

● Provide transcriptions or incur additional costs for translations from Arabic to English for any qualitative data.

 

Through this final evaluation, ISRYP seeks to build an understanding of the project’s challenges and achievements. Certain evaluation criteria and questions have been identified for inclusion in the final evaluation based on the project proposal, theory of change, and results framework. Others may also be added later as the design of the evaluation progresses. The key criteria and questions, based on OECD’s Better Criteria for Better Evaluation, are presented below for consideration in the ISRYP final evaluation:

 

1. Relevance (Phase 2):

a. To what extent do the project objectives and design respond to participants’, national, global, and local institutions’ needs and priorities?

b. How did the different target groups participate throughout the project cycle and in the construct of activities?

 

2. Effectiveness (Phase 2):

a. Did the project achieve its objectives?

b. What are the unintended consequences (positive and negative) of the intervention (if any)?

c. To what extent was the project agile in responding to extraneous factors and shocks?

d. How effectively was capacity strengthening the Community Action Hubs enhancing the operational, technical, and networking skills of youth to promote the CAH as an accessible safe space for youth and community that engages them in workshops, engagement with government officials in decision making and potentially refer them to work opportunities?

e. What approaches most effectively empower the participation of women in peace and good governance processes at the community level and in local governments?

f. How do trauma-sensitive approaches shape norms and behaviours relate to Mercy Corps’ governance and peacebuilding outcomes?

 

3. Efficiency (Phase 2):

a. How well were resources used?

b. How inclusive and accessible are the CAHs as safe spaces for youth?

c. To what extent did the project deliver results in an economic and timely way?

d. “Economic” is the conversion of inputs (funds, expertise, human resources, time, etc.) into outputs and outcomes, in the most cost-effective way possible, as compared to feasible alternatives in the context. “Timely” delivery is within the intended timeframe. Efficiency indicators and metrics will be agreed with the consultant during the inception phase.

 

4. Impact (Phase 1& 2):

a. What difference did the project make?

b. To what extent did the project generate or is expected to generate significant positive or negative, intended or unintended, higher-level effects?

c. What changes in the capacity, ability, behaviours, and/or attitudes of the target group can be observed?

d. Were benefits distributed fairly between gender, age groups, disability status, and across target locations?

e. How many community-based initiatives have been led or co-led by women as a result of project support? Percentage of champions who report participating in local community initiatives that are not related to the project directly, number of champion community initiatives developed and implemented by the end of project.

 

5. Sustainability (Phase 1):

a. To what extent are net benefits/effects of the project likely to continue beyond project lifetime?

b. Will the project continue to respond to participants’, national, global, and local institutions’ needs and priorities so if circumstances change (particularly if the political, economic, and/or social context changes (positively or negatively) in Jordan in the next 3-5 years)?

These topics are to be addressed for of ISRY’s phase 2, while only Impact Sustainability to be addressed for phase 1 (details to be shared with the consultant during the inception phase).

 

Consultant Deliverables - Timeframe / Schedule:

 

Timeframe

Mercy Corps Jordan is looking to contract a consultant that meets the stated requirements as soon as possible. The deliverables of these terms are due based on the timeframe set on the “Schedule of Tasks and Deliverables” section. The starting date is 19 January 2025, and the ending date is 18 March 2025. The estimated duration for this engagement is 59 working days.

The consultant will be requested to attend a Safeguarding and Community Accountability Reporting Mechanism Orientation to go through Mercy Corps’ Policies and Commitments, prior to signing the contract.

 

Schedule of tasks and deliverables

The consultant is expected to complete the tasks and produce the deliverables listed below. The consultant will be expected to provide a revised schedule during the inception phase, noting that the start and end dates of the consultancy period are fixed, and a timesheet should be submitted monthly as outlined below:

 

Deliverable 1

 

● Mercy Corps Orientation sessions: The consultant will be requested to attend an orientation session on MC policies and standards in regards of Safeguarding & Community Accountability Reporting Mechanism standards (Duration 1 day (19 January)

● Inception Report (Suggested Duration 14 days; timeframe 20 Jan to 2 Feb): The inception report should include a detailed description of the planned evaluation including:

○ Methodology to answer the evaluation questions with the proposed source of information and data collection approaches (to be co-developed with the ISRYP team and the donor)

○ Detailed work plan

○ Desk review summary: Summary of the internal ISRYP documents reviewed by the consultant

○ Primary data collection tools (including interview protocols and guidelines for each group of stakeholders)

 

Deliverable 2

 

● Primary data collection (Suggested Duration 20 days; timeframe 3 Feb to 22 Feb): Consultant will share with ISRYP a list of the identified interviewees and the timeline for the scheduled interviews.

● First Draft of the Evaluation Report (Suggested Duration 7 days; timeframe 23 Feb to 1 Mar): The first draft report should include the following main sections (Page limit: 30-40 pages, excluding annexes):

○ Cover Page

○ List of Acronyms

○ Table of Contents

○ Project Description

○ Methodology and Limitations

○ Findings (for the three components)

○ Conclusions

○ Recommendations (for future projects)

 

Deliverable 3

 

● ISRYP review of First Draft and feedback submission (Suggested Duration 8 days; timeframe 2 Mar to 9 Mar)

● Final Draft of the Evaluation Report (Suggested Duration 8 days; timeframe 10 Mar to 17 Mar): The final draft report should include the following main sections (Page limit: 40-50 pages, excluding annexes):

○ Cover Page

○ List of Acronyms

○ Table of Contents

○ Executive Summary (2-3 pages)

○ Project description

○ Methodology and Limitations

○ Findings (for the 3 components)

○ Conclusions

○ Recommendations (for future projects)

○ Annexes (list of people interviewed, key documents consulted, data collection tools and guides, etc.)

○ Sources & Acknowledgements

● Evaluation presentation (Suggested Duration 1 day; timeframe 18 Mar): The consultant will present the results of the evaluation to ISRYP and the donor.

 

 

The Consultant will report to:

ISRY Monitoring, Evaluation and Learning Coordinator.

 

The Consultant will work closely with:

ISRY Monitoring, Evaluation and Learning Coordinator.

 

Required Experience & Skills:

The evaluation should be led by an individual consultant or a team of individual consultants with an extensive background in project/program evaluation within the INGO/development sector. The successful consultant must be able to work independently within the project structure to deliver the required outputs within a tight deadline and with a large and complex interface with a variety of internal and external stakeholders. In particular, the consultant should demonstrate the following through CVs submitted:

 

● Must hold a bachelor’s degree in research, political science, development studies, economics, or any other relevant field.

● One member of the team must have at least 10 years of professional work experience in the areas of program and impact evaluation, preferably in youth civic engagement, and/or institutional development.

● Extensive conceptual and methodological skills and experience in applying qualitative and quantitative research evaluation methods.

● Extensive experience in qualitative data analysis.

● Experience using quantitative data analysis software tools.

● Experience in facilitating participation in interviews/focus group discussions and in communicating/coordinating with different stakeholders.

● Experience conducting project/program evaluations in Jordan is a must.

● At least one senior member of the team must have full proficiency (speaking, reading, and writing) in Arabic to conduct interviews.

● Fluency in English is a must.

 

Proposal technical requirements

 

Consultants applying for this evaluation will need to bid by submitting a detailed proposal to

jo-recruitment@mercycorps.org by 30 December 2024, with the email subject “ISRY Final Evaluation Consultant”.

The proposal should include all of the following main sections:

● Document 1: Technical proposal outlining a basic methodology and technical approach to the SoW, team composition (if team), and initial timeline/work plan (2-5 pages)

● Document 2: Financial proposal (separated from the technical proposal) in Jordanian dinars (and its equivalent in USD) with a breakdown of budget covering consultancy fees and all related expenses to implement the activities of the consultancy including travel and transportation, accommodation (if needed), all logistical requirements for organising data collection activities, and translation. The financial proposal should be built around the deliverables outlined in the “Schedule of Tasks and Deliverables” section.

● Document 3: 2-3 previous work samples (program evaluation and/or impact research)

● Document 4: CVs of all members of the evaluation team (not more than 3 pages each)

● Document 5: Two-three reference letters (full letters, not just list of references) from previous clients, including their contact information (full name, email address, phone number)

 

Consultant payment schedule

 

Payments shall be made within 30 days from the completion of each deliverable (as approved by Mercy Corps Jordan) and receipt of invoice and completion certificate as proposed below:

• 20% of the payment amount will be made for successful completion, submission and approval of Deliverable 1: Inception Report with final methodology and data collection tools.

• 30% of the payment amount will be made for successful completion, submission and approval of Deliverable 2: First draft of the Evaluation Report.

• 50% of the payment amount will be made for successful completion, submission and approval of Deliverable 3: Final draft of the Evaluation Report with all data sets and transcripts.

 

Evaluation Criteria

 

• Experience and Qualification

Applicant’s experience, contextual knowledge, capabilities, and core research team including clear identification of the roles of the team members who will be part of the assignment.

Submittal : A copy of the candidate’s curriculum vitae demonstrating the relevant experience. Weight 20%

• Evaluation Skills

Track record of final evaluations with a multidisciplinary team. A list of previous evaluations that are relevant to the context and subject matter of this assignment, indicating the role played by the consultant applying.

Submittal: List of previous evaluations (where applying mixed methods are done) in addition to two sample endline reports. Weight 20%

• Facilitation Skills

Ability to facilitate and conduct the final evaluation, and its aspects through the methodology and evaluation plan articulated in the proposal.

Submittal: Submission to the evaluator’s methodology through the proposal. Weight 5%

• Interview

A screening process in which the shortlisted applicant will offer a summary presentation of their approach and team, and MC team will lead an interview with questions related to the assignment and in accordance with this SOW.

Only bidders who score a total of 30% or more in the above technical criteria will be eligible for the interview and the financial evaluation. Weight 15%

• Price Submittal: Financial Offer. Weight 40%

 

Diversity, Equity & Inclusion

Achieving our mission begins with how we build our team and work together. Through our commitment to enriching our organization with people of different origins, beliefs, backgrounds, and ways of thinking, we are better able to leverage the collective power of our teams and solve the world’s most complex challenges. We strive for a culture of trust and respect, where everyone contributes their perspectives and authentic selves, reaches their potential as individuals and teams, and collaborates to do the best work of their lives.

We recognize that diversity and inclusion is a journey, and we are committed to learning, listening and evolving to become more diverse, equitable and inclusive than we are today.

 

Equal Employment Opportunity

We are committed to providing an environment of respect and psychological safety where equal employment opportunities are available to all. We do not engage in or tolerate discrimination on the basis of race, color, gender identity, gender expression, religion, age, sexual orientation, national or ethnic origin, disability (including HIV/AIDS status), marital status, military veteran status or any other protected group in the locations where we work.

 

Safeguarding & Ethics

Mercy Corps team members are expected to support all efforts toward accountability, specifically to our stakeholders and to international standards guiding international relief and development work, while actively engaging communities as equal partners in the design, monitoring and evaluation of our field projects. Team members are expected to conduct themselves in a professional manner and respect local laws, customs and MC's policies, procedures, and values at all times and in all in-country venues.