Call for Proposals - Final Evaluation of the Action “Community Action for Full Inclusion in Education”, Armenian Caritas BNGO

Armenian Caritas BNGO


Deadline:

November 15, 2018


Opportunity Cover Image - Call for Proposals - Final Evaluation of the Action  “Community Action for Full Inclusion in Education”, Armenian Caritas BNGO

BACKGROUND

Title of the Action to be evaluated

“Community Action for Full Inclusion in Education”

Country / Location

Armenia / Shirak region

Budget of the Action to be evaluated

Total project costs:   EUR 239.980,00€

  • EU Contribution:  EUR 179.985,00€

  • ADC Contribution:  EUR 48.000,00€

  • Caritas Austria (Innsbruck) 11.995,00€

Project/Programme Number

  • European Commission:

NEAR-TS/2016/381-341   “Community Action for Full Inclusion in Education”

  • Austrian Development Agency:

ADC Nr.: 2325-11/2017

  • Caritas Austria:

PNR 1650005  “Community Action for Full Inclusion in Education”

Date of the Action to be evaluated

  • Start: 01.02.2017

  • End: 31.01.2019

The action to be evaluated has the following Overall Objective (OO) and Specific Objectives (SO):

OO: Children with disabilities who are currently out of education enjoy their human rights for inclusive education, as specified in Article 24 of the UNCRPD ( UN Convention on the Rights of people with Disabilities)

SO1: Ensure that children with disabilities who are currently out of education in the region of Shirak receive quality education in their neighbourhood mainstream schools

SO2: Strengthen community capacity to monitor and enforce enrolment of disabled children to school and report on cases of its violation

The target groups include:

(1) An estimated 200 children with disabilities in school age who are out of education in Shirak region

(2) 12 schools (6 urban, 6 rural) in Shirak region (7 communities: 1 urban and 6 rural) with a particular focus on school authority, teachers (350) and all children (3150)

(3) The parents of children with disabilities who do not attend school in Shirak region (estimated 300)

(4) Regional Department of Education in Shirak region

The main activities are:

Activity 1.1. Selection of 12 Schools

Activity 1.2. Conduct a Mapping survey to identify children out of school in the region of Shirak.

Activity 1.3. Establish a Case file for every disabled child out of education.  

Activity 1.4. Aggregate the case files into a Regional roster (with all case files integrated).

Activity 1.5. Translate into Armenian the International Index for Inclusion tool.

Activity 1.6. Provide training to 12 school administration and teachers in Shirak on the Index for inclusion and support schools and their communities to introduce it.

Activity 1.7. Provide counselling to the families of disabled children out of school.

Activity 1.8. Conduct public awareness campaign on the need of inclusive education for all.  

Activity 2.1. Develop a monitoring system to ensure ongoing collection of information about children with disabilities out of education and their school enrolment status.

Activity 2.2. Produce bi annual reports highlighting cases of violation and submit them to the relevant authorities for resolution

For more information please refer to the logframe of this Action and to the project documents that will be made available upon assignment.

DESCRIPTION OF THE EVALUATION ASSIGNMENT

Type of evaluation

Final evaluation

Coverage

the Action(s) in its entirety

Geographic scope

Shirak region – six out of twelve project target schools are located in Gjumri, the other six schools are located in rural areas (Akhurik, Artik, Ashotsk, Azatan, Getq, Maralik)  

Period to be evaluated

the entire period of the Action to date

Objectives of the evaluation 

Systematic and timely evaluation of its programmes and activities is an established priority of the European Commission. The focus of evaluations is on the assessment of achievements, the quality and the results of Actions in the context of an evolving cooperation policy with an increasing emphasis on result-oriented approaches and the contribution towards the implementation of the SDGs.

From this perspective, evaluations should look for evidence of why, whether or how these results are linked to the EU intervention and seek to identify the factors driving or hindering progress.

Evaluations should provide an understanding of the cause and effect links between: inputs and activities, and outputs, outcomes and impacts. Evaluations should serve accountability, decision making, learning and management purposes.

The main objectives of this evaluation are to provide the relevant services of the European Union and the implementing partners with:

  • an overall independent assessment of the past performance of the Action to be evaluated, paying particular attention to its results measured against its expected objectives; and the reasons underpinning such results;

  • key lessons learned, conclusions and related recommendations in order to improve future Actions.

In particular, this evaluation will serve to further develop the supporting actions in the field of inclusive education.  Therefore, the evaluation shall focus on the learning aspects in order to be a good basis for shaping the follow up activities in the respective field.

The main users of this evaluation will be the EU Delegation to Armenia, the Austrian Development Agency, Caritas Austria, Armenian Caritas and the National Association of School Parents Council Members in Armenia; in other words those that are involved in the implementation and the financing of the Action to be evaluated.

Requested services

Scope of the evaluation

The evaluation will assess the Action using the five standard DAC evaluation criteria, namely: relevance, effectiveness, efficiency, sustainability and the perspectives of the impact. In addition, the evaluation will assess two EU specific evaluation criteria:

the EU added value (the extent to which the Action brings additional benefits to what would have resulted from Member States' interventions only);

the coherence of the Action itself, with the EU strategy in Armenia and with other EU policies and Member State Actions.

The evaluation team / evaluator shall furthermore consider whether poverty reduction, RBA, and gender were mainstreamed; the relevant SDGs and their interlinkages were identified; the principle of Leave No-One Behind and the rights-based approach methodology was followed in the identification/formulation documents and the extent to which they have been reflected in the implementation of the Action, its governance and monitoring.

The specific Evaluation Questions as formulated below are indicative. Based on the latter and following initial consultations and document analysis, the evaluation team/evaluator will discuss them with the Evaluation Manager and propose in their Inception Report a complete and finalised set of Evaluation Questions with indication of specific Judgement Criteria and Indicators, as well as the relevant data collection sources and tools.

Once agreed through the approval of the Inception Report, the Evaluation Questions will become contractually binding.
Specific Evaluation Questions

Relevance

  • To what extent are the objectives of the Action still valid for the partner country, the partner organization and the beneficiaries?

  • Are the expected outputs of the Action consistent with the outcomes and the impact (as part of the analysis of the logframe matrix/programme theory and the presentation of the theory of change and its underlying assumptions)?

  • What is the relevant support parents, children, teachers and other stakeholders wish for the future?

Effectiveness

  • What were the major factors influencing the achievement or non-achievement of the outcomes/outputs? Also consider any which were possibly beyond the control of the Action.

  • To what extent have all Action’s stakeholders collaborated as planned?

Efficiency

  • Was the Action implemented in the most efficient way (time, personnel resources)?

(Perspectives of the) Impact

  • What has already changed in the lives of the girls and boys with disabilities (disaggregated data on gender, as well as on the number of girls and boys from poor and extremely poor families)?

  • To what extent became the cultures and practices in target schools more inclusive?

Sustainability

  • What did the implementation organisations or any actor involved in the Action learned from the Action and how has this learning been utilised and disseminated?

  • What are the changes in the attitude of different involved stakeholders/beneficiaries to inclusive education?

Phases of the evaluation and required outputs

The evaluation process will be carried out in three phases:

  • Inception & Desk

  • Field

  • Synthesis & Reporting

Phases of the evaluation

Key activities

Outputs and meetings

Inception  & Desk Phase

  • Initial document/data collection

  • Background analysis

  • Stakeholder analysis

  • Reconstruction of the Intervention Logic, description of the Theory of Change

  • Methodological design of the evaluation (Evaluation Questions with judgement criteria, indicators and methods of data collection and analysis) and evaluation matrix

  • In-depth document analysis (focused on the Evaluation Questions)

  • Methodological design of the Field Phase

  • Scheduling of fieldwork activities (incl. schedule of planned interviews)

  • Kick-off meeting with Caritas project implementing team [in Gyumri]

  • Meeting/interview with the EU Delegation (in Yerevan)

  • Preliminary schedule of Interviews (if these are planned for this phase)

  • Draft Inception report

  • Final Inception report after discussion with / feedback of Caritas project implementing team  

Field Phase

  • Gathering of primary evidence with the use of the most appropriate techniques

  • Data collection and analysis based on the defined Evaluation Questions

  • Intermediary Note

  • Debriefing with the  project implementing team  [in Gyumri]

Synthesis phase & Reporting

  • Final analysis of findings (with focus on the Evaluation Questions)

  • Formulation of the overall assessment, conclusions and recommendations

  • Reporting

  • Draft Final Report

  • Executive Summary according to Annex II

  • Final Report after final meeting with / feedback of the EU Delegation and Caritas project implementing team  and involved expert

  • Slide presentation in Armenian and English  

The main content of each output is described in Chapter 5.

The different phases of the evaluation are explained in more detail in Annex I.

EXPERTISE REQUIRED

Key Qualifications should be:

  • Relevant academic degree (master level) in social sciences and/or education  

  • A minimum of three years’ experience and expertise in the field/sector of education, most ideally of inclusive education

  • Conducted at least three evaluations in the last five years, ideally in the relevant field

  • Knowledge of Armenia with focus on topics such as (inclusive) education

  • Experience in project cycle management

  • Experience in project level evaluations

  • Familiarity with donor funded projects, preferably with EU

  • Experience preparing and analysing a theory of change

  • Experience in social science methods

  • Excellent oral and written Armenian & English skills

  • Sound MS Office and IT skills

The consultants must not have been involved in the design, implementation or monitoring of this project.

DURATION / TIMETABLE

Action

Responsible

Until

Contract signed and documents provided / Kick-Off meeting

Contract signed between Armenian Caritas  and Evaluator

3rd week of November 2018

Inception & Desk Phase

   

Desk Study  / First Interviews (based on a shared preliminary schedule of Interviews) / Submission of draft inception report

Evaluator

2nd week of December 2018

Feedback on Inception report

Evaluation Manager

3rd week of December 2018

Inclusion of comments in inception report + Submission of final inception report

Consultant

4th week of December 2018

Field phase

   

Field Visit, interviews, feedback workshop, etc.

Evaluator

2nd – 4th week of January 2019

Intermediary Note

Evaluator

1st week of February 2019

Synthesis phase

   

Submission of draft final report (incl. draft executive summary)

Evaluator

18th of February 2019

Feedback on draft final report (and executive summary)

Evaluation Manager

28th of February 2019

Inclusion of feedback in final draft report (and executive summary)

Evaluator

1st - 2nd week of March 2019

Submission of final evaluation report, incl. the executive summary and the slide presentation (hard copy and electronic copy) to contractor; in English and Armenian.

Evaluator

15th of March 2019

REPORTING

The outputs must match quality standards. The text of the reports should be illustrated, as appropriate, with maps, graphs and tables; a map of the area of Action is required (to be attached as Annex).

 

Nr. of Pages (excl. annexes)

Main Content

Timing for submission (see timetable under Chapter 4)

Inception Report (including Desk Phase)

15 - 20 pages

  • Intervention logic

  • Stakeholder map

  • Methodology for the evaluation, incl.:

  • Evaluation Matrix: Evaluation Questions, with judgement criteria and indicators, and data analysis and collection methods

  • Field visit approach [including the criteria to select the field visits]

  • Analysis of risks related to the evaluation methodology and mitigation measures

  • Work plan & draft schedule of fieldwork activities

  • Preliminary answers to each Evaluation Question, with indication of the limitations of the available information

  • Data gaps to be addressed, issues still to be covered and hypotheses to be tested during the field visit

End of Inception & Desk phase

Intermediary Note

5 - 10 pages

  • Activities conducted during the field phase

  • Difficulties encountered during the field phase and mitigation measures adopted

  • Key preliminary findings (combining desk and field ones)

End of the Field Phase

Draft Final Report

25 – 35 pages

  • Cf. detailed structure in Annex II

End of Synthesis Phase

Draft Executive Summary

Max 1 page

  • Cf. detailed structure in Annex II

End of Synthesis Phase

Final report

25 – 35 pages

  • Same specifications as of the Draft Final Report, incorporating any comments received from the concerned parties on the draft report that have been accepted

1st of March 2019

Executive Summary

Max. 1 page

  • Same specifications as for the Draft Executive Summary, incorporating any comments received from the concerned parties on the draft report that have been accepted

Together with the final version of the Final Report

Language

All reports shall be submitted in English.
The final report, the final executive summary and the slide presentation shall in addition also be submitted in Armenian.

Formatting of report
All reports will be produced using Font Arial or Times New Roman (letter size 11 and 12 respectively), single spacing, double sided. They will be sent in Word and PDF formats.

Procedure and Logistics
The Evaluator covers all travel expenses within Armenia; as well as all expenses related to printing, copying, data collection and other support services. The Evaluator is solely responsible for the quality of the work to Armenian Caritas.

Armenian Caritas reserves the right not to pay the Contractor or withhold part of the payable amount if one/more requirement(s) established for this assignment are not met or if the deadline set for the accomplishment of the tasks is missed.

TERMS OF APPLICATION

The deadline for the submission of the application is the  15 th  of November 2018. Applications should include:

  • CV

  • Cover Letter: max 300 words, should explain why you think you are qualified for this post and also indicate when you can start to work.

  • Brief Concept Note not exceeding 3 pages describing the approach and suggestions for the evaluation.

  • Proposed budget of all inclusive fee in AMD, including separate lines  for the Consultancy fee and travel costs (e.g travel, translation, data collection assistance, printing etc,), as deemed necessary for the assignment.

Qualified candidates should send the requested documents to info@caritas.am and in CC to Gayane Norikyan (g.norikyan@caritas.am).

Applications that do not include all the required documents will be disqualified.

Applications received after the deadline will not be accepted.

Annex I: Phases of Evaluation

Inception & Desk Phase

This phase aims at structuring the evaluation and clarifying the key issues to be addressed.

The phase will start with initial background study, to be conducted by the evaluators from home. It will then continue with a kick-off session in Gyumri between the project implementing team and the evaluator. Half-day presence of the evaluator is required. The meeting aims at arriving at a clear and shared understanding of the scope of the evaluation, its limitations and feasibility. It also serves to clarify expectations regarding evaluation outputs, the methodology to be used and, where necessary, to pass on additional or latest relevant information.

In the Inception phase, the relevant documents will be reviewed. Further to a desk review, the evaluator, based on the Intervention Logic, will develop a narrative explanation of the logic of the Action that describes how change is expected to happen within the Action, all along its results chain, i.e. Theory of Change. This explanation includes an assessment of the evidence underpinning this logic (especially between outputs and outcomes, and between outcomes and impact), and articulates the assumptions that must hold for the Action to work, as well as identification of the factors most likely to inhibit the change from happening.

Based on the Intervention Logic and the Theory of Change the evaluators will finalise i) the Evaluation Questions with the definition of judgement criteria and indicators, the selection of data collection tools and sources, ii) the evaluation methodology, and iii) the planning of the following phases.

The methodological approach will be represented in an Evaluation Design Matrix, which will be included in the Inception Report. The methodology of the evaluation should be gender sensitive, contemplate the use of sex- and age-disaggregated data and demonstrate how actions have contributed to progress on gender equality.

The work plan for the overall evaluation process will be presented and agreed in this phase; this work plan shall be in line with that proposed in the present ToR. Any modifications shall be justified and agreed with the Evaluation Manager.  

The in-depth desk phase is phase is when the document analysis takes place. Selected interviews with the management and other stakeholders may be realised. The activities to be conducted during this phase should allow for the provision of preliminary responses to each evaluation question, stating the information already gathered and its limitations. They will also identify the issues still to be covered and the preliminary hypotheses to be tested. During this phase the evaluation team shall fine-tune the evaluation tools to be used during the Field Phase.

On the basis of the information collected, the evaluation team should prepare an Inception Report; its content is described in Chapter 5.

Field Phase

The Field Phase aims at validating / changing the preliminary answers formulated during the Desk phase and further completing information through primary research.

If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation or not respecting the end of the validity of the specific contract, these elements are to be immediately discussed with the Evaluation Manager and, regarding the validity of the contract, corrective measures undertaken.

During the field phase, the evaluator shall ensure adequate contact and consultation with, and involvement of the different stakeholders; with the relevant government authorities and agencies. Throughout the mission the evaluator will use the most reliable and appropriate sources of information, respect the rights of individuals to provide information in confidence, and be sensitive to the beliefs and customs of local social and cultural environments.

At the end of the field phase, the evaluation team will summarise its work, analyse the reliability and coverage of data collection, and present preliminary findings in a meeting with programme management.

At the end of the Field Phase an Intermediary Note will be prepared; its content is described in Chapter 5.

Synthesis Phase

This phase is devoted to the preparation by the contractor of two distinct documents: the Executive Summary and the Final Report, whose structures are described in the Annex II; it entails the analysis of the data collected during the desk and field phases to answer the Evaluation Questions and preparation of the overall assessment, conclusions and recommendations of the evaluation.

The evaluator will present, in a single Report with Annexes, their findings, conclusions and recommendations in accordance with the structure in Annex II; a separate Executive Summary will be produced as well (see Annex II).

The evaluator will make sure that:

  • Their assessments are objective and balanced, statements are accurate and evidence-based, and recommendations realistic and clearly targeted.

  • When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place.

The evaluator will deliver and then present in Gyumri the Draft Final Report to discuss the draft findings, conclusions and recommendations. One half day of presence is required.  

The Evaluation Manager consolidates the comments and sends them to the evaluation team for the report revision

The evaluator will then finalise the Final Report and the Executive Summary by addressing the relevant comments.

Annex II: Structure of the Final Report and of the Executive Summary

Executive Summary

A short, tightly-drafted, to-the-point and free-standing Executive Summary. It should focus on the key purpose or issues of the evaluation, outline the main analytical points, and clearly indicate the main conclusions, lessons to be learned and specific recommendations.

The main sections of the evaluation report shall be as follows:

1. Introduction

A description of the Action, of the relevant country/region/sector background and of the evaluation, providing the reader with sufficient methodological explanations to gauge the credibility of the conclusions and to acknowledge limitations or weaknesses, where relevant.

2. Answered questions / Findings

A chapter presenting the answers to the Evaluation Questions, supported by evidence and reasoning.

3. Overall assessment (optional)

A chapter synthesising all answers to Evaluation Questions into an overall assessment of the Action. The detailed structure of the overall assessment should be refined during the evaluation process. The relevant chapter has to articulate all the findings, conclusions and lessons in a way that reflects their importance and facilitates the reading. The structure should not follow the Evaluation Questions, the logical framework or the evaluation criteria.

4. Conclusions and Recommendations

 
 

4.3 Lessons learnt

Lessons learnt generalise findings and translate past experience into relevant knowledge that should support decision making, improve performance and promote the achievement of better results. Ideally, they should support the work of both the relevant European and partner institutions.

 

4.1 Conclusions

This chapter contains the conclusions of the evaluation, organised per evaluation criterion.

In order to allow better communication of the evaluation messages that are addressed to the Commission, a table organising the conclusions by order of importance can be presented, or a paragraph or sub-chapter emphasizing the 3 or 4 major conclusions organised by order of importance, while avoiding being repetitive.  

 

4.2 Recommendations

They are intended to improve or reform the Action in the framework of the cycle under way, or to prepare the design of a new Action for the next cycle.

Recommendations must be clustered and prioritised, and carefully targeted to the appropriate audiences at all levels, especially within the Commission structure.

5. Annexes to the report

The report should include the following annexes:

 

The Terms of Reference of the evaluation

 

The names of the evaluators (CVs can be shown, but summarised and limited to one page per person)

 

Detailed evaluation methodology including: options taken, difficulties encountered and limitations; detail of tools and analyses.

 

Evaluation Matrix

 

Intervention logic / Logical Framework matrices (planned/real and improved/updated)

 

Relevant geographic map(s) where the Action took place

 

List of persons/organisations consulted

 

Literature and documentation consulted

 

Other technical annexes (e.g. statistical analyses, tables of contents and figures, matrix of evidence, databases) as relevant

 

Detailed answer to the Evaluation Questions, judgement criteria and indicators

Full call of proposals read here.



Host Country
Job Type
Publish Date
November 02, 2018




0 Comments