HHC UK and HHC Bulgaria is seeking an individual consultant or a consortium to conduct the final evaluation of the project
Deadline for sending questions and clarifications – 22nd December 2020
Deadline for submitting the proposal and budget in response to the ToR – 3rd January 2021
- Background information
Hope and Homes for Children (HHC) is the leading organisation focusing on Deinstitutionalisation (DI) globally – the process of reforming child protection systems from institutional to community and family-based care.
Globally, an estimated eight million children are growing up in institutional care (UNICEF 2009). For every child in institutional care there are thousands more on the brink of separation from their families. Evidence from as early as the 1890s demonstrates the profoundly negative impact of institutional care on children’s physical, emotional and behavioural development (Chapin 1890, Bowlby 1940, Browne 2009). Institutional care leads to intergenerational transmission of poverty. Resources can be more effectively used to combat poverty if reallocated from institutions to community-based support (Bilson and Cox, 2007). Supporting the transition from institutional to family-based child protection provides an opportunity to break the cycle of poverty.
Over 20 years HHC has pioneered its model in Central and Eastern Europe, catalysing regional transformation; reducing the number of children in institutions from over 1.2 million to less than 400,000. HHC played a leading role in the implementation of Bulgarian and Moldovan National Child Protection Strategies, with a specific focus on the deinstitutionalisation of children under 3 and the systemic reform of child protection systems. In Moldova, HHC led the closure of 10 institutions. The overall number of children living in institutions reduced from 13,880 in 2007 to circa 1,000 today. In Bulgaria, since 2012, HHC has led the closure of 16 baby institutions; at national level the number of children under three in institutions reduced from over 2,400 in 32 institutions to circa 400 in 12 remaining institutions. HHC had a critical role in these national achievements; working directly with children in institutions and children and families at risk, with local professionals in the range of stakeholder agencies and with a large number of actors, including NGOs. Direct work with children built an evidence base identifying the main vulnerabilities leading to children’s separation, which interventions are successful and why. It further informed the development of training designed to strengthen the capacity of statutory agencies and service providers to deliver family support and family based alternative care. Evidence collected at institutional care and community level helped inform national policy dialogue and engagement with EU and EC representatives. It helped build national alliances in Bulgaria, ‘Childhood 2025’, and strengthen NGO collaboration in Moldova. HHC’s work was recognised in Bulgaria through the prestigious Tulip Foundation Award and in Moldova – ERSTE Foundation Social Integration Award. Through pan-European advocacy campaign ‘Opening Doors for Europe’s Children’, HHC’s work in Moldova and Bulgaria helped with the inclusion of deinstitutionalisation as a priority at European level.
In Moldova, HHC UK works through its partner – a national NGO CCF Moldova – children, communities, families.
- OAK project description
This is a three year project (1st October 2017 – 30th September 2020) funded by OAK Foundation. OAK’s contribution to the project totals €818,180. The project was implemented in Bulgaria and Moldova (including Transnistria).
The project’s goal was to end the harmful practice of institutionalisation of babies and young children, including those with disabilities. In line with Bulgaria’s Action Plan for the national strategy ‘Vision for deinstitutionalising the children of Republic of Bulgaria’, and Moldova’s, 2014-2020 national Child Protection Strategy and 2016-2020 Action Plan, the project sought to play a significant role in reforming the child protection systems in Bulgaria, Moldova and the region.
This project entails five interrelated work streams:
- Supporting and influencing the implementation of the law and policy development to accelerate child protection reform, end institutionalisation of children under 3 and set quality standards for prevention and alternative care services for all children;
- Influencing and shifting norms, values and attitudes to ensure prevention of separation and that no child is placed in institutional care;
- Providing parent and caregiver support, ensuring access to quality, appropriate and accessible services to prevent child and family separation in 7 districts in Bulgaria and 28 districts in Moldova and the Transnistria region;
- Strengthening, developing and ensuring adequate service provision for the deinstitutionalisation and prevention of separation of babies and young children in 7 districts in Bulgaria and 28 districts in Moldova and the Transnistria region;
- Using innovation, developing a pathway for global collaborative action to champion child protection and care as a global priority.
- Key purpose and objectives of the evaluation
These terms of reference outline the key purpose and objectives, expected methodology and deliverables of the evaluation. The evaluation is expected to highlight the project results, impact and the processes which led to these, key lessons learnt, and the sustainability of the project benefits.
- Evaluation purpose
The main purpose of the external final evaluation of this project is to generate learning about strategies developed to enable Bulgaria to advance towards ending institutional care for young children and prevent children’s separation from their families. A special attention will be paid to documenting the impact of HHC’s work on the continuation of child care reform in both national contexts.
The evaluation report will also be used to inform OAK’s understanding of HHC’s performance at the project level and to inform how OAK’s support has contributed towards the wider achievements in childcare reform in the two targets countries.
- Evaluation objectives:
- Identify and document lessons learned and include recommendations for improvement for HHC’s ongoing work on DI and gatekeeping;
- To review to what extent the project has been implemented in line with HHC’s principles; and
- To assess to what extent the project has achieved its intended outcomes/impact. Assessment of the project’s outcome will be in line with OECD-DAC Criteria, and include sub-questions on:
Relevance – have we done the right thing?
Effectiveness – have we done the right thing well?
Efficiency (and value for money) – have we got the most (and best) results for our inputs?
Impact – what changes (positive or negative, intended or unintended) to beneficiaries’ lives has the project contributed to?
Sustainability – will what we have done remain after the project ends?
6. Use and users of the evaluation
The primary intended users of this evaluation are HHC’s programme and policy staff and senior managers. It is expected that the evaluation report will be used by programme staff to deepen their understanding of effective strategies to enable communities to prevent children’s separation and reduce reliance on institutional care; by HHC policy staff to support evidence-based policy and advocacy activities; and by senior managers within HHC to make informed decisions about strategic priorities. In addition, the evaluation report will be shared with OAK to evidence how OAK funding has been spent. The report will also be shared with HHC partners and with other external stakeholders, government and other NGOs etc., as appropriate.
- Evaluation questions
The evaluator(s) should adapt and respond to the questions below. All evaluators are encouraged to structure their research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, impact and sustainability. In fulfilling the aims and objectives of this Terms of Reference, the evaluation is also expected to address and answer the following questions:
- To what extent has the project been able to realise its intended outcomes? (related to all outputs and outcome level changes and assessed in line the aforementioned OECD-DAC Criteria)
- To what extent have the knowledge, attitudes and practice of different stakeholders, including project beneficiaries, changed and what has brought about any changes?
- What is the impact of HHC’s work on the continuation of child care reforms in both national contexts? How the impact can be maximised and what are the lessons learnt?
- Evaluation methods
An evaluator will be recruited to design the overall approach and methodology for the evaluation. The overall approach will be a mixture of qualitative and quantitative data from primary and secondary sources in Bulgaria. The evaluator is expected to use a variety of methods to collect and analyse data/information. We welcome suggestions for methodological approaches that address the objectives and evaluation questions set out in these Terms of Reference.
The approach will include the following:
- Inception phase. The inception phase will include introductory briefings with HHC Bulgaria and UK Technical Adviser, and a literature review of existing project and contextual documents, existing external evaluations. During the inception phase, through an inception report, the evaluator will be expected to develop a detailed methodology, data collection tools and work plan, including but not limited to:
- Reviewing existing data/information to identify key indicators for which comparable data is available and that can be used to demonstrate changes over time;
- Finalise and agree overall methodological design and approach for the evaluation, including details of sampling strategy, size and analytical framework;
- Finalise management and coordination arrangements for the evaluation in both countries;
- Adapt/develop specific data collection tools or processes (i.e. online discussion guides etc);
- Develop clear and simple ethical protocol and guidelines for the evaluation.
Various information/data sets already exist for the evaluation and it is expected the evaluator will draw on the information/data that is already available. If additional data is required this should be agreed in consultation with relevant HHC staff and partners.
- Interviews with relevant stakeholders, including beneficiaries in both countries.
- Production of draft and final evaluation reports.
9. Evaluation Standards
HHC expects all evaluations to be carried out in line with HHC best practice standards. These standards include the need for the evaluation and the evaluator to:
- Feedback: the evaluator should as a minimum commit to feeding key findings back to HHC and partners involved in the evaluation before including them in the final report. Options for validation of results and feeding back final evaluation findings must be considered.
- Commit to transparency and ethical standards: the evaluator should explain clearly to partners and all those involved in the evaluation what the purpose of the exercise is, and how people’s information will be used. The evaluation should follow HHC’s ethical standards for research and data collection. As a minimum the evaluation should ‘do no harm’.
- Listen to stakeholders’ voices: it is essential that the evaluation reflects the voices of all stakeholders involved in the project. People should be consulted as part of the evaluation, and their voices should be included in the evaluation report as direct quotes and case studies. In line with HHC’s mandate the evaluation should prioritise families living in poverty and children.
- Be transparent about methodology: the evaluation should include a detailed and transparent discussion of the methodology used and key decisions taken in designing and implementing the evaluation. This should include information about the numbers of stakeholders covered, to what extent it was representative, what tools were used and why, methodological limitations and gaps in the evaluation.
- Contractual and reporting arrangements
HHC is looking for an evaluator/evaluators to lead the evaluation of the OAK-funded project across two countries. The evaluator(s) will develop the methodology and tools for the project evaluation in the two countries, in consultation with HHC/CCF programmes staff. They will be responsible for consolidating all data collected, analysing it, and producing a final evaluation report.
The evaluator for Bulgaria will report to HHC Bulgaria country director and UK Technical Adviser and will be supported by the programmes team. HHC Bulgaria will lead the coordination of the evaluation in Bulgaria, and will provide all necessary project information; introductions to relevant stakeholders; and any general logistical support and advice.
Timetable
The OAK grant finished on 30th of September 2020 and the evaluation needs to be completed by the 29th of March 2020, giving April to compile the information and produce a final report.
11. Outputs/Deliverables of the Evaluation
In line with the purpose and methodology of the evaluation, the evaluator will deliver the following outputs:
- Inception report with detailed methodology, tools and work plan;
- Meetings/Calls with stakeholders debriefing (as a note or a presentation remotely);
- Draft evaluation report;
- Final evaluation report that incorporates evidence, learning, case studies, feedback received from HHC and partners and meets agreed quality standards. The report should be precise, must answer each evaluation objective and question and should at least contain the following (this can be discussed within the inception phase):
- Cover page(title of the evaluation report, date, name of evaluator).
- Contents table.
- Executive summary(of no more than 2 pages outlining the key purpose of the evaluation, main points of analysis, key findings, conclusions and recommendations).
- Introduction(outlining the background to the organisation and project; purpose and objectives of the evaluation; logic and assumptions of the evaluation; overview of OAK funded activities.
- Evaluation Methodology (evaluation plan; strengths and weaknesses of selected design and research methods; summary of problems and issues encountered and limitations of the evaluation).
- Findings (overall results; assessment of accuracy of reported results; relevance; effectiveness; efficiency; sustainability; and impact).
- Conclusions (summary of achievements against evaluation questions; overall impact and value for money of OAK funded activities)
- Lessons learnt (project level – management, design, implementation; policy level; sector level).
- Recommendations.
- Annexes(such as independent final evaluation terms of reference; evaluation research schedule; evaluation framework; data/information collection tools; list of people consulted; list of supporting documentary information; HHC management response* to report findings and recommendations).
- Draft summary report(summarising key findings and recommendations, up to 4 pages) that can be used to disseminate findings, written in English, in clear and plain language and style suitable for all stakeholders.
- The raw data(all transcripts, quantitative data) must be handed over to HHC together with the evaluation report.
The report should not exceed 30 pages, without annexes.
*Management Response: The report will be followed by a management response from HHC, outlining areas that we agree with and will take forward in future responses; responding to areas highlighted as requiring improvement; outlining any findings that we disagree with which have not been resolved through providing comments on the draft report, and indicating how learning will be taken on board in this and future responses.
- Profile of the Independent evaluation provider
The evaluator should be a suitably-qualified and experienced consultant. The consultant profile should include:
- An evaluation specialist with a minimum of 5 years’ experience in programme/project evaluation in social field;
- Experience of results-based monitoring and evaluation;
- Ability to design and plan the evaluation approaches and research methodologies, including quantitative and qualitative research methods;
- Understanding of family and community strengthening strategies, deinstitutionalisation, child protection system reform;
- Substantial experience conducting qualitative research and data analysis;
- Understanding of and commitment to ethical issues in research/evaluations;
- Experience in managing and coordinating evaluation/research exercises, including with or through country-based partners, delivering agreed outputs on time and on budget;
- Ability to write high quality, clear, concise reports in English;
- Appropriate country knowledge/experience. This includes language proficiency to conduct the research required or that resources be made available (e.g. translator or social gatekeeper) to enable the research to proceed smoothly;
- No conflict of interest with the on-going activities of HHC and partners;
- Experience in programme/project evaluation in an international development context will be considered an advantage;
- A financial proposal that offers good value for money and that maximises potential efficiencies to deliver the outputs within budget.
The selected evaluator will be expected to sign and abide by HHC key policies.
- How to apply
We invite interested individuals to submit the following application documents:
- CV of the evaluator who will undertake the research and evaluation (maximum 3 sides of A4);
- Proposal (maximum 4 sides of A4) detailing a) how the evaluator meets the selection criteria and b) their understanding of the Terms of Reference and methodology as well as the evaluation budget.
- Please ensure your proposal explicitly answers the following questions:
- What evaluation approach and methods do you suggest to answer the evaluation questions given in the Terms of Reference?
- How will your proposed approach involve the stakeholders participating in the project in the project countries?
- What risks do you foresee in relation to this evaluation consultancy and how will you mitigate them?
- One recent example of similar evaluation report written by the applicant (if joint authored to include a description of the role of the named evaluator in the report);
- Contact details of two independent referees.
Please send your applications to:
The deadline for applying is 3rd January 2021 to start the contract in mid-January, to be completed by mid-April.
- Terms of payment
Payment will be done according to an agreed time frame/arrangement:
Instalments | Amount | Time line |
1st instalment | 30% of the total amount | After inception meeting and sharing inception report from the evaluator with detailed methodology, tools and work plan |
2nd instalment | 50% of the total amount | After presenting data/information collection summary and sending draft report |
3rd instalment | 20% of the total amount | After satisfactory completion of final report according to agreed quality criteria and summary external report |
- Lines of communication:
HHC will conduct an initial briefing for the evaluator. Subsequent communications will be with programme staff based in the field. The evaluator will report to HHC Bulgaria Country Director Georgy Simeonov and HHC UK Technical Adviser Galina Bisset. Contact details will be provided.