Consultancy: AI-Assisted Analysis of Learning Assessment Data, Montreal, Canada

## Consultancy: AI-Assisted Analysis of Learning Assessment Data, Montreal, Canada - Organization:UNESCO - United Nations Educational, Scientific and Cultural Organization - Office:UNESCO Institute for Statistics, Montreal Type of contract :Consultant Contract Level :Level 3 - Senior Hiring Unit :Institute for Statistics (UIS) Duty Station :Montreal Work location :Remote Duration of contract:4 months Hiring open to :External candidates Application deadline (Midnight UTC-4 Time) :09/10/2026 UNESCO Core Values: Commitment to the Organization, Integrity, Respect for Diversity, Professionalism As the custodian agency for SDG 4 indicators, the UNESCO Institute for Statistics (UIS) supports countries in producing, analysing and reporting data on student learning outcomes to inform education policy, planning and monitoring. Many countries have built up substantial learning assessment data through national, regional and international assessments, often with rich contextual information on students, households, schools and teachers. However, this data is often used only to report national averages, proficiency levels or basic group differences. More detailed analysis of learning distributions, inequality, vulnerability, school variation and contextual factors remains limited. Countries often find it difficult to frame policy-relevant questions, use methods that account for the technical features of assessment data, interpret results soundly, and turn them into actionable policy messages. Recent advances in artificial intelligence (AI) make it possible for national analysts to carry out sophisticated analyses without first becoming experts in specific statistical software. One example is estimating standard errors using balanced repeated replication (BRR) and plausible values (PV). Capacity building can therefore focus on which questions to ask, which analyses are appropriate, how to judge whether results are sound, and how to interpret them for policy. UIS is seeking a consultant to lead a demonstration project in one case-study country, to be confirmed, preferably in Southern or Eastern Africa. The consultant will: (a) produce policy-relevant tables, figures and findings on learning quality and equity from the country's assessment data; (b) develop and field-test an updated analytical framework with AI-assisted analytical protocols that analysts in other countries can apply; (c) strengthen the capacity of national analysts and policy officials; and (d) recommend how the approach can be replicated elsewhere. The work builds on two UIS studies by Douglas J. Willms:Learning Divides: Ten Policy Questions about the Performance and Equity of Schools and Schooling Systems(2006) andLearning Divides: Using Data to Inform Educational Policy(2018). Under the overall authority of the Head of the Foresight, Research and Methodological Innovation (FRM) Section, and in close collaboration with national assessment analysts and policy officials in the case-study country, the consultant will carry out the assignment in three stages. This structure is indicative. Candidates will propose their own detailed methodology and work plan. - Consult UIS and national counterparts to confirm the country, dataset(s) and participants. Prepare an inception note with the analytical plan and work plan. - Review the quality, documentation and limitations of the dataset, including its sample design, and identify which analyses are feasible. - Work with national analysts to identify priority policy questions and the analyses most useful for informing education policy. - Develop and test AI-assisted analytical protocols that let analysts carry out and reproduce the priority analyses with their own data. These must include methods that account for complex sample design and plausible values, such as BRR/PV standard errors. - Work intensively with a small group of national analysts for approximately four days. - Meet senior policy officials to examine the findings, their implications and the policy questions they raise. Consolidation and finalization: - Prepare a policy-relevant country report with statistical tables, figures and findings for national planning and decision-making. - Draft the updated analytical framework as a UIS publication, using the case study and the results of the field test. - Prepare a final report covering lessons learned, an assessment of the demonstration, and recommendations for replicating the approach in other countries. The contract will run for approximately four months, from mid-October 2026 to mid-February 2027, with an estimated workload of no more than 20 working days. The fee will be calculated on the basis of actual working days, up to a maximum of 20 days. It is structured around three stages: preparatory work, an in-country mission, and consolidation and finalization. The in-country mission is expected to take place in late November or early December 2026; exact dates will be agreed with UIS and national counterparts. - Inception note confirming the country, dataset and participants, and setting out the analytical plan, work plan, and the AI tools and methods to be used -by 6 November 2026. - AI-assisted analytical protocols, tested and documented, with guidance on which questions to ask, appropriate analyses, how to check the soundness of results, and how to interpret them for policy -by 22 January 2027, following the in-country mission. - Country report with statistical tables, figures and findings on learning quality and equity. It will include a summary of the policy implications and questions raised with senior policy officials -by 22 January 2027. - Final manuscript of the updated analytical framework, for publication by UIS, which builds on the twoLearning Dividespublications, uses the case-study country as illustration, and incorporates the AI-assisted protocols -by 12 February 2027. - Final report summarizing activities, methods, field-test findings and lessons learned, with an assessment of the demonstration and recommendations for replication in other countries -by 12 February 2027. Editing, layout and publication of the analytical framework will be handled by UIS after the contract ends. Payment will be linked to the satisfactory delivery and acceptance of the deliverables, in accordance with UNESCO procedures, in three instalments: - 20% upon acceptance of the inception note (Deliverable 1); - 40% upon acceptance of the AI-assisted analytical protocols and the country report (Deliverables 2 and 3); - 40% upon acceptance of the analytical framework manuscript and the final report (Deliverables 4 and 5). OWNERSHIP AND AUTHORSHIP All outputs produced under this contract, including the analytical framework, protocols, code and reports, are the property of UNESCO. The consultant will be credited as author of the analytical framework publication in accordance with UIS publication policy, and the contributions of national analysts will be acknowledged. The consultant may not publish, reproduce or use the outputs or the assessment data for any other purpose without UNESCO's written authorization. The consultancy requires international travel to the case-study country for the in-country mission. Travel arrangements will be agreed with UIS in advance and made in accordance with UNESCO rules and procedures. Travel costs will be agreed as a separate lump sum, distinct from the fee, and may be paid in advance. Before travelling, the consultant must complete the UN BSAFE security awareness training and obtain security clearance through the UNDSS Travel Request Information Process (TRIP). COMPETENCIES - Core (C) & Managerial (M) - Knowlegde sharing and continuous improvement (C) - Planning and organizing (C) For detailed information, please consult the UNESCO Competency Framework. REQUIRED QUALIFICATIONS - A PhD or equivalent advanced degree in statistics, econometrics, psychometrics, education or a related field; OR a Master's degree in a related field. A P