An SDG indicator on education is safe, but the fight isn’t over

Education assessment experts say they are “confident” that a key learning indicator measuring reading and math levels among early-grade school children will not be dropped from the United Nations Sustainable Development Goals monitoring framework — however, not everyone is on board with the plan to save it. The fate of SDG 4.1.1a — which focuses on the proportion of children in grades 2 and 3 “achieving at least a minimum proficiency level” in reading and mathematics, by sex — was put into question last October, creating panic among some in the education community. Indicator 4.1.1a falls within SDG 4, which aims to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” by 2030. The news was presented during a meeting of the United Nations Inter-Agency and Expert Group on SDG Indicators, or IAEG-SDG, in which the body recommended that 4.1.1a be “demoted” from Tier I to Tier II, putting it at risk of being deleted from the SDG reporting framework entirely unless more countries started reporting data. The news sparked an outcry among prominent global education experts and donors who called for swift action to increase reporting and save the indicator, fearing that recent progress toward improving foundational learning in low- and middle-income countries would be lost if the indicator was dropped. It also triggered a fierce online debate over whether and how to save the indicator, centering on which data from existing learning assessments should be included in the global tally. Then in June, the people in charge of the indicator — the UNESCO Institute for Statistics, or UIS, which is the U.N. technical agency that tracks progress toward SDG 4 and other goals related to UNESCO’s mandate — came out with a strategy to “restore” the indicator to Tier I status. Under the plan, UIS is ready to include learning data being generated from a range of existing learning assessments — which were previously not counted toward SDG 4.1.1a reporting due to concerns over reliability and comparability — as long as they meet certain eligibility criteria developed by UIS. Although UIS was unable to confirm to Devex that the indicator is now safe, people close to the process say that these changes mean it won’t be dropped from the SDG reporting framework. “Various discussions … have made UIS fairly confident that the proposal for an alternative strategy to track early grade literacy progress will be accepted [and] that there will be sufficient information to ‘save’ the indicator,” Barbara Bruns, member of the Global Alliance to Monitor Learning, a group of experts convened by UIS to advise on the process, told Devex. Similarly, Armando Ali, CEO of PAL Network, a network of citizen-led learning assessments across India, Pakistan, and East Africa, told Devex that the head of UIS has publicly said that “the possibilities of being downgraded are nonexistent.” A new plan While the indicator may now be safe, aspects of UIS’s plan to restore it — and how the episode has been handled — have generated mistrust among some in the education assessment community. In particular, a group of assessors from the global south have accused UIS of sidelining their assessment efforts and knowledge and overcomplicating the whole issue. The major change announced in the June blog post by UIS is that the agency could now include data from several existing learning assessments — including the Early Grade Reading Assessment, or EGRA, the foundational learning module of the Multiple Indicators Cluster Survey, or MICS, the citizen-led assessments of the PAL Network, as well as a range of national and regional learning surveys — when it compiles its figures for indicator 4.1.1a. This step goes some way to addressing previous criticisms from donors and other stakeholders that UIS needs to be more “pragmatic” about which data it includes to bring reporting levels above the threshold needed to save the indicator. Under the current reporting methodology, data for learning levels for grades 2 and 3 — the grades captured by 4.1.1a — is only available for 20% of countries, according to UIS, when it needs to be at least 40% to meet reporting thresholds set by the U.N. But including data from these other assessments will likely enable UIS to meet the data threshold and report on the percentage of young children who have mastered basic reading and math skills. Knowing where children are in the early grades is crucial, advocates argue, since children who fall behind at this age will find it much harder to catch up. Having data about learning levels at this age, therefore, offers governments time to course correct by 2030. With an estimated 70% of children in low- and middle-income countries in “learning poverty” — unable to read and understand a simple sentence by age 10 — it is vital to keep a pulse on foundational learning levels, advocates argue. Lingering concerns However, UIS’s plan has attracted criticism from some assessment experts from the global south. Soon after the June blog came out, Baela Jamil, CEO of Idara-e-Taleem-o-Aagahi, which leads the Annual Status of Education Report, or ASER, the largest citizen-led household assessment of learning levels, in Pakistan, wrote about her fears that UIS’ new approach was couched in a “language of a new colonisation in assessments,” and accused the agency of sidelining citizen-led assessment and proposing an approach which would render low-income countries dependent on technical experts from the global north. “This recent blog deserves a hard read in the assessment space for foundational learning as we begin to see perhaps inadvertently ‘the global north pushing for language, competencies and more complex layers for literacy (EGRA/Language) to be interpreted and done by a select/chosen few,’” she continued. “I do not want to offend anyone, those who I value a lot but caution is important — the Global South must be on its toes … ‘assessment must not become the new veil of we know better.’” Jamil and others also accuse UIS of overcomplicating the process of reporting on SDG 4.1.1a. “Let us stop moving the goalposts and measure collective progress towards reading and numeracy,” Sara Ruto, former head of the PAL Network, told Devex. Ruto’s point was echoed by Ali, PAL Network’s current CEO, who said: “The awareness about the learning crisis was done using simple tools. It is not the complexity of the tools that will bring the reality of poor learning outcomes in the global south.” But according to Luis Crouch, chair of the UIS governing board and co-author of the June blog, the new UIS eligibility criteria is “not that hard” and assessment organizations are “already responding well,” he told Devex. Furthermore, the short-term pain will be worth it in the long run. “Low learning levels as an issue do not end in 2030. By putting measurement on a more solid basis, we will be better positioned to keep momentum past 2030, and protect the indicator and the goal it implies,” he said. “Things seem complicated now only because the organizations with the new assessments had explicitly not intended them as reporting tools,” he wrote. “They have to retro-fit for reporting. They’ll manage!”