Measuring What Matters During a Pandemic

That included the continuation of school meals, access to technology for remote learning, social-emotional support, and connecting families with local services. All this while continuing with academics and trying to retain some normalcy about what a school day looked like. 

During an ordinary year, students would have spent the last few months of the school year mastering the remaining grade-level standards and preparing for end-of-year state testing. But due to the unprecedented situation, the federal education department waived state testing requirements for the 2019-2020 school year. So rather than introducing students to new standards while simultaneously navigating an abrupt transition to remote learning, teachers at University Academy focused on helping students retain the knowledge they had gained before school buildings closed.

“It's not the students' fault that they don't have internet and they don't have a device, and they have to wait two weeks to get a hotspot,” Knapp says. “So we take all of those things into account because you're trying to set a kid up for success, not punish them for something they can't control.” 

School systems across the country had a very similar ending to a tumultuous school year, skipping end-of-year standardized tests and focusing instead on students’ primary needs and learning retention. 

With statewide testing canceled last spring, many state and local education leaders still do not have a full picture of the severity of the pandemic’s impact on student learning. Projections from education analysts suggest school building closures could cause academic slide and lasting learning losses that will disproportionately impact students of color and low-income students who may not have equitable access to basic necessities, technology, quality remote instruction, and support to learn from home. 

Student data is critical for decision-making and school accountability, especially when so much about the pandemic’s impact on students is unknown. But due to the disparities between students who have access to technology and resources for remote learning and those who don’t, some experts have questioned whether using standardized tests to assess students’ learning right now would actually benefit students or produce reliable data. A recent report from the Economic Policy Institute states that “such tests could overwhelm or label children when what they need now are diagnostic assessments and needs-based assessments that assess where they are across a range of domains and what they need going forward.”

As education experts continue to debate the potential value and limitations of federally mandated testing in general given the widening equity gaps, schools still need a way to evaluate how students are doing academically right now, as well as assessing other needs. Among the recommendations in a report from the Center for Reinventing Public Education, a panel of assessment experts suggest “the priority should be on assessments closest to classroom instruction that help teachers know what to do next. To the extent possible, systems should be using the assessments they already have.” 

With another uncertain school year underway, many school systems are doing just that: using common diagnostic and benchmark assessments as a way to quickly identify students’ learning needs and guide instruction this fall. Now, a few months into the new school year, some insights are starting to emerge. In a new report from NWEA, researchers analyzed the results of more than 4 million students in grades 3-8 who took the MAP Growth assessment during the 2019-20 school year and again this fall. Overall, students in the sample are making progress in reading and math, albeit, at a slower pace than during a normal year. Students’ reading growth is nearly on par with a typical school year, while close to a third of students have moved down a quintile or more in math since school buildings closed. Students attending higher poverty schools have comparatively lower test scores in reading and math and have fallen further behind this year.

While the report suggests “COVID slide” may not as dramatic as predicted, there is an alarming caveat to the finding: 25 percent of students in the sample who took the MAP test last winter are unaccounted for this year. This includes a disproportionate number of students from marginalized communities who may be struggling to access remote learning, chronically absent, and possibly disengaged from school for economic or health reasons.

We still have a lot to learn about the full extent of the pandemic’s impact on students learning and these assessments can help school leaders gather valuable data on student growth and adjust plans accordingly. But navigating these tests while social distancing presents a number of practical challenges that schools have never encountered before. How do you administer an assessment for kindergarteners over Zoom? How do you make assessments equitable for students who may not have access to reliable internet or a quiet environment at home? And how do you help students recover academic ground while also staying focused on their social-emotional wellbeing—all in a remote environment?

Supporting Young Learners With Virtual Testing

At University Academy, Knapp and her team spent the first few weeks of the school year helping students and families get acclimated to distance learning and assessing whether they had the devices, technology, and support to connect from home. Knapp met with parents, caretakers, and even daycare providers to make sure students would be supported with remote learning.

Later in the fall, students completed their first round of virtual formative assessments. Knowing that young learners would need extra assistance, Knapp and her team met individually with each of their K-2 students for live one-on-one sessions over Zoom, often over multiple 20-minute increments. The team used an in-house assessment for math and The Fountas & Pinnell Benchmark Assessment for reading. Teachers and staff presented students with questions on math and English language arts, and students talked through how they arrived at their answers. 

Knapp and her team work together as a tight-knit unit, so she has a pretty good pulse on where teachers left off last spring, and which standards they didn’t cover. And for the most part, she is relieved at seeing the initial results from these assessments, given the disruptions last spring. Now that she has some baseline data, Knapp says she is focused on student growth this year, rather than what learning they missed.

“We're not doing that because that does nothing but tear down the success that students have had,” Knapp says. “This is where they are, period. This is where they're going to be in May of next year. And here's how we're going to get there.”

About ten miles north of University Academy at Kansas City Girls Prep, School Leader, Tara Haskins is also focused on assessing academic growth, the amount of learning her fifth and sixth graders will need to gain over the course of a year. A few weeks into the new school year, students took the NWEA MAP assessment to measure their growth since last spring.

“I want a student to be able to say, this is where I started, and look how much I've grown. And I want them to be able to connect the habits that help them get there,” Haskins says.

This was also the first time students at Kansas City Girls Prep took these assessments in a remote environment. In order to ensure things ran smoothly, they spent time preparing during advisory period working online in small groups or “prides” to run through the instructions, what to expect, and how to get help. The school team was also mindful of students’ test anxiety and held space for students to discuss their feelings about test-taking, especially in a remote environment. During these discussions, teachers were intentional about framing the test as a way to help students grow, not a reflection of their self-worth.

Students took the test online over Zoom together with their pride, while each pride leader proctored the test. Students could work one-on-one with their pride leader in a breakout room if they needed tech support. The school offered make-up days for students who ran into technical issues or needed extra time to find a quiet space for testing. 

After reviewing the data, Haskins is not surprised by seeing some of the skill gaps, such as reading which is an area where students also struggled last year. She’s also seeing that many students are on track to meeting their growth goals from last year. Haskins is encouraging her staff to celebrate the progress students have made and use this data to set individual growth goals for the year ahead.   

“I'm seeing some amazing things, despite the fact that they missed seven months of learning,” Haskins says. “Some students met their yearly growth and they haven't been in school in seven months. This is incredible. And I can't wait for them to see it and feel it and recognize it.”

Testing Out In-Person Assessments

At E.D. Walker Middle School in Dallas, Principal Ashley Toole (Dallas-Forth Worth ‘11) says her school has had multiple “openings” so far this school year. The first was in early September when students began the first four weeks online. As the school navigated plans to reopen for a hybrid of face-to-face and remote learning, the district’s MAP testing was an opportunity to diagnose students’ academic growth since last spring, as well as pilot bringing students back in person in small groups, or what Toole refers to as the “second” school opening.

While there were some logistical hurdles to overcome, the benefit of having students take the MAP test in person at school meant all students had access to the same environment, resources, and support to take the test. Leading up to the test, teachers worked with small groups of students from their advisory period called “crews” and helped students prepare by going over the safety guidelines such as arrival procedures and bathroom breaks. Everyone was required to wear a mask (the district provided every student with three reusable cloth masks). The district also issued students a portable plexiglass divider to use at their desks, which are carefully spaced six feet apart. In order to maintain social distancing guidelines, each crew took turns coming to school to take the test on a different day while the rest of the students continued online learning at home. The school librarian proctored the test remotely for the small number of students who were unable or did not feel safe coming to school.

The whole process took a few weeks. But Toole and her team now have a pulse on how students are doing. Ordinarily, students would be on an accelerated trajectory with math standards, where seventh graders take pre-AP math and eighth-graders take algebra. After the transition to remote learning last year, Toole is seeing gaps in what students retained from last year, and in standards that weren’t covered, particularly in seventh grade math. This will be an area where students will need targeted support.

“I think the summer off paired with the long spring break—let's call it that—it did affect a lot of regression from where we expected students to be in the spring,” Toole says.

'Grace Before Grades'

Students at Franklin County Early College High School (FCECHS) in North Carolina also completed their district benchmark exams at the beginning of the school year, only this time online. Dr. Erica Shoulders-Royster (Eastern North Carolina '98), the school's principal, is seeing that some scores have dipped. Still, she is relieved to see that math scores are relatively consistent with last year. On average students were just short of the 70 percent pass rate by two points. 

“Honestly, I feel like that's good considering the circumstances and the situation we're under. I think by the second benchmark or even the third one, I think we'll be well above our goals,” Shoulders-Royster says.

Using this benchmark data, Shoulders-Royster is working with her team to set learning goals for the year. She’s also keeping close tabs on students’ individual situations and what support they will need to meet their goals. There are pockets of kids at FCECHS who are feeling overwhelmed, stressed, and unmotivated. Shoulders-Royster estimates about 10 percent of her students aren't regularly logging into class. Some live in remote areas where even if they have a hotspot, cell service is spotty. Many older students have increased their responsibilities at home, caring for siblings, and taking on part-time work during school hours in order to support family members impacted by unemployment. 

Shoulders-Roysters works closely with teachers to identify students who are disengaged and reaches out personally, to let them know that she’s someone they can talk to.

“I want students to know while I am concerned about your grades. I'm more concerned about you as an individual and as a person. That matters more to me than anything else,” Shoulders-Royster says.

She encourages students to not be hard on themselves and to focus on self-care and alleviating stress. She advises students to tape a positive quote to the bathroom mirror as a way to stay grounded. For Shoulders-Royster, it’s a quote from Dr. Brad Johnson, a veteran educator and author of several books on school leadership: “Relationships before rigor. Grace before grades. Patience before programs. Love before lessons.”

"That’s been part of my mantra for this year," Shoulders-Royster says. "I truly believe that we all could benefit from that this school year."

Focusing on Growth and Targeted Support

In a recent report from the Center on Reinventing Public Education, assessment experts recommend that this is a year for schools to rethink how they structure learning in response to the prolonged disruption to in-person learning time. That could include flexible schedules, combining students from multiple grades, and small-group and independent learning, in order to close gaps in learning from the spring and summer.

At University Academy, Knapp and her team are putting these ideas into practice and pulling out all the stops to help young elementary students reach their growth goals for this school year.

The school offered a virtual summer school program to help continuing students target learning gaps and get a jump start on the year ahead. Incoming kindergarteners also attended a few weeks of virtual “bootcamp,” which helped  Knapp and her team make informed decisions about matching students with teachers who will be most effective based on students’ individual needs. 

With the new school year underway, remote instruction continues. Students attend live virtual instruction with their class in the mornings and then meet with a specialist in the afternoons for one-on-one or small group instruction to focus on skill development. Knapp describes her team as “Renaissance” staff, who have stepped into new roles to fill emerging needs. Administrators and support staff pitch-in to lead small group learning so that every student has access to personalized support.

“I have TAs, a resource teacher, and other staff who, within a remote environment, it's like, 'okay, well, what am I supposed to do?' So now they do assessments, small groups, and one-on-one learning. They do all of it,” Knapp says.

In Dallas, Toole is also piloting new ways to structure the school day to help students who need to make the greatest academic gains. Toole and her teachers plan to use the MAP results from this fall to break down every student’s growth goal. She’s particularly interested in identifying which specific students and how many of them will need to exceed their goals in order to move on to the next grade level.

“That's where I start thinking about that equity lens. Which of those kids can we put with the really high performing teachers and in an intervention class,” Toole says.

In this case, Toole is thinking about pairing her veteran teacher-coaches with students who need to make the most progress on their growth goals. She’s also looking into whether the students who had a low score on the MAP test are also struggling with remote learning. And if they are, she’s considering working with the counselor and assistant principal to see if those families would be open to having their children return to in-person school. It’s challenging because many families, especially those with caretakers who are able to work from home, are understandably wary of sending their kids back to school.

“I've had many parents say we're waiting until the second nine weeks to come. Some people are waiting to see how safe it really is,” Toole says.

Students’ Wellbeing Is Priority #1

School leaders are balancing multiple needs, both academic and social-emotional as students are experiencing multiple traumas and uncertainties surrounding the pandemic, national uprisings against systemic racism, and the election. In Franklin County, Shoulders-Royster and her team at FCECHS are keeping a constant pulse on students’ mental health and wellness.

She meets with student government leaders every other week to seek their input on activities for engaging students and leads a biweekly virtual town hall to support parents with remote learning. In the virtual classroom, her freshman civics teacher weaves current events into the curriculum and makes space for students to discuss national events. The science teacher gives students virtual exit tickets on which they answer a question about their wellness habits. Then she follows up individually with each student to see how they are doing and offers encouragement. 

“That personal connection is what we have thrived on and done really well with over the past couple of years. And that has contributed to our school success,” says Shoulders-Royster.

At Kansas City Girls Prep Haskins hopes to offer more social-emotional supports that are organic and responsive to students' needs, rather than always being led by a staff committee. She’s sending around a survey to ask students for feedback on what supports and experiences would be most meaningful to them.

“It's trying to find a happy medium. Kids always know what they want. They may not know exactly how it looks. In my mind, it's the role of a school to connect with the community and make it happen,” Haskins says.

Rethinking Learning for Long-Term Retention

As Haskins looks to the year ahead, she’s focused on living into the school’s commitments, including “leave no community member behind.” The school is prioritizing family engagement and setting goals for bi-monthly phone calls as well as sharing real-time data on attendance and grading.

On the academic side, Haskins’s goal for her students is that 100 percent will reach 1.25 years of growth in ELA, math, and science. There are a ton of challenges and uncertainties ahead with remote learning. But having some diagnostic data on students’ growth is giving her hope and a clearer picture of where to go from here.

“If we did this while missing a whole unit of instruction during a pandemic when we knew less in March than we do now, I fundamentally believe this can happen,” Haskins says.

There is pressure to cover a lot of ground this year in order to ensure that kids don’t fall behind. Haskins recalls a moment where she realized a relentless sense of urgency, which is often rooted in the education system's long history with white supremacy culture, might be getting in the way of sharing more decision-making power with teachers, students, and families about how to best meet students’ needs this year. While this moment demands so much from teachers, Haskins wonders: What would it look like this year for teachers to slow down?

Haskins and her staff are rethinking how they approach the year ahead to ensure students have a rich learning experience and aren’t rushing through standards. That means focusing on quality over quantity, and going deep on certain areas so her students "don’t come up empty."

She’s planning to use her students’ fall assessment data to figure out where the team needs to proactively take more time, and where they can speed up and build on the knowledge that students already have.

“I love our team for cautiously holding that mirror up and helping us realize—how can we do less, better?” Haskins says. “What if we slow down for one lesson and go deep and kids really master it?"

"If we really want growth, that is what we need to do.”

This story was updated on December 1, 2020, to include results of the December 2020 NWEA study.