Boston, MA

Boston district leaders turned to Cognia when they needed a set of common assessments that could support different curricula and pacing in place within grades.

Challenges

Multiple curricula, inconsistent data

  • Lack of comparable measurement across multiple curricula and pacing in 125 schools required customized test designs
  • Need for local online assessment aligned to the common state standards
  • Need for assessment literacy and buy-in from district educators
  • Required online assessment and flexible reporting

Solutions

Well-planned pilot, responsive professional development

  • Unified district team from curriculum, assessment, and technology
  • Customized selection and sequence of items for each curriculum
  • Attention to equity
  • Gradual, voluntary implementation
  • Focused professional development on using data

Results

Clear goals and communication made phase-in successful

The new customized assessments are at the right level of rigor for each grade level and tightly aligned to state standards. With these assessments in place, the district can push the conversation beyond individual student learning to look at the broader curriculum and pacing questions, and to identify grade-level gaps at the district level. “As opposed to using the assessments solely as a gauge of how our students are doing, we can now also use the data to explore why students are performing at particular levels,” said the BPS project leader.

Careful planning and input from multiple stakeholders contributed to the success of the project. The gradual phase-in allowed early implementers to lead the way showing how the data could inform instructional practices. Extensive training and feedback gave teachers confidence and familiarity with using the data. Teachers now go beyond merely grading assessments to having a deeper lens into their students’ learning. Plus, students’ exposure to the online platform gave them practice working in an online environment in advance of the requirement to take their statewide test online. Especially helpful to those in lower grades, students had the opportunity to gain confidence in keyboarding skills, navigating an online test, and reading lengthy passages that required scrolling.

In order to ensure that educators in the Boston Public Schools have access to high-quality assessments to inform educator practice and next steps with individual students, we selected Cognia.
Nicole Wagner Lam
Executive Director
Office of Data and Accountability, RFP Review Panel Boston Public Schools

Keys to Success

More than 46 schools participated in the voluntary administrations of Boston’s custom assessments. Administrations occurred in October, February, and April, with increasing numbers of schools participating in each test session, and assessing more than 25 percent of the district’s third- through sixth-graders at its peak. Reflecting on the gradual phase-in approach, Rubino and his team identified several notable aspects of the initial experience.

Gradual implementation

The superintendent and his senior leadership team decided not to require all schools to use the BPS customized assessments during the first year. The idea was that early implementers can lead the way to show how the data can inform instructional decisions. The phase-in also allowed some teachers and students to experience the online platform before it was rolled out citywide.

“The goal is really to improve instruction, with the gradual rollout of customized assessments now available to give us concrete data,” Rubino explained.

One team: Curriculum, assessment, and technology

Implementation of the new online platform was easier than the district anticipated, thanks to a concentrated effort to involve multiple stakeholders early in the process. In October 2016, Boston Public Schools, Measured Progress, and Illuminate staff held eight different meetings for principals to explain the partnership and the project.

“It’s a very seamless relationship in terms of working together with Measured Progress,” said Wagner Lam. “Their attitude,” added Rubino, is always “How can we best support you and your students?”

The kickoff sessions outlined test designs and provided a synopsis of the reasons and rationale for high- quality assessments. Principals saw how high-quality content flowed into clear data reporting on the Illuminate platform. A unified team of Measured Progress experts, technology support specialists, and Boston curriculum and assessment staff all worked together to ensure that the training was meaningful and successfully communicated key information.

Exposing students to technology for skill building and confidence

Technology was a critical success factor in the project. “The key is to prepare students for the technology-based world. There is a concern that technology is increasing the achievement gap,” Rubino said. “We want to close that gap.” The online format of the new assessments gives all students exposure and practice working in an online environment.

This exposure will serve them well, as Massachusetts students will also be required to take MCAS online in 2019. Students gained confidence in keyboarding skills, navigating an online test, and reading lengthy passages that required scrolling.

Rapid response to reporting realizations

One of the early “aha” moments after the first administration was the realization that teachers weren’t quite sure what to do with the reporting results, even though they were enthusiastic about receiving data.

In response, Boston Public Schools set up 20 professional development sessions: 10 focused on data analysis and 10 targeted to specific schools that hadn’t previously used data systematically.

During those sessions, the Office of Data and Accountability team devoted time to showing teachers how to use the assessment data to support potential classroom shifts in instruction. As a result, teachers are set up to move from merely grading assessments to having a deeper lens into their students’ learning.

Match assessment use to purpose for curriculum decisions

The new customized assessments are at the right level of rigor for each grade level and tightly aligned to state standards. With these assessments now in place, the district is ready to push the conversation beyond individual student learning. Now district leaders can look at the broader curriculum questions. For example, they can use the data to inform potential pacing changes recommended by instructional coaches or to identify grade-level gaps at the district level. “Now that the rigor level is correct, we can adjust the curriculum, technology, and teacher support services to get everything lined up,” said Rubino.

He continued, “We will be thinking about the quality of our curriculum and how that will drive teacher professional development. As opposed to using the assessments solely as a gauge of how our students are doing, we can now also use the data to explore why students are performing at particular levels.”

Items are rigorous and culturally relevant to our students.
Michael Rubino
Formative Assessment Manager
Office of Data and Accountability Boston Public Schools