Standard Setting Validation
ACS TEAM ON PROJECT
-
Chad Buckendahl, Ph.D., Partner
-
Andrew Wiley, Ph.D., Partner
-
Russ Keglovits, MS. Ed. Assessment Specialist
CLIENT
Educational Test Publisher in service to a U.S. State Department of Education
ACS was subcontracted by an educational publishing company to complete an independent standards validation for their high school alternate assessments in one U.S. State.
PROJECT
More than 30 educators from across the state were convened as panels to review the alternate high school assessment in 1) English, 2) Mathematics, 3) Science, and 4) Social Studies. The project was initiated due to some content shifts that had occurred in the test content and was completed in the summer of 2022. Panelists reviewed the current examinations and evaluated whether the current cut scores remained appropriate. ACS developed all materials, facilitated the workshop and produced comprehensive documentation for all activities.
UNIQUE VALUE
The development of a standards validation model can provide critical information for educational organizations to consider when making small modifications to their assessment program. When making more minor adjustments, a difficult question is whether the changes require updated cut scores. Establishing cut scores from scratch is a time-consuming and costly activity. Completing a validation activity can be a less taxing method for organizations to investigate and determine if the current cut score(s) are appropriate or if they need to be adjusted to reflect changes in the program or testing population.
SUSTAINABILITY
ACS provided comprehensive documentation that allowed the State to determine if the current cut scores were still appropriate based on input from their local experts. The State was able to use the resulting documents and reports to facilitate discussions across the state with critical representatives and to make a final judgment on what, if any, changes were necessary in the cut scores.