Evaluating changes in pass rates for a certification program
ACS TEAM ON PROJECT
-
Susan Davis-Becker, Ph.D., Partner
-
Jaime Kavanaugh, MS. Ed., Psychometrician
CLIENT
Certification client in Personal Training
PROJECT
The Client observed declines in passing rates that raised concerns for stakeholders. To evaluate factors that contributed to the observations, ACS designed a study to collect information about why pass rates may have declined for the program, if any of the cause(s) should be addressed by program leadership, and, how to communicate the findings to stakeholders.
Considering the effects of training program eligibility policies and practices, disruptions to learning opportunities for candidates, and how this contributed to test performance, ACS compiled sources of evidence to assist with better understanding of the effects and how they could be mitigated. Communications to stakeholders and collaboration on strategies about how to potentially support the candidate community were part of a report to the program.
UNIQUE VALUE
ACS understands that policies and practices of components of a comprehensive program can have a ripple effect on outcomes like passing rates. Understanding how these related considerations provide qualitative evidence that interacts with quantitative evidence helps ACS better support questions that programs may have about shifts in psychometric characteristics of tests as well as candidate performance.
SUSTAINABILITY
Because pass rates and trends are a regular part of certification program communication, disruptions to expected trends can raise questions; most times about the test. These instances can threaten the credibility of a testing program among its stakeholders. Understanding how various factors can contribute to observed changes in expected performance can help a program be more proactive in their communications about how and why divergent observations may occasionally be shared with stakeholders.