Testing Disruptions in a Credentialing Test Administration
ACS TEAM ON PROJECT
- Chad Buckendahl, Ph.D., Partner
- Andrew Wiley, Ph.D., Partner
CLIENT
National organization representing a professional organizations with over 210,000 members in Canada and around the world.
PROJECT
Client engaged ACS to assist in their review of test taking disruptions that occurred during their test administration. It is an open-book exam with resources provided to the candidates directly via the online delivery engine. Disruptions caused some test candidates to not receive the full range of support materials, while others were denied access to the test entirely for brief periods of time. The test taking disruptions did not appear to follow any patterns. Some candidates within the same testing center had no disruptions while others were not able to login at all.
ACS and the Client completed a series of analyses that investigated specific candidates as well as candidates across multiple key criteria, such as language and testing centers. These analyses were shared with the Client’s Board to support them in their recommendations. These recommendations were designed to maintain the integrity of the CPA credential while still honoring the time and commitment of all test candidates.
UNIQUE VALUE
ACS team accommodated the immediate need of the Client to address testing disruptions experienced by the CPA candidates. ACS was able to engage as the Client’s psychometric consulting partner and able to work with them in a fast-paced environment, with topics and data shifting in and adjusting in almost daily basis.
SUSTAINABILITY
ACS completed the review and identified the nature and extent of the disruptions that required Client to develop a unique set of solutions for different candidates. Rather than creating only one policy and applying it to all candidates, ACS and the Client developed a set of reasonable classifications for the candidates’ disruptions that were experienced. Based upon these classifications, Client was able to develop a series of recommendations for candidates that meaningfully addressed the specific disruptions and provided candidates the best opportunity to demonstrate their knowledge and skills in the area.