OUR SERVICES


DESIGN

Program Design & Redesign

The ACS team has extensive experience guiding programs through the fundamental questions that must be addressed about every assessment program.  When setting up a program for the first time or considering substantive changes to your program, it is essential that the underlying pillars of the assessment program be thoroughly evaluated before they are adopted.  ACS can help identify these foundational pillars and assist your organization with the process of ensuring that these decisions accurately reflect the core principles and values of your company.  

Assessment Design & Redesign 

In the development or redevelopment of any assessment, there are key design decisions made early in the process that set up the framework for the subsequent steps. By creating a development roadmap for your assessment, the subsequent phases of the process are less daunting and conducted in a way that aligns with the intended use of scores. The ACS team has helped numerous programs execute on the design process and work with staff members to plan for the development and implementation of their assessment program.  

Research Design

As part of the continual test maintenance process, program leaders are often faced with specific questions or issues that need to be addressed. The challenge arises in determining how to investigate these topics and collect evidence that will help guide decision making. ACS staff have worked with program leaders to design such research studies to investigate policy, validity, and fairness issues within an existing program. The results of such studies help to guide future program planning and redesign.



OPERATIONAL SUPPORT

Standard Setting

For almost every assessment program, at some point a single or multiple cut points must be determined in order to make critical pass/fail or proficiency decisions.  ACS staff are experts in all aspects of standard setting, and will take the time to fully understand the nature of the assessment program and its candidates, and the precise nature of how and when the cut scores will be implemented.  Once the most appropriate methods for conducting the standard setting, ACS can handle all aspects of the project, and will deliver a comprehensive report supporting the validity and fairness of the cut score recommendations.  

Outsourced Psychometric Staff

Psychometrics are a key element in the continual development and maintenance of any assessment. For many programs, the ideal arrangement is to work with external psychometric resources for program maintenance, oversight, and evaluation. ACS staff have worked with programs – from small to large – to provide psychometric direction and operational support for their assessments. 

Alignment

A critical component of any assessment program is the alignment of the key elements (e.g., content specification, content, opportunity to learn). Although it is often assumed that appropriate test development will automatically result in alignment, many programs have realized the benefit of a formal alignment review to identify the strengths and areas in need of improvement. ACS has conducted numerous independent alignment studies and conducted research on this topic.  

Psychometric Analysis

An essential component of every assessment program is the psychometric foundation of the program.  These foundations can be established through fairly simple item analyses and item calibration procedures that will help ensure that all items on the assessment are functioning as expected.  It can also be established through the psychometric equating or linking of multiple forms of the assessment, ensuring that all candidates who take each test for are treated fairly and equitably.  At ACS, we are committed to not only delivering the highest quality psychometric analyses, but also to fully explaining the analyses to our clients, including the important insights that can be derived while reviewing the results.  

Technical Documentation

Documentation of assessment design, development, and maintenance efforts is critical to maintaining the defensibility of a testing program and for ongoing evaluation. However, determining what and how to document can be challenging and many programs have sought external assistance to maintain accurate documentation. ACS staff have experience in creating such documentation for both internal and external purposes.



QUALITY ASSURANCE

Psychometric Oversight

Many programs rely on the services of an external vendor for assessment design, development, administration, and documentation. These services are critical for the sustainability for their program but relying on external sources introduces risk to the program. The ACS team has experience providing psychometric oversight for a number of programs that are looking for assurance that the decisions being made on behalf of their program adhere to professional standards and recognized acceptable practices. 

Program Review and Audit

For many assessment programs, the opportunity to conduct an independent review or audit of the program can be a valuable chance to take a step back and review the state of the assessment program.  All too often, as programs work to satisfy the day to day demands of the program, the activities and behavior can slowly shift over time, and program owners may not realize the inconsistencies that have developed between the stated mission of the assessment program and its actual activities.  ACS team members have conducted numerous such reviews, and are focused on not only completing a comprehensive review of any assessment program, but to also delivering a usable final report, with action items that can be reviewed and considered by the program as they move forward.  

Program Evaluation

When new educational programs are introduced, it is essential that the direct and indirect impacts of these programs are thoroughly investigated.  A rigorous program evaluation can provide valuable insight into the effectiveness of these programs as well as the critical inputs that influence the effectiveness.  ACS staff have worked with programs to design and conduct the independent evaluation of educational programs and can guide programs in critical steps through the process to ensure that the evaluation delivers a comprehensive and valid evaluation of the programs.  

Technical Advisory Committees

Our team members serve on state-, national-, and international-level technical advisory committees to provide oversight and quality assurance to numerous assessment programs. The use of such an advisory committee shows a programs dedication to quality assurance and maintain a strong assessment system for their stakeholders.