Test Development Services

Design Program
Develop the infrastructure and framework for your testing program with the intersecting psychometric, policy, legal, security, and business considerations in mind. Alpine Testing Solutions' staff is skilled at working with program stakeholders to identify and document the program goals, audience, value propositions, framework, and infrastructure.

Design Test
Determine and document the intended and unintended uses of the test, the intended examinee population, the score reporting needs, and additional information necessary to communicate the defining elements of the test. Develop a validation plan that is aligned with the intended uses and interpretations of scores for the testing program. Alpine Testing Solutions' staff works closely with test sponsors and subject matter experts to identify and document the defining design considerations for a test or track of tests.

Analyze Domain
Identify through a structured process (e.g. practice analysis, domain analysis, job-task analysis) the relevant content domain and create a detailed breakdown of the content to be addressed in the test. Alpine Testing Solutions' staff has expertise in guiding test sponsors, stakeholders, and subject matter experts through a variety of structured processes involving focus groups, workshops, and surveys.

Develop Blueprint (Test Specifications)
Use information collected in the domain analysis (e.g. committee input, survey data) to finalize the content and the weighting of the content to be addressed in the test. Determine other specifications (e.g. cognitive complexity, item types, scoring rules) necessary to guide the development of test items and/or tasks and the assembly of items and/or tasks into test forms. Alpine Testing Solutions staff is skilled at guiding committees through the interpretation of domain analysis results and discussions designed to finalize blueprint decisions.

Develop Content and Review Content
Develop and review test items and tasks to create or enhance a bank of items that aligns with the requirements specified in the test blueprint. Alpine Testing Solutions' staff has experience:

training and mentoring subject matter experts in the art of creating draft items and tasks;
editing draft items and tasks for grammar, style, psychometric guidelines, and bias;
facilitating subject matter experts in reviewing edited items for congruence and technical accuracy; and,
faciliting or performing additional reviews as necessary to ensure a quality bank of items and/or tasks.

Pre-Test and Analyze
Collect response data for items and/or tasks and determine the usefulness of the items and/or tasks based upon statistical characteristics (e.g. difficulty, discrimination, model fit). Alpine Testing Solutions staff is experienced with and use classical test theory (CTT), item response theory (IRT), and Rasch analyses to evaluate and interpret the psychometric quality of test content.

Assemble Operational Test
Assemble operational test forms that meet the blueprint specifications and are balanced for content, difficulty, discrimination, test time, reliability, and standard error. If standard setting has already occurred, statistically equate new operational forms to a benchmark (pre-defined) cut score. Alpine Testing Solutions staff has expertise using CTT, IRT, and Rasch analyses to create and equate operational test forms.

Conduct Standard Setting
Translate performance standards into one or more cut points on a test form. Alpine Testing Solutions’ staff members have extensive experience designing, conducting, and thoroughly documenting standard setting studies using a variety of test- and examinee-based standard setting methods. Such approaches include Angoff (and its variations), Bookmark (and its variations), contrasting groups, Analytical Judgment Method, and Hofstee.

Maintain Test

Implementation of activities developed in conjunction with the validity framework designed at the outset of the program and adapted over time as the needs of the program change. Such activities include ongoing psychometric consultation and program design (e.g. strategic and tactical initiatives), components of the test development process (e.g. analyze domain, develop blueprint, develop content, review content, pre-test and analyze, assemble operational forms, conduct standard setting), additional item- and form-level analyses (e.g. differential item functioning (DIF), drift, analysis of balance among operational forms), security analyses, and consequential validity analyses.