Presented at the ATP Innovations in Testing 2024 Conference
This session introduces a novel application of AI and NLP tools to detect enemy and variant items. While generative AI and natural language processing have gained attention for content generation, the identification of enemy and variant items is often overlooked. These items pose risks to validity, and traditional manual review processes by subject matter experts (SMEs) face limitations due to time and resource constraints—especially with long forms and large item banks. This session presents an approach that combines automated enemy and variant item detection using AI and NLP tools with SME-based item review. Presenters from licensure and certification programs will share their experiences using this approach and discuss the benefits and implications. The session aims to inspire discussions on the wider adoption of automated detection methods to improve the efficiency of the development processes and the validity of the use of the results.
Speakers:
- Nick Respecki, NCARB
- Natalie Flacco, Project Management Institute
- Russell Smith, Alpine Testing Solutions
To view the slides from this session, click the button below:
To learn more about the ATP Innovations in Testing 2024 Conference, click here.