Everyone understands the value and importance of test security. It is one of the most critical aspects of any assessment program and one that requires constant attention because of the potential damage that can be done to a program’s brand and reputation. In this post, we will review the three broad categories that are essential in any security program – prevention, detection, and enforcement – along with an overview of what can be done within each area. In future posts, we will discuss greater detail on each area, and provide specific examples as well as a discussion of some innovative solutions that organizations have developed. While these broad categories certainly contain a fair amount of overlap and dependencies across each category, it can be useful to talk about each one, because it can highlight the steps required within each.
When we start thinking about security, prevention is a critical category that must be addressed. While creating a bulletproof program that is immune to any and all security issues is impossible, it is essential that every program develop a rigorous set of procedures designed to prevent security breaches from occurring. A comprehensive prevention program needs to include education, policies and procedures that address security at every stage of your program, and most importantly proactive test design. These considerations such as secure procedures for the item development process, the number of items in the pool and the rotation of test forms or exposure of items. These will also include confidentiality and nondisclosure agreements with internal staff and external contractors, copyrighting your item banks and tests, comprehensive policies for the delivery of test forms to your candidates, and the development of procedures for people to report any possible security breaches.
While most assessment programs have policies designed to prevent security breaches, there is a danger in allowing these policies to become outdated. One of the most critical components of any prevention program is the constant monitoring of the program, and the occasional change in policies such as item pool usage to help ensure that the policies do not become known by the nefarious characters out there trying to best your program.
Although prevention is desirable, it cannot eliminate all risks. Therefore, we encourage programs to also incorporate a rigorous set of procedures in place for the detection of any security breaches. These procedures may be qualitative and quantitative in nature. Quantitatively, there are psychometric procedures that can help in this process by helping to identify items that have been exposed, individuals whose scores are suspect, or group level concerns that may suggest anomalies at a given test center or location. These type of psychometric data forensics can be built seamlessly into your scoring process, and can be designed to be completed before any final test scores are provided to candidates.
Some of the qualitative detection opportunities may include external secret shopper programs, and monitoring social media or other digital media for content exposure. In addition, it is also valuable for testing programs to establish policies and infrastructure that support ethical practices. These policies should allow those on the front lines of your program – test developers, test administrators, your candidates – to report any security breaches they have observed in a way that respects the rights of all parties involved.
Lastly, if a security breach is identified, investigation and enforcement of any policies are then critical to maintain the credibility of the program. Most testing programs have a clear set of policies in place for any candidate or individual who has been flagged for potentially fraudulent behavior. But what many programs do not have is a clear set of procedures for how these policies will be enforced. For example, programs may know that they will invalidate test scores for any candidate proven to have cheated on their test, but they may not have a clear set of policies for how these decisions will be communicated, how the procedures will be documented, and who will be responsible for things like testifying in court or communicating with the candidate population or the media. The time to identify these procedures is prior to, and not in the middle of, the chaos that can occur during a serious test security breach.
Every assessment program has its own unique set of risks and concerns when we discuss options for security. However, every program is encouraged to evaluate how to address and prioritize prevention, detection, and enforcement strategies when thinking about security. In future posts, we will provide further detail on each, and discuss some innovative solutions that have been applied to help address these concerns.