Foley (2019) proposed using a variation of Yen’s Q3 statistic (1984) for collusion detection, calling it B3. Where Yen’s Q3 statistics uses item residual correlations to detect dependencies between pairs of items, the B3 statistic uses person residual correlations to detect dependencies between pairs of test takers. This paper will compare and contrast the advantages and disadvantages of the B3 statistic to Answer Similarity Index (ASI) analyses using the generalized binomial model (van der Linden & Sotaridona, 2006). The B3 statistic has a few potential advantages for detecting collusion. It is readily available via commercial software (e.g. Winsteps, Linacre (2019)), it is generalizable to polytomous models, and it accounts for in both the number of matching scores and the likelihood of the scores given the test takers’ abilities and the difficulty of the individual items. One potential disadvantage of the B3 statistic is that the underlying distributions for B3 is not known making it challenging to estimate probabilities. Real data with known exposure problems as well as simulated data will be used to evaluate the proposed statistic and its viability by specifically comparing it to ASI.
Russell Smith, Ph.D., Alpine Testing Solutions