Title
Author(s)
Date
blue-carrot
Date: Jun 14, 2021

Author(s): Wolkowitz, Amanda A.

Description:

As published in the Journal of Educational Measurement (JEM), June 14, 2021

Abstract:

Decision consistency (DC) is the reliability of a classification decision based on a test score. In professional credentialing, the decision is often a high-stakes pass/fail decision. The current methods for estimating DC are computationally complex. The purpose of this research is to provide a computationally and conceptually simple method for estimating DC that produces results comparable to, and at times potentially better than, the widely used Livingston-Lewis method.

Authors and Affiliations:

  • Amanda A. Wolkowitz, Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
blue-carrot
Date: Feb 1, 2021

Author(s): Wolkowitz, Amanda A.; Foley, Brett P.; Zurn, Jared

Description:

As published in the Journal of Applied Testing Technology (JATT) Vol 22(1), 12- 24, 2021

Abstract:

As assessments move from traditional paper-pencil administration to computer-based administration, many testing programs are incorporating alternative item types (AITs) into assessments with the goals of measuring higher-order thinking, offering insight into problem-solving, and representing authentic real-world tasks. This paper explores multiple applications of AIT items and the psychometric properties of these items, including item response time, difficulty, item-total score correlations, and distractor analyses. The appropriate use of these items is also discussed in the context of professional credentialing exams.

Author Affiliations:

  • Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
  • Director of Professional Credentialing and Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
  • Vice President, Examination, National Council of Architectural Registration Boards, 1401 H St NW #500, Washington, DC 20005, United States, United States
blue-carrot
Date: 2018

Author(s): Eckerly, Carol; Smith, Russell; and Sowles, John

Description:

Eckerly, Carol; Smith, Russell; and Sowles, John (2018) “Fairness Concerns of Discrete Option Multiple Choice Items,” Practical Assessment, Research, and Evaluation: Vol. 23 , Article 16.

blue-carrot
Date: 2017

Author(s): Wolkowitz, A., Impara, J. C., & Buckendahl, C. W.

Description:

Wolkowitz A.A., Impara J.C., Buckendahl C.W. (2017) Closing the Loop: Providing Test Developers with Performance Level Descriptors So Standard Setters Can Do Their Job. In: Blömeke S., Gustafsson JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham

Available online.

blue-carrot
Date: Jan 3, 2017

Author(s): Foley, B. P.

Description:

Presentations:

Foley, B. P., Terry, J. E., & Zurn, J. (2016, November [accepted]). Communication strategies for improving stakeholder buy-in. Presentation at the annual conference of the Institute for Credentialing Excellence, Colorado Springs, CO. Foley, B. P., Terry, J. E., & Wilkins, J. R. (2015, October) Small-scale credentialing programs: Balancing security, fairness, and candidate-friendliness. Presentation at the annual conference of the Institute for Credentialing Excellence, Portland, OR.

Buckendahl, C. W., & Foley, B. P. (2015, April). Policy linking as cut score moderation: Considerations for practice. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, Il.

Foley, B. P. (2014, April). Evaluating an impact percentage smoothing vertically moderated standard setting design. Paper presented at the annual meeting of the National Council on Measurement in Education, Philadelphia, PA.

Abernathy, T., Bliss, T.J., Vineyard, R., Wentworth, N., and Foley, B. P. (2013, October). Evaluating the effects of the Common Core State Standards initiative and its associated assessment consortia testing programs on teacher training programs. Panel discussion presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY

Lim, G. S., Buckendahl, C. W., & Foley, B. P. (2012, July). Evaluating test design through complementary standard setting studies. Paper presented at The 8th Conference of the International Test Commission, Amsterdam, Netherlands.

Foley, B. P., & Lupher, D. A. (2012, April). Evaluating the stability of cut scores over time in a statewide assessment system. Paper presented at the annual meeting of the National Council on Measurement in Education, Vancouver, BC.

Foley, B. P. (2011, April). Realistic Expectations: State-level Changes in the Percentage of Proficient Students 2002-2008. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.

Workshops:

Foley, B. P. (2015, April). Using visual displays to inform assessment development and validation. Workshop presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.

Foley, B. P. (2011, October). Validity by design: A validity-centered assessment development approach. Workshop presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY.

Foley, B. P. (2009, October). Prerequisites for power analysis: Questions you need to ask before deciding the sample size for your study. Workshop presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY.

blue-carrot
Date: 2016

Author(s): Foley, B. P.

Description:

Foley, B. P. (2016). Visual displays of test fraud data. In G. J. Cizek & J. A. Wollack (Editors). Handbook of quantitative methods for detecting cheating on tests. New York, NY: Routledge.

blue-carrot
Date: 2016

Author(s): Foley, B. P.

Description:

Foley, B. P. (2016). Getting lucky: How guessing threatens the validity of performance classifications. Practical Assessment, Research and Evaluation, 21(3).

Available online.

blue-carrot
Date: 2016

Author(s): Wolkowitz, A., Davis-Becker, S. L., & Gerrow, J. D. 

Description:

Wolkowitz, A., Davis-Becker, S. L., & Gerrow, J. D.  (2016).  Releasing content to deter cheating:  An analysis of the impact on candidate performance.  Journal of Applied Testing Technology, 17(1), 33-40.

blue-carrot
Date: 2015

Author(s): Wolkowitz, A. & Davis-Becker, S.

Description:

Wolkowitz, A. & Davis-Becker, S.  (2015).  Evaluating common item block options when faced with practical constraints.  Practical Assessment, Research, & Evaluation, 20(19).

Available online.

blue-carrot
Date: 2015

Author(s): Foley, B. P.

Description:

Foley, B. P. (2015). Tailoring visual displays to improve test score interpretation: Including indicators of uncertainty. In M. McCrudden, G. Schraw, & C. Buckendahl (Editors). Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 265-298). Charlotte, NC: Information Age.

blue-carrot
Date: 2014

Author(s): Foley, B. P.

Description:

Foley, B. P. (2014). Modeling the relationships among measures of teacher quality and student performance in high school geometry. The Researcher, 26(1), 28-33.

blue-carrot
Date: 2013

Author(s): Foley, B. P., & Buckendahl, C. W.

Description:

Foley, B. P., & Buckendahl, C. W. (2013). Using visual displays to inform assessment design and development. In G. Schraw, M. McCrudden, & D. Robinson (Editors). Learning through visual displays (pp. 417-445). Charlotte, NC: Information Age.

blue-carrot
Date: 2013

Author(s): Wolkowitz, A. & Skorupski, W. P.

Description:

Wolkowitz, A. & Skorupski, W. P.  (2013). A method for imputing response options for missing data on multiple-choice assessments.  Educational and Psychological Measurement, 73(6), 1036-1053.

blue-carrot
Date: 2013

Author(s): Foley, B. P., Dwyer, A. C., Chuah, D., Rawls, A.

Description:

Foley, B. P., Dwyer, A. C., Chuah, D., Rawls, A. (Producers), & Foley, B. P. (Director). (2013). Testing in the movies and on television [Motion Picture]. United States: National Council on Measurement in Education.

View online.

blue-carrot
Date: 2011

Author(s): Wolkowitz, A.

Description:

Wolkowitz, A.  (2011).  Multiple attempts on a nursing admissions examination:  Effects on the total score.  Journal of Nursing Education 50(9), 493-501.

blue-carrot
Date: 2011

Author(s): Buckendahl, C. W. & Foley, B. P.

Description:

Buckendahl, C. W. & Foley, B. P. (2011). High Stakes Uses of Intelligence Testing. In J. Bovaird, K. Geisinger, & C. Buckendahl (Editors). High Stakes Testing in Education – Science and Practice in K-12 Settings [Festschrift to Barbara Plake] (pp. 191-210). Washington, DC: American Psychological Association Press.

blue-carrot
Date: 2010

Author(s): Wolkowitz, A. & Kelley, J.

Description:

Wolkowitz, A. & Kelley, J. (2010) Academic predictors of success in a nursing program. Journal of Nursing Education, 49, 498-503.

blue-carrot
Date: 2010

Author(s): Wolkowitz, A.

Description:

Wolkowitz, A. (Ed.) (2010).  Study Manual for the Health Occupations Basic Entrance Examination – Version V.  Stilwell, KS:  Assessment Technologies Institute®.

blue-carrot
Date: 2009

Author(s): Wolkowitz, A.

Description:

Wolkowitz, A. (Ed.)  (2009).  Study Manual for the Test of Academic Skills – Version V.  Stilwell, KS:  Assessment Technologies Institute®.

blue-carrot
Date: 2009

Author(s): Wolkowitz, A.

Description:

Wolkowitz, A. (Ed.) (2009).  Learning Strategies:  Your Guide to Classroom and Test-taking Success.  Stilwell, KS:  Assessment Technologies Institute®.

blue-carrot
Date: 2006

Author(s): Jones, P., Smith, R., and Talley, D. M.

Description:

Jones, P., Smith, R., and Talley, D. M. (2006). Developing test forms for small-scale achievement testing systems. In Downing, S.M., and Haladyna, T.M., Handbook of test development. Mahwah, NJ: Lawrence Erlbaum Associates.