Contemporary Issues in Scaling, Linking & Equating

The Contemporary Issues in Scaling, Linking, & Equating (SLE) SIGIMIE is dedicated to:

  • Exploring emerging measurement challenges in establishing and maintaining score scales
  • Upholding fundamental principles by ensuring that our rapidly evolving fields remain grounded in rigorous, principled approaches that are explainable, interpretable, and defensible
  • Fostering the integration of theory and practice; and bringing together professionals from diverse backgrounds to exchange knowledge and to develop solutions
Join a SIGIMIE
Webinars & References
Conference Sessions
SIGIMIE Leadership

Chair: Tim Moses
Vice Chair: Jonathan Weeks
Secretary: Seongeun Kim
Graduate Student Liaison: Nick Trout
Board Liaison: Laura Hamilton

Advisory Board

Michael J. Kolen, University of Iowa
Neil Dorans, ETS (retired)
Won-Chan Lee, University of Iowa
Alina von Davier, DuoLingo

Webinars & References

Online webinars will be hosted bi-annually and offset from the timing of NCME annual conference. Webinars are intended to increase member engagement and enhance members’ skill sets and would last 1 hour in duration and be hosted via Zoom. For some webinars, we’d like to invite either a well-known SLE scholar or an individual who has recently published on a topic related to SLE. They could present their recent research or another SLE-related topic of their choice for the first 30 to 40 minutes and attendees would have the chance to ask questions in the remaining 20-30 minutes. Other webinars could include small-scale trainings and demonstrations of open-source or freely available software and could be broken down into several sessions, if interest is indicated. Alternatively, graduate students could be invited to present their SLE research and/or ideas, and gather feedback from established professionals in the field. 

History of Test Equating Methods and Practices Through 1985 is the first online webinar that featured Michael Kolen on August 27th, 2020. 

Storytelling the History of Scaling, Linking, & Equating and Looking to the Future is an online webinar series that featured SLE legacy scholars as well as scholars with on-going SLE-related research agendas, in April to July, 2021. Each presenter led a 1-hour webinar discussion on topics such as how they ended up researching SLE methods, their biggest accomplishments, unfinished business, filed projection over the next 10 years, and perceived impact by the COVID pandemic. Recordings of each webinar can be found below.

SLE Training Sessions is a series of free online training sessions on topics directly related to scaling, linking, and equating (SLE) in October, November, and December, 2021. These sessions were geared towards graduate students or those who have not had the opportunity to receive formal trainings in SLE methods. Each session was offered by an SLE scholar and ran 1 ½ to 2 hours. 

Scaling: Approaches and Interpretations featured Tim Moses on Jan.18th, 2024. The webinar reviewed and discussed scaling approaches established to support a range of interpretations in the practical context, and also with respect to historical and current perspectives in the field.  

Linking Responsive Assessments: Tradeoffs and Implications for Inferences featured Tim Moses and Neil Dorans on Nov. 18th, 2024.  This talk examines three types of socioculturally-responsive assessments in terms of how they address trad-offs in validity for specific populations versus comparability within and across populations.

The biggest test equating study in the world … Ever featured Tom Benton and Matthew Carroll on Mar. 26th, 2025 to introduced and discussed a large empirical study to understand which equating methods work best under which circumstances.

SLE-related Publications 2010-2022

SIGIMIE-Specific NCME Sessions

Score Comparability Challenges in Applied Assessment Contexts
(Contemporary Issues in Scaling, Linking & Equating SIGIMIE Session)
Ongoing challenges in assessment include efforts to redesign assessments while maintaining score comparability. This session presents several examples of these challenges and approaches to addressing them. The first presentation examines concordance tables for college admissions tests, highlighting concerns about their accuracy when constructs, testing populations, and subpopulations differ or change across the concorded tests. The second presentation focuses on linking a licensure test before and after a transition to reduce and integrate assessment components, including analyses of linkings, calibrations, and performance evaluation, item difficulty and comparability. The third presentation covers a K-8 exam’s transition from a two-step testing (one fixed-form and one multi-stage) to an integrated multi-stage test, including efforts to ensure comparability in assessment performance, reliability, and (differential) item functioning. The fourth presentation explores comparability challenges arising from automated procedures for generating test items and their scoring parameters, and the use of automated, non-human scoring for open ended responses. The fifth presentation describes empirical evaluations of digital versus paper-based administrations of NAEP, including mode effects across grades and subjects, exam results that account for mode effects, and self-reported perceptions. These papers provide new insights for supporting score comparability in the recently and rapidly changing assessment field.

Chair: Tim Moses

Presentations:

  • College Admissions Issues Related to Linking and Equating (Kurt Geisinger)
  • 1+1>2: Linking Separate Knowledge and Skills Examinations to An Integrated Examination (Hao Song, Association of State and Provincial Psychology Boards)
  • Evaluating Score Comparability of a Redesigned Multistage Assessment (Shirley Li, Savvas Learning Company; Jinnie Choi, Savvas Learning Company; Rongchun Zhu, Savvas Learning Company)
  • AI Applications for Assessment Automation: Implications for Score Comparability (Tim Moses, Buros Center for Testing; Jinghua Liu, The National Board of Osteopathic Medical Examiners; Seongeun Kim, National Commission on Certification of Physician Assistants; Nick Trout, Michigan State University)
  • Digital Assessment Mode Effects in K-12: Insights from Large-Scale NAEP Data in Reading, Mathematics, Science, and Social Science (Paul Jewsbury, ETS; Yue Jia, ETS Institute; Nuo Xi, ETS; Meng Wu, Educational Testing Service; Adrienne Geijer, ETS Research Institute; xueli xu, ETS Institute; Carol Eckerly, ABIM; John Donoghue, ets)

    Scaling, Linking, and Equating with Test Redesign (SIGIMIE-Organized Session)
    With the acceleration of knowledge updating and the advancing technology, testing programs are more frequently expanding the capability of their assessment to improve the measurement of the intended construct and/or to provide better testing experience for target population. Example changes in test redesign include updating contents, redefining specification, evaluating higher cognitive abilities/skills using innovative item types, the self-growing item pool from automatic item generation, and the shorter administration time in adaptive testing. To ensure the success of a test redesign, it is challenging but critical to keep the score use interchangeably and/or interpretably. This session brings together practitioners from four recently redesigned large-scale standardized testing programs to share their practical considerations, issues, and solutions in scaling, linking, and equating during the test redesign process. Open-floor discussion is planned for exchanging insights and guidance benefiting current and future redesigned assessments and researchers of interests.

    Chair: Min Wang, College Board

    Moderator: Jinghua Liu, The National Board of Osteopathic Medical Examiners

    Presenters:

    • Jaime Malatesta, GMAC
    • Dongmei Li
    • Joe Betts, NCSBN
    • Weiwei Cui

    The SLE SIGIMIE Session for the 2024 NCME conference (April 2024) brought professionals from both academia and industry together to discuss the applications and issues of SLE in AIG.

    Theme: Scaling, Linking, and Equating in Automatic Item Generation

    Panelists:

    • Richard Luecht, University of North Carolina Greensboro
    • Ikkyu Choi and Sandip Sinharay, ETS
    • Weiwei Cui and Thomas Proctor, College Board

    The SLE SIGIMIE Session for the 2023 NCME conference (April 15, 2023) brought professionals together who work closely with technology-enhanced assessments in various testing fields to share their expertise in scaling, linking, and equating practices.

    Theme: Scaling, Linking, Equating in Technology-Enhanced Assessments


    Panelists:

    • Xia Mao, National Board of Osteopathic Medical Examiners (NBOME); Larissa Smith, ETS
    • Shu-Chuan Kao, National Council of State Boards of Nursing (NCSBN); Joe Betts, National Council of State Boards of Nursing (NCSBN); Chih-Kai (Cary) Lin, National Council of State Boards of Nursing (NCSBN)
    • Ye (Cheryl) Ma, Amazon Web Services (AWS); Jennifer Davis, Amazon Web Services (AWS)

    Discussant:​

    • Alina von Davier, Duolingo

    The SLE SIGIMIE Session for the 2022 NCME conference (April 9, 2022) brought professionals together from various industries such as medicine, psychology, and business to discuss their unique challenges and approaches to SLE, connected approaches used in these unique settings to those used in traditional educational settings, and provided a framework within which scholars can expand their SLE research lens. 

    Theme: Scaling, Linking, Equating Beyond Our Comfort Zones

    Panelists:

    • James Ingrisone, Pearson VUE
    • Huijuan Meng, Amazon Web Services (AWS)
    • Stephen Stark, University of South Florida
    • Seung W. Choi, University of Texas at Austin

    Discussants:​

    • Tim Moses, College Board
    • Neil Dorans, ETS

    The SLE SIGIMIE Session for the 2021 NCME conference (May 25, 2021) featured an expert panel discussion based on questions raised from our SIGIMIE members. Questions concerned general issues in SLE as well as special designs and considerations related to the impact of COVID-19. 

    Theme: Scaling, Linking, & Equating Du Jour: A Discussion with Experts

    Panelists:

    • Michael Kolen, University of Iowa
    • Marie Wiberg, Department of Statistics, USBE
    • Tim Moses, College Board
    • Jorge Gonzalez, Pontificia Universidad Catolic

    Discussant:

    • Robert Brennan, University of Iowa

    Past Leadership and Advisory Boards

    2026–2027
    • Chair/Co-chair(s): Tim Moses (Buros Center)
    • Vice Chair: Jonathan Weeks (Stanford)
    • Secretary: Seongeun Kim (NCCPA)
    • Grad Stud Liaison: Nick Trout (MSU)
    2025–2026
    • Chair/Co-chair(s): Jinghua Liu (NBOME)
    • Vice Chair: Tim Moses (Buros Center)
    • Secretary: Seongeun Kim (NCCPA)
    • Grad Stud Liaison: Nick Trout (MSU)
    2024–2025
    • Chair/Co-chair(s): Min Wang (College Board)
    • Vice Chair: Jinghua Liu (NBOME)
    • Secretary: Seohee Park (ABIM)
    • Grad Stud Liaison: Seongeun Kim (UNCG)
    2023–2024
    • Chair/Co-chair(s): Kyung Yong Kim (UNCG)
    • Vice Chair: Min Wang (College Board)
    • Secretary: Seohee Park (ABIM)
    • Grad Stud Liaison: Seongeun Kim (UNCG)
    2022–2023
    • Chair/Co-chair(s): Jaime Malatesta (Pearson) & Stella Kim (UNCC)
    • Vice Chair: Kyung Yong Kim (UNCG)
    • Secretary: Kanages Suppiah (UUM)
    • Grad Stud Liaison: Tong Wu (UNCC)
    2020–2022
    • Chair/Co-chair(s): Jaime Malatesta (Pearson)
    • Vice Chair: Stella Kim (UNCC)
    2022–2027
    • Michael Kolen, Neil Dorans, Alina von Davier, Won-Chan Lee
    2020–2022
    • Mengyao Zhang, Min Wang