In this digital ITEMS module, Dr. Nana Kim, Dr. Jiayi Deng, and Yun Leng Wong discuss the conceptual framework of IRTree models and demonstrate examples of their applications in the context of both cognitive and noncognitive assessments.
Module Overview
Item response tree (IRTree) models (Böckenholt, 2012; De Boeck & Partchev, 2012; Jeon & De Boeck, 2016), an item response modeling approach that incorporates a tree structure, have become a popular method for many applications in measurement. IRTree models characterize the underlying response processes using a decision tree structure, where the internal decision outcome at each node is parameterized with an item response theory (IRT) model. Such models provide a flexible way of investigating and modeling underlying response processes, which can be useful for examining sources of individual differences in measurement and addressing measurement issues that traditional IRT models cannot deal with. In this module, we discuss the conceptual framework of IRTree models and demonstrate examples of their applications in the context of both cognitive and noncognitive assessments. We also introduce some possible extensions of the model and provide a demonstration of an example data analysis in R.
Nana Kim
- Assistant Professor in Quantitative Methods in Education at University of Minnesota, Twin-Cities
- Research Interest: Development and applications of item response theory (IRT) models for improving measurement in education, psychology, and social sciences
- Teaches courses on educational and psychological measurement, item response theory, and factor analysis
Jiayi Deng
- Research scientist in Human Resources Research Organization (HumRRO)
- Research Interest: Detection and handling aberrant test-taking responses by process data, item response models, and linking and equating in international large-scale assessments
- Taught an educational and psychological measurement course at University of Minnesota
Yun Leng Wong
- Ph.D. candidate in Quantitative Methods in Education at University of Minnesota, Twin Cities
- Research Interest: Educational measurement, item response times, non-effortful test responses
Introduction
Upon completion of this ITEMS module, learners should be able to:
- Comprehend the conceptual framework of IRTree models
- Specify an IRTree model for their hypothesized response process
- Perform data analysis using a specified IRTree model
- Evaluate benefits and limitations/challenges of IRTree models
- Critically read the literature on IRTree models and their application
Section 1: An Overview of IRTree Models
Upon completion of this section, learners should be able to:
- Understand the conceptual framework of item response tree (IRTree) models
- Specify a tree structure based on hypothesized internal decision process
- Transform observed item responses into pseudo-item responses
- Show how to capture the probability of a terminal response based on a specified IRTree model
Section 2: IRTree Models for Response Styles
Upon completion of this section, learners should be able to:
- Describe the concept of response styles within the context of noncognitive assessments
- Specify a tree structure for modeling different types of response styles
- Illustrate statistically the probability of selecting a specific response category based on the IRTree model accounting for response styles
- Explain benefits and challenges of the IRTree approach for modeling response styles
Section 3: Modeling Test-Taking Behaviors
Upon completion of this section, learners should be able to:
- Explain the tree structure of IRTree models within the context of test-taking behaviors
- Specify the IRTree model for answer change and rapid guessing in the context of cognitive assessment context
- Interpret item parameters in the context of test-taking behaviors
- Evaluate the assumptions and advantages of the IRTree models for test-taking behaviors
Section 4: Some Extensions of IRTree Models
Upon completion of this section, learners should be able to:
- Discuss some potential limitations of typical IRTree models
- Discuss possible extensions of the IRTree models
Section 5: R Demonstration
Upon completion of this section, learners should be able to:
- Recode observed item responses into pseudo-item responses using R
- Fit a specified IRTree model using R
- Interpret the R output obtained by fitting an IRTree model
- Evaluate the model fit of an IRTree model