BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//NCME - ECPv6.17.5.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://ncme.org
X-WR-CALDESC:Events for NCME
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20270314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20271107T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20260930T130000
DTEND;TZID=America/New_York:20260930T140000
DTSTAMP:20260908T164932Z
CREATED:20260908T164740Z
LAST-MODIFIED:20260908T164932Z
UID:10000048-1790773200-1790776800@ncme.org
SUMMARY:Teaching and Learning Measurement in the Age of AI: Metacognition\, Evaluation\, and Exercises
DESCRIPTION:This is presented by the Training and Professional Development Committee.  \n\n\n\nLarge language models make difficult work easy\, which means the learning in a measurement course now emphasizes the work a model cannot do for the learner. This session presents a method for teaching and learning educational measurement with AI\, built on two habits: attending to the model’s reasoning as well as one’s own in terms of metacognition and executive function\, since every answer rests on silent inferences about the learner’s goal and about the standard of good practice that applies to it; and attention to the popular misconceptions and salient misunderstandings that these models have uncritically learned from the broad literature and now reproduce with confidence. Participants will see example exercises that put these habits to work — coding collaboratively with a model on a measurement problem\, casting the model as “Reviewer 2” against a classic paper\, tracing how a foundational study’s ideas grew or were left behind\, and testing a claimed gap through simulation — and will learn to write their own AI skills: short\, reusable instructions for LLMs that fix the model’s goal\, standard\, and procedure in advance\, so that executive control of the task stays with the learner rather than passing to the model. \n\n\n\nParticipants will be able to: \n\n\n\n\nIdentify the two inferences a language model makes before answering a measurement question\, and evaluate both alongside the answer itself.\n\n\n\nAnticipate the measurement topics on which AI has uncritically learned the field’s popular misconceptions\, and check its answers against what the discipline can actually defend.\n\n\n\nAdapt classroom exercises — collaborative coding\, adversarial review\, conceptual tracing\, and simulation — that use AI to build understanding of measurement rather than to bypass it.\n\n\n\nDraft a reusable AI skill that specifies a goal\, a standard\, and a procedure\, so that the learner sets the terms of the task in advance.\n\n\n\n\nRegister Now\n\n\n\nAbout the Presenter\n\n\n\nJoseph H. Grochowalski\, PhD\, is a Senior Psychometrician at the College Board whose work includes measurement validity\, test security\, standard setting\, machine cognition in AI-based scoring\, and the assessment of durable skills. He is an adjunct professor at Teachers College\, Columbia University\, and New York University\, and he consults medical schools with a focus on novel measures of interaction-based skills. He currently co-chairs the NCME Training and Professional Development Committee and offers a workshop on the use of large language models in learning and teaching educational measurement.
URL:https://ncme.org/event/teaching-and-learning-measurement-in-the-age-of-ai-metacognition-evaluation-and-exercises/
CATEGORIES:Webinars
END:VEVENT
END:VCALENDAR