In this digital ITEMS module, Dr. Ye (Cheryl) Ma and Vinita Talreja go over the basics of machine learning (ML) and Generative AI (GenAI).
Module Overview
Machine learning (ML) and generative artificial intelligence (AI) are rapidly transforming the field of educational measurement. This module focuses on illustrating the process of (1) automated machine learning (AutoML) using AutoGluon via an application of detecting aberrant test behavior and (2) AI-based item generation using Amazon Bedrock. To support these demonstrations, two tools are used: AutoGluon, an open-source automated machine learning (AutoML) system, and Amazon Bedrock, a fully managed AWS service for accessing foundation models from leading AI providers. By the end of this module, participants will (1) understand the key concepts and fundamentals underlying these two applicaitons and (2) be able to programmatically train a classification ML model via AutoML using the provided data, as well as conduct AI-based item generation via the LLMs.
Ye (Cheryl) Ma, Ph.D.
Ye (Cheryl) Ma is a senior psychometrician/research scientist at Amazon Web Services (AWS) Training and Certification Team. Her research focuses on leveraging machine learning and artificial intelligence in educational measurement and psychometric related applications. She has also had research and industry experience in computerized adaptive testing. She earned her Ph.D. in educational measurement from the University of Iowa in 2020.
Vinita Talreja, M.S.
Vinita Talreja is a data scientist at Amazon Web Services (AWS) Training and Certification Team. Her research focuses on leveraging artificial intelligence to innovate certification exam development processes. She has been a regular presenter at the National Council on Measurement in Education (NCME) and the Association of Test Publishers (ATP). She earned her Master of Science degree in business analytics from the University of Texas–Dallas in 2021.
Section 1: Machine Learning via Amazon AutoGluon in Psychometric Applications
Upon completion of this section, learners should be able to:
- Describe the key components in machine learning analysis and projects
- Understand the basics of AutoGluon key features and its key functions for running automatic machine learning (AutoML)
Section 2: Code Demonstration of AutoGluon on Tabular Data for Classification Problems
Upon completion of this section, learners should be able to:
- Train and deploy models using AutoGluon for tabular data classification problems in the local environment
Learning Activities (zip)
Section 3: Amazon Bedrock and Generative AI Applications
Upon completion of this section, learners should be able to:
- Define Generative AI and understand
- Learn about Amazon Bedrock its use cases
- Learn about Amazon Bedrock Prompt Management and Knowledge Bases
- Perform Item Generation using Generative AI
Section 4: Code Demonstration of Amazon Bedrock on AI-Based Item Generation
Learning Activities (zip)