*******************************************
SPEAKER: Tanja Kaser, EPFL (Lausanne, Switzerland)
TITLE: Modeling and Individualizing Learning in Open-Ended Learning Environments

ABSTRACT: Open-ended learning environments (OELEs) allow students to freely interact with the content and to ideally discover important principles and concepts of the learning domain on their own. Effectively using such environments is challenging for many students and therefore, adaptive guidance has the potential to improve student learning. Guidance in the form of targeted interventions or feedback therefore has the potential to improve educational outcomes. Providing effective support is, however, also a challenge because it is not clear how effective learning strategies in such environments look like. In this talk, I will first present examples for the early prediction of learning outcomes in educational games and interactive simulations. In the second part, I will focus on the use of RL agents in OELEs as a basis for understanding and intervention.

BIO: Tanja Kaser is an assistant professor at the School of Computer and Communication Sciences (IC) at EPFL. Her research lies at the intersection of machine learning, data mining, and education. She is particularly interested in creating accurate models of human behavior and learning. Prior to joining EPFL, Tanja Kaser was a senior data scientist with the Swiss Data Science Center at ETH Zurich. Before that, she was a postdoctoral researcher with the AAALab at the Graduate School of Education of Stanford University. Tanja Kaser received her PhD degree from the Computer Science Department of ETH Zurich; her thesis was distinguished with the Fritz Kutter Award for the best Computer Science thesis at a Swiss university.

*******************************************
SPEAKER: Simon Woodhead, Eedi (London, UK)
TITLE: Eedi and the NeurIPS 2020 Education Challenge Dataset

ABSTRACT: Why would you want to use the NeurIPS 2020 Education Dataset? Where does it come from? What is the pedigree? In this talk we describe the novel features of this dataset, some examples of the tasks which have been investigated so far, and how it has led to research which is having real-life impact.

BIO: Founder and data scientist in the educational technology sector, creating award-winning edtech solutions through innovative data capture, analysis and visualisation. Senior executive of three edtech companies, with extensive knowledge of product and business development. Established success in turning ideas into reality, taking concepts from inception, through minimal viable products, to successful solutions.

*******************************************
SPEAKER: Jose Miguel Hernandez Lobato, University of Cambridge (Cambridge, UK)
TITLE: Deconfounding Reinforcement Learning in Observational Settings

ABSTRACT: We propose a general formulation for addressing reinforcement learning (RL) problems in settings with observational data. That is, we consider the problem of learning good policies solely from historical data in which unobserved factors (confounders) affect both observed actions and rewards. Our formulation allows us to extend a representative RL algorithm, the Actor-Critic method, to its deconfounding variant, with the methodology for this extension being easily applied to other RL algorithms. In addition to this, we develop a new benchmark for evaluating deconfounding RL algorithms by modifying the OpenAI Gym environments and the MNIST dataset. Using this benchmark, we demonstrate that the proposed algorithms are superior to traditional RL methods in confounded environments with observational data. To the best of our knowledge, this is the first time that confounders are taken into consideration for addressing full RL problems with observational data.

BIO: Jose Miguel is a University Lecturer (equivalent to US Assistant Professor) in Machine Learning at the Department of Engineering in the University of Cambridge, UK. Before, he was a postdoctoral fellow in the Harvard Intelligent Probabilistic Systems group at Harvard University, working with Ryan Adams, and a postdoctoral research associate in the Machine Learning Group at the University of Cambridge (UK), working with Zoubin Ghahramani. Jose Miguel completed his Ph.D. and M.Phil. in Computer Science at the Computer Science Department in Universidad Autonoma de Madrid (Spain), where he also obtained a B.Sc. in Computer Science from this institution, with a special prize to the best academic record on graduation. Jose Miguel's research is at the intersection of Bayesian methods and deep learning with a focus data-efficient and robust learning.

*******************************************
SPEAKER: Min Chi, North Carolina State University (Raleigh, USA)
TITLE: The Impact of Pedagogical Policies on Student Learning - A Reinforcement Learning Approach

ABSTRACT: Reinforcement Learning (RL) offers one of the most promising approaches to data-driven decision-making for improving student learning in interactive e-learning systems. RL algorithms are designed to induce effective policies that determine the best action for an agent to take in any given situation so as to maximize a cumulative reward. Optimal decision-making in complex interactive environments is challenging. In ITSs, for example, the system's behaviors can be viewed as a sequential decision process where at each step the system chooses an appropriate action from a set of options. Pedagogical strategies are policies that are used to decide what action to take next in the face of alternatives. Each of these system decisions will affect the user's subsequent actions and performance. Its impact on outcomes cannot be observed immediately and the effectiveness of each decision is dependent upon the effectiveness of subsequent decisions. A number of researchers, including the author, have studied the application of existing RL algorithms to improve the effectiveness of interactive e-learning systems. In this talk, we will describe two case studies on applying RL to improve the effectiveness of educational systems.

BIO: Dr. Min Chi is an Associate Professor in the Department of Computer Science at North Carolina State University. Her research area lies in the interaction of Artificial Intelligence, machine learning, and human-computer interaction. She has established a foundational R&D  portfolio with impactful advancements across four major lines of research, including Reinforcement Learning (RL)-based policy induction. She has served as the PI and Co-PI for a series of federally funded grants from NSF, NIH, and DOE and has led multidisciplinary collaborations. She has received numerous awards for her research expertise and impact, including an NSF CAREER Award, an Alcoa Foundation Engineering Research Achievement Award, and eight Best Paper, Best Student Paper, and Outstanding Paper Awards.

*******************************************
SPEAKER: Emma Brunskill, Stanford University (Stanford, USA)
TITLE: More Practical Reinforcement Learning Inspired by Challenges in Education and Other Societally-Focussed Applications

BIO: Emma Brunskill is an associate professor in the Computer Science Department at Stanford University. Her research goal is to create AI systems that learn from few samples to robustly make good decisions, motivated by applications to healthcare and education. Prior to joining Stanford, Emma Brunskill was an assistant professor at Carnegie Mellon University. Emma's research work has been honored by early faculty career awards (National Science Foundation, Office of Naval Research, Microsoft Research) and several best research paper awards.

*******************************************
SPEAKER: Joe Austerweil, University of Wisconsin-Madison (Madison, USA)
TITLE: Is Reinforcement Just a Value to be Maximized?

ABSTRACT: Modern reinforcement learning has heralded some of the biggest successes in machine learning, such as superhuman Atari and Go performance. Albeit impressive and important, translating these successes to education has not had the same level of success. There are successful human-machine teachers in some domains. However, humans remain the best teachers for other humans. In this talk, I will argue that understanding the relation between human learning and knowledge, conducting basic research in psychology and educational sciences, is the key to improving machine teaching using reinforcement learning. I will present empirical results demonstrating that for humans, reinforcement is more than a value to be maximized. I will conclude by discussing how mutually beneficial collaborations between human and machine learning researchers can enhance applications of reinforcement learning to education, and our understanding of human learning as well.

BIO: Joe Austerweil is an associate professor in the Department of Psychology at the University of Wisconsin, Madison. His research investigates how people make decisions, reason, learn, and encode knowledge. In particular, he uses recent advances in statistics and computer science to formulate ideal learner models to see how they solve these problems and then test the model predictions using traditional behavioral experimentation.

*******************************************
SPEAKER: Shayan Doroudi, University of California Irvine (Irvine, USA)
TITLE: Reinforcement Learning for Instructional Sequencing - Learning from Its Past to Meet the Challenges of the Future

ABSTRACT: For decades, researchers have used the tools of what we now call reinforcement learning to try to adaptively sequence instructional activities for students. This research has come in many shapes and forms over the years, making it difficult to have a global view of the field. In this talk, I will briefly present results from our review of the empirical literature in this area that divides these studies into four major categories. This enables us to identify areas where reinforcement learning has been relatively more or less successful. For the rest of the talk, I will discuss some of the key challenges I believe the field faces going forward and steps we might take towards addressing some of these challenges.

BIO: Shayan Doroudi is an assistant professor at the University of California, Irvine School of Education and (by courtesy) Department of Informatics. His research is at the intersection of educational data science, educational technology, and the learning sciences, and his work draws inspiration from the histories of these fields. He is broadly interested in what he calls the foundations of learning about learning and how they relate to the design of socio-technical systems that improve learning. He has a MS and PhD in Computer Science from Carnegie Mellon University.

*******************************************