As an intellectual hub where exciting new ideas about data-informed education arise, germinate and spark action, LEARN is committed to helping people interested in learning analytics get up to speed on the latest in the field. We have organized some informative and timely pieces from leaders in the field and key areas of research. If you are new to LA, we suggest you start with the Overview and Development section. The rest of the sections are organized topically and can be visited in any order. Efforts were made to select openly available materials; for cases where this was not possible, we are working on finding alternative ways to offer access.
For an eclectic collection of other interesting resources about Learning Analytics check out our LEARN team favorite readings.
Do you have amazing LA 101 resources to share? Give us your suggestions and get acknowledged on the site!
Key Areas of Research
Overview & Development
Learning analytics is an area of research and practice that uses computational analysis of learning process data to better understand and improve learning. A key component of the work involves providing timely information to educational stakeholders (teachers, students, designers, administrators) to support better decision making.
The resources in this section offer a general introduction to the field.
1. Get Oriented
What is Learning Analytics?
A high level overview of the field of learning analytics from NYU-LEARN Director Alyssa Wise with a focus on how we can use data to better understand and inform teaching and learning. Walking through common kinds of learning analytics data and computational approaches, she highlights the confluence of learning design, theory and methodological choice in generating insights that can lead to action. The talk concludes by situating learning analytics as part of a larger socio-technical system that interacts with existing educational structures for which issues of ethics, privacy, agency and equity must be considered. The talk was given as the opening keynote for the Learning Analytics in STEM Education Research (LASER) Institute Summer Workshop.
Learning Analytics in a Nutshell
A concise introduction video on the definition, purpose, key elements and process of learning analytics from Yi-Shan Tsai and the Society for Learning Analytics and Research (SoLAR). It provides a quick but clear explanation of why learning analytics leverages data, but is driven by human decisions and a focus on students.
Introduction to Learning Analytics
This short video from the Data Analytics and Learning Massive Open Online Courses (MOOC) introduces the potentials of big data in education and how learning analytics as technology is being developed to make use of it. It also briefly discusses some of the challenges of using learning analytics to transform education.
2. Dive Deeper
Educational Data Mining and Learning Analytics
This 2012 article by Ryan Baker and George Siemens points out some of the key similarities and differentiators between the fields of learning analytics and educational data mining as well as the benefits of healthy competition and collaboration between the two communities. The goal is to help researchers across the two areas better understand each other and identify ways to collaborate.
This 2019 book chapter by NYU-LEARN Director Alyssa Wise provides a concise overview of the overarching goal of learning analysis as enabling data-informed decision-making by students and educators and highlights three aspects that make it a distinct and impactful technology to support teaching and learning. Specifically, the book chapter describes the characteristics of data used in learning analytics, classes of methods commonly used to analyze it, and the kinds of pedagogical uses that result. Key issues for the future of learnIng analytIcs are also identified to spark further thinking.
Learning Analytics: Drivers, Developments and Challenges
This 2012 paper by Rebecca Ferguson examines how technological, educational and political factors shaped the emergence of learning analytics, first from a data-driven and then from a learning-focused perspective. Considering the relationship between the related but distinct areas of learning analytics, educational data mining and academic analytics, it shows anticipated future developments and the challenges perceived as the field began.
The Journey of Learning Analytics
This 2019 paper by Srećko Joksimović, Vitomir Kovanović and Shane Dawson describes how the field of learning analytics has evolved since the first Learning Analytics and Knowledge Conference in 2011. Addressing both technical issues related to data generation and analysis as well as socio-technical issues related to ethics and adoption, it considers the future of the field through a broad lens.
The Need for Learning Theory in Learning Analytics
This 2015 paper by NYU-LEARN Director Alyssa Wise and David Williamson Shaffer discusses the important role of theory in learning analytics in framing the selection of relevant data, applying appropriate models, and interpreting results to generate actionable insights. It also provides pointers to a set of four other papers, each with a critical commentary, in which authors intentionally worked to incorporate learning theory into their learning analytics research.
Predictive modelling is a core learning analytics technique that uses supervised machine learning to identify patterns of relationships in past data that can be used to make predictions about the future. If you are new to the field or want a quick refresher, we recommend the following resources.
Thank you to Craig Thompson, Dr. Chris Brooks and Dr. Quan Nguyen for providing some of the resources in this section.
1. Get Oriented
Predictive Modelling in Teaching and Learning
This 2017 Handbook of Learning Analytics chapter by Christopher Brooks and Craig Thompson provides an overview of predictive modeling through the lens of an educational data scientist. The authors also delve into some of the most prominent predictive modelling techniques.
A Visual Introduction to Machine Learning
A short two-part, high-level introduction to machine learning by R2D3 using dynamic visualization of decision trees to explain the logic underlying classification models.
2. Dive Deeper
A Large-Scale Implementation of Predictive Learning Analytics in Higher Education
This 2019 paper by Christothea Herodotou, Bart Rienties, and colleagues examined the use and impact of predictive learning analytics by teachers in a distance learning institution with over a thousand students. Findings show both how the data qualitatively impacted teaching practices and quantitative effects on students’ performance.
Student Success Prediction in MOOCs
This comprehensive 2018 paper by Josh Gardner and Christopher Brooks reviews the literature on predicting student success in MOOCs (Massive Open Online Courses). Looking at different kinds of outcomes, predictors and underlying theoretical models, the article overviews the main data sources, feature engineering methods, modeling algorithms, and evaluation methods used. The paper also points out key methodological gaps in the literature and highlights opportunities for future research.
Practical Machine Learning Tools and Techniques
This 2016 book by Ian Witten, Eibe Frank and colleagues is an accessible introduction to machine learning that discusses inputs, outputs, common algorithms and evaluation methods used in this data mining approach. It also includes coverage of more advanced topics including data transformations, bayesian networks, deep learning and semi-supervised approaches
3. Tools & Tutorials
Data Science in Education Using R
This 2020 book by Ryan Estrellado, Emily Bovee and colleagues offers several great walkthroughs on how to use data commonly available in online courses to predict student performance through both linear regression (Chapters 7 & 8) and random forest models (Chapter 14). It also offers an introduction to R and R Studio (Chapters 5 & 6) for those just getting started.
Hands-On Machine Learning (in Python)
This 2019 book by Aurélien Géron explains and demonstrates the use of a variety of predictive modeling techniques in Python, ranging from simple linear regression to deep neural networks. It also offers exercises to practice each technique. While the book requires a purchase, a free 10-day trial is available.
Network analytics are useful for identifying patterns of relationships or interactions between people, learning materials or other entities. This page offers resources for those starting out in network analytics and links to introductory courses on conducting network analytics in both R and Python.
Thank you to Drs. Bodong Chen, Sasha Poquet and Tobias Hecking for suggesting some of the resources in this section.
1. Get Oriented
What is Social Network Analysis?
This 2015 video tutorial by ModU at Duke University introduces the unique affordance of social network analysis for analyzing, visualizing and understanding the connections and relationships between data points. Multiple examples of practical questions are used to illustrate the potentials of social network analysis.
The Social Network Perspective and Educational Research
This 2014 introductory chapter by Brian Carolan explains the distinct theoretical and methodological approach of network analysis as a focus on ties between actors and the properties of groups that emerge from these relationships, examining recent advances in the field and applications for education. Chapter one is freely available in the preview of Google Books. The remainder of the Social Network Analysis and Education: Theory, Methods & Applications book details methods, measures, applications and examples for those who wish to go further.
2. Dive Deeper
Social Network Analysis as a Learning Analytics Tool
This 2019 paper by Mohammed Saqr and Ahmad Alamro demonstrates several typical uses of network learning analytics to describe the level and structure of interactions among group participants, identify central and isolated students, and predict academic performance.
Analyzing Learning and Teaching Through the Lens of Networks
This 2020 webinar by Sasha Poquet and Bodong Chen introduces the notion of learning as a networked phenomenon and explores the use of network analysis for understanding learning. It focuses on three specific ways of using network analysis in learning analytics: one, for visualizing connections of all sorts; two, in offering metrics of various dimensions of learning; and three, to develop explanatory theories about learning processes. For each area, case studies of real-world learning and teaching scenarios were presented.
SNA Key Concepts
This 2017 video by Bodong Chen offers useful clarification of many of the specific terminology used in social network analysis. It introduces key concepts of social network analysis, including actors, ties, networks, and matrices as well as the alternative language used to refer to them across different research areas.
3. Tools & Tutorials
Applied Social Network Analysis in Python
This four week course offered by the University of Michigan through Coursera provides a basic introduction to network analysis and the NetworkX Python library. The course covers the motivations for and key concepts of network analysis, ways to measure centrality of a node in a network, models of network generation and the link prediction problem. Basic familiarity with Python is required.
Social Network Analysis Tutorial with R
This 2016 tutorial by Katya Ognyanova introduces the igraph package for network analysis and visualization in R. It provides step-by-step guidance on how to create networks (including reading network data from files), turn networks into igraph objects and plot networks with igraph. For learners who are not familiar with R, a basic introduction to R is also provided at the beginning of the tutorial.
Text mining and natural language processing (NLP) methods in LA use computational linguistic techniques to generate insight into student writing, learning dialogues, teacher talk and other forms of educational text. Start off with some overviews of the uses of text mining in learning analytics and then try out one of several tutorials for conducting text mining in either R or Python.
Thank you to Dr. Jelena Jovanovic for providing some of the resources in this section.
1. Get Oriented
Handbook of Learning Analytics Chapters
Three chapters in the 2017 Handbook of Learning Analytics provide complementary perspectives on the use of texting mining in learning analytics.
The Content Analytics (CA) chapter by Vitomar Kovanovic and colleagues describes a set of methods to evaluate, index, filter, recommend, and visualize digital learning content. Considering both educator-created learning materials and student-created learning products, they provide an overview of key text mining methods useful for analysis.
The Natural Language Processing (NLP) chapter by Danielle Macnamara and colleagues introduces a set of NLP tools commonly used in educational contexts and the linguistic features they index with specific examples of their application.
The Discourse Analytics (DA) chapter by Carolyn Rosé and colleagues summarizes different representations of text and outlines popular unsupervised and supervised analytic methods for DA as well as their strengths and limitations.
2. Dive Deeper
Text Mining for Learning Content Analysis
This 2019 Learning Analytics Summer Institute (LASI) workshop by Jelena Jovanovic serves as an introduction to text mining through the lens of LA. It covers the main steps of the text mining flow, including text preprocessing, text transformation, feature selection, data mining, and interpretation/evaluation and provides concrete examples.
AcaWriter: A Learning Analytics Tool
This 2020 paper by Simon Knight, Antonette Shibani and colleagues introduces the open-source AcaWriter tool, which uses text mining techniques to provide feedback to students on their use of rhetorical moves in their academic writing. The paper describes the tool’s theoretical background and technical implementation as well as discusses three examples of its use.
3. Tools & Tutorials
Text Mining in R
Introduction to Text Analytics with R
This 2019 tutorial by Data Science Dojo provides introductory coverage of analyzing text using R. It focuses on the creation of predictive classification models based on textual features and uses practical examples to walk through all phases of the process. This includes pre-processing textual data, extracting textual features, developing and training classification models using these features, and evaluating accuracy of the trained classification models. All the code and the datasets used in the examples are made available in the description of each video.
Text as Data
This free course by Chris Bail covers a wide range of ways to obtain and analyze textual data. It offers a detailed introduction and tutorials with sample data and R codes for commonly used techniques in collecting, processing, and analyzing text-based data. Topics covered include mining data from application programming interfaces (APIs), screen-scraping, basic text analysis, dictionary-based text analysis, topic modeling, text networks, and word embeddings. Basic familiarity with R is required.
Text Mining with R – A Tidy Approach
This 2020 book by Julia Silge and David Robinson is a great introduction for those who are familiar with R but new to text mining. It introduces the tidy text format as a way to structure text data and offers methods to work with text in tidy format using the tidytext package and other tidy tools in R. Compelling examples of real text mining problems are provided to demonstrate the application of these methods.
Text Mining in Python
Natural Language Processing in Python Tutorial
This 2018 introductory level tutorial by Alice Zhao covers text preprocessing techniques, machine learning techniques and Python libraries for Natural Language Processing. It walks you through all of the steps of a text analysis project using an example in Jupyter Notebook. Some familiarity with programming in Python is required.
Modern NLP in Python Tutorial
This 2016 tutorial by Patrick Harrinson offers an introduction of preparing, modeling, visualizing, and analyzing textual data in Python through examples run on a dataset about user reviews published by the business review service Yelp. It assumes a working knowledge of Python, but does not require any pre-knowledge of text analysis.
Introducing Text Analytics and Natural Language Processing with Python
This free EdX course is a practical and scientific introduction to NLP with Python. It takes you through the overall text analytics process (from data collection and preprocessing to evaluation of the results) while also explaining the underlying science of NLP and how artificial intelligence views language differently from humans. It also discusses the limitations of and provides ethical guidelines for applying NLP to real-world problems. The course materials are useful to both those who are new to Python and those who have Python programming experience.
Visual LA & Dashboards
Visual LA relates to the graphical presentation of data, either to aid in the analysis process or as a tool to communicate the results of analyses to various stakeholders. LA Dashboards are one common format for sharing data visualizations with stakeholders. This section compiles some high quality resources that can guide you in learning about and improving data visualizations and dashboard designs.
Thank you to Dr. Sharon Hsiao for providing some of the resources in this section.
1. Get Oriented
Learning Analytics Dashboards
This 2020 paper by Katrien Verbert, Xavier Ochoa and colleagues describes both early research on visualization and prediction that demonstrated the potential of learning analytics dashboards and current work to integrate participatory design methods, tailor dashboards for particular users and expand the theoretical underpinning for the work. Drawing on an International Conference on Learning Analytics & Knowledge (LAK) 2019 workshop on Visual Approaches to Learning Analytics, they articulate common practices and challenges in the domain and offer an agenda to shape future research.
An introduction to data visualization by Arizona State University that explains what stories can most effectively be told by different types of data visualization, how important the central questions about the data are in determining what type of visualization to use, and how design choices can influence the stories told by the visualization.
2. Dive Deeper
Driving Teachers’ Attention through Educational Data Storytelling
This 2018 paper by Vanessa Echeverria, Roberto Martinez-Maldonado and colleagues presents a human-centered approach to designing learning analytics dashboards focused around the narratives educators construct about student progress using data. In it, they describe how Data Storytelling can help align educational visualizations with intended learning designs and present a case study to explore its effectiveness.
Pitfalls of Learning Analytics Dashboards
This 2017 paper by Ioana Jivet, Maren Scheffel and colleagues offers a systematic review of the literature on learning analytics dashboards, with a particular focus on theoretical foundations and the application of educational concepts. Their critical examination suggests cause for concern as current designs provide problematic frames of reference that lead to a focus on learner competition rather than knowledge mastery.
Towards User-Centered Analytics: User-Adaptive Visualizations
This keynote by Cristina Conati at Learning Analytics and Knowledge Conference 2018 problematizes a one-size-fits-all approach to data visualization and argues for the needs of user-adaptive visualizations. It discusses what user characteristics may impact their experiences with data visualization, how to model these characteristics in real-time using data including eye-tracking measures, and how to use these models to provide adaptive support to improve user experiences with data visualizations.
Visual Analytics: Definition, Process and Challenges
This 2008 paper by Daniel Keim, Gennady Andrienko and colleagues goes beyond the simple question of visualization to highlight the larger purpose of visual analytics and other important elements of it such as data management and human-computer interaction. Add one more sentence
Guiding and Motivating Students Through Open Social Student Modeling
This 2017 paper by I-Han Hsaio and Peter Brusilovsky describes the development of a visual analytics system to present students with a model of their learning progress on specific programming language topics. Following the tradition of Open Student Modelling, the system allows students to navigate through their own model and identify the most relevant learning resources. In addition, the system introduces a social component into the visualization to allow students to compare their own progress with those of the class and individual peers.
3. Tools & Tutorials
Data to Viz
A tool by Yan Holtz and Conor Healy that helps you choose the most appropriate chart type for your data and context. The tool works as a decision tree that is both guided and allows for exploration. Each visualization type is presented with a description of what it is useful to show and common mistakes to be avoided in using it. Links to examples with code in R, Python and D3.js are provided.
Design in LA
The process of designing LA has a critical influence on the tools that result and has recently received a great deal of attention in the field. Recent approaches emphasize the inclusion of stakeholders in the design process, including intended end-users, with specific attention to including members of often overlooked or underrepresented groups.
The resources in this section offer an introduction to human-centered and participatory design of LA.
1. Get Oriented
Human-Centered Learning Analytics
This 2019 paper by Simon Buckingham Shum, Rebecca Ferguson and Roberto Martinez-Maldonado lays out the importance of human-centered design in the field of learning analytics and explains the benefits and challenges of a human-centered approach for addressing issues associated with learning analytics implementations. The paper also provides an overview of five studies about theoretical foundations, techniques, and application of human-centered learning analytics.
Designing LA for Humans with Humans
This 2019 webinar by LEARN Director Alyssa Wise presents a diverse set of examples of the ways that NYU-LEARN is including educators and students in the process of building and implementing learning analytics. Dr. Wise provides examples of how to: involve students in the creation and revision of learning analytics solutions for their own use; work with instructors to align analytically available metrics with valued course pedagogy; and partner with an educational team to design and implement interventions based on at-risk students predictions. Webinar attendees gain a sense of both the conceptual issues and practical concerns involved in designing learning analytics for humans with humans.
2. Dive Deeper
Engaging Students as Co-Designers
This 2020 paper by Juan Pablo Sarmiento, Fabio Campos and Alyssa Wise provides a concrete example of how to involve university students in the design of a LA tool. It describes a series of co-design workshops conducted with students and shares three strategies that were found useful for engaging students in the design process.
Working Together in LA Towards the Co-Creation of Value
This 2019 paper by Mollie Dollinger, Danny Liu and colleagues offers a good overview of the current literature on Human-Centered Design in LA. It clarifies important differences between similar-sounding terms (e.g. user-centred vs.human-centred vs. participatory design) and summarizes the major challenges in adopting such approaches in LA. The paper concludes with a case study of the redesign of an educator-facing analytics system to demonstrate the use and benefits of human-centered approaches to design.
Towards Value-Sensitive Learning Analytics Design
This 2019 paper by Bodong Chen and Haiyi Zhu introduces the theory, methodology, and methods of Value Sensitive Design, with the aim of addressing the ethical and privacy concerns and human values considerations in learning analytics systems. It reports on two case studies where Value Sensitive Design methods are used to 1) investigate an existing learning analytics tool for future tool refinements and 2) design a recommendation system holistically.
3. Tools & Tutorials
6 Techniques for Participatory LA Design
This 2021 workshop led by Juan Pablo Sarmiento offered through the Learning Analytics Learning Network (LALN) offers concrete techniques and methods you can leverage to create participatory workshops with educators and students. It serves as a good introduction to some of the participatory tools that are often used in PD for LA and can help you think through different ways to involve stakeholders in a project.
Co-Designing LA Tools with Learners
This 2018 chapter by Carlos Prieto-Alvarez, Roberto Martinez-Maldonado, and Theresa Anderson describes and provides illustrative examples of a set of tools and techniques that can be used to include learners in the loop of learning analytics innovation. It is particularly aimed at LA practitioners and researchers who are looking for ways to involve students in the design process of LA and enhance the adoption and success of the resulting applications
Ethics & Equity
Learning analytics operate as part of socio-technical systems in which issues of privacy, agency, power, equity, consent and accountability must be considered. Yet these are new areas of thinking for the field.
This section includes a selection of resources to introduce and raise awareness about these issues and a number of tools for taking them into account when designing and implementing LA.
1. Get Oriented
Learning Analytics: Ethical Issues and Dilemmas
This 2013 paper by Sharon Slade and Paul Prinsloo examines early research on learning analytics from a sociocritical perspective to systematically document ethical issues that need to be considered in higher education. Looking at questions surrounding the role of power, the impact of surveillance, the need for transparency, and an acknowledgment that student identity is a transient, temporal, and context-bound construct, the paper proposes six principles as an initial framework to help guide higher education institutions in addressing these ethical issues.
Subversive Learning Analytics
This 2021 paper by LEARN team members Alyssa Wise, Juan Pablo Sarmiento, and Maurice Boothe Jr. introduces the notion of Subversive Learning Analytics as arising from the premise that data should be used as a transformative force for change rather than to perpetuate an often-problematic status quo. It describes how learning analytics researchers and practitioners can take a critical stance to reveal and challenge the power structures and inequities in education.
Free access via SoLAR
Algorithmic Bias in Education
This 2021 video by Ryan Baker and Aaron Hawn offers an overview of what we currently know (and don't know) about different sources of algorithmic bias in education and the kinds of harm to particular subgroups of students that can result. The latter half of the talk offers suggestions to identify and reduce bias by improving data collection practices, developing targeted tools and broadening the community of people who are involved in algorithmic development.
2. Dive Deeper
Empirical Evidence for Algorithmic Bias in Education Wiki
This wiki, maintained by the Penn Center for Learning Analytics, provides a place to find and share peer-reviewed publications on algorithmic bias in educational settings. The research is organized both by the groups studied (e.g. race, ethnicity, gender and other categories) and by the kind of situation investigated (e.g. grade prediction, essay scoring, affect detection, etc.).
(Re)centring Students in Learning Analytics
This 2019 paper by Christine Broughan and Paul Prinsloo discusses the issue of exclusion of students from current learning analytics practices and explores the importance of student-centered learning analytics in response. Using the teachings of Paulo Freire as a framework, they examine ways to include students as partners rather than mere subjects in the collection, analysis and use of their data, thereby recentering students as data owners, rather than data objects.
How Colonial Continuities Underlie Algorithmic Injustices in Education
This 2021 paper by Shamya Karumbaiah and Jamiella Brooks argues that many of the inequities and structures created by colonialism are pervasive in educational and technological systems, and thus should be taken into account when building learning analytics tools. Using the lens of sociotechnical systems and critical race theory, the authors identify a number of different, often surprising, ways in which algorithmic systems replicate and can even deepen the ongoing inequities that arise from coloniality.
3. Tools & Tutorials
Learning Analytics Strategy Toolkit
This 2020 toolkit created by Tyton Partners in partnership with Every Learner Everywhere provides guidance and resources for higher education institutions to implement equity-focused Learning Analytics at scale. Specifically, it characterizes the current state of higher education in using learning analytics to address educational equity, offers evidence-based principles to guide institutional learning analytics implementation, and shares strategies, tools and case studies to support the application of the principles and strategies.
This tool by People + AI Research (PAIR) works as an extension in Jupyter, Colaboratory, and Cloud AI Platform notebooks for conducting model understanding and evaluation tasks. It can be used to analyze and visualize regression and classification model behaviors in hypothetical situations, evaluate the importance of data features, and understand model performance and fairness. Easy-to-follow tutorials and examples are provided to walk you through how to set up and use the tool.
Privacy by Design
This 2011 framework by Ann Cavoukian offers a “design-thinking” perspective to approach the issue of privacy in our present-day information society. Building on the universal principles of Fair Information Practices, Ann Cavoukian establishes seven foundations for Privacy by Design as a framework to guide the protection of privacy in the modern era.