Causal Inference and Bias in Learning Analytics: A Primer on Pitfalls Using Directed Acyclic Graphs

Learning Analytics is an applied field of research with the goal of producing actionable knowledge to improve student learning. This requires knowledge about cause-and-effect. However, randomized experiments, the usual vehicle for causality, are not always feasible nor desirable. Researchers are then left with observational data, from which they are, understandably, hesitant to draw causal inferences. Fortunately, there has been a lot of progress on the topic of causality in the last two decades. One prominent framework uses Directed Acyclic Graphs (DAGs) to graphically reason about cause-and-effect and/or bias.

This primer, authored by Joshua Weidlich, Dragan Gasevic, and Hendrik Drachsler, published in the Journal of Learning Analytics, introduces DAGs to Learning Analytics. Using fictitious and published examples, we show how DAGs are a principled approach to a) improve causal inferences for varied research questions and b) identify main sources of bias, alongside approaches to circumvent bias in the future.