Towards Big Data in Education: the case at the Open University of the Netherlands

This chapter describes the architecture for an enterprise level big data infrastructure at the Open University of the Netherlands. This infrastructure aims to capture dispersed data from various heterogeneous data sources into a single ledger of immutable data, which allow the development of so called ‘smart services’. Four major requirements for this infrastructure are derived: scalability, availability, reliability and flexibility. These requirements together with the sheer size of the ledger lead to an architecture that is horizontal scalable by using distributed data systems . Next, the Lambda and Kappa reference architectures are discussed as they both meet the requirements. However, the authors argue that the Kappa architecture is more appealing due to a reduced complexity by using a single technology stack. Finally the resulting architecture is outlined in detail.