Using metrics mined from group programming repositories to assist in individual guidance

Abstract

Group work projects are a common aspect of software engineering courses in higher education. In these group projects multiple students work on the same assignment to deliver a single software product. These projects serve as both a way of showing their ability, to continue learning their craft, and learning to work in a group. However, it is not a given that each student participates equally or learns new things in relation to programming.
An experimental evaluation was performed at Saxion Hogescholen HBO-ICT in the first year Software Engineering class. In this experiment a tool was introduced in a group programming course that mined metrics from Git repositories and displayed them to students.
The following metrics were analyzed and presented: Lines of code, comments and doc-umentation (.md files) were split according to added, modified and deleted. Git Blame was used to display a division of last modified code by students. Introduction of method complexity and size were displayed, and lastly time of commit was displayed in a plot.
In the course students were required to develop a game in Android over the course of 3 sprints. Teachers were asked to use the tool, sharing it with students to help in their guidance. At the end of the project teachers were interviewed following a semi-structured interview and students were asked to fill out a survey.
The interviews were labeled according to themes and categories. The survey was analyzed and filtered. These results were structured according to three main topics, teachers, students and metrics.
The primary approach by teachers was discussing the metrics with students and asking them to explain the metrics that are being presented. They also used it as way of explaining concepts of software quality using examples from the report. There was a strong desire to use a system such as this in future projects as it saved time.
The overall response of students was positive. Students who saw the tool very late in the project stated they would have liked to have seen it sooner. General responses was being able to see who did how much was seen as positive.
Lines of code, complexity, and Method Size yielded the biggest response among both teachers are students. Lines of Code gave a reasonable indication of productivity, while complexity and method size introduced quality concepts to students. These metrics also allowed teachers to ask more directed questions during guidance sessions.
While a reasonable number of metrics were examined and tested, it was clear that lines of code due to its simplicity is easy to understand and process. Combined with that complexity and method size allowed teachers to compare the lines of code with the ‘quality’ of the code being delivered.
The Hawthorn effect creates a potential positive effect of students being more honest in their productivity, due to student knowledge of being monitored.
However, it was very clear from responses that method such as this should never be used directly in grading. It should always require a teacher interpreting and discussing the metrics with the students.
Recommendations for the future involve better research into which metrics are most suitable in this context. Additionally student behavior could be researched more in depth in regard to how it affects them and whether or not they should have access to the tool.

Date of Award	2 Oct 2020
Original language	English
Supervisor	Efthimia Aivaloglou (Examiner) & Bastiaan Heeren (Co-assessor)