Using peer-support to connect learning network participants to each other: An interdisciplinary approach

A large-scale experiment is presented which examines the feasibility of using a new method of peer-support called Ad Hoc Transient Groups (AHTGs) to foster Social Capital of Learning Network participants. In AHTGs participants that have a request are helped by other participants in a dedicated private space (‘ad-hoc’) which exists for a limited amount of time only (‘transience’). To test the hypotheses that AHTGs foster Social Capital, AHTGs were introduced to a subset of the eTwinning Learning Network (+130.000 teachers). To validate the results, a No-Intervention group and a comparison group that used a Forum to ask questions instead of AHTGs were also examined. Results show that AHTGs seem to foster Social Capital on the level of Relationship Characteristics and Mutual Support. Results on Sense of Connectedness were inconclusive. It is concluded that AHTGs have a decentralizing effect, making the network less dependent on a few key participants. Furthermore, AHTGs have clearly been shown to have a low threshold to ask a question. Within the Forum group only a few core participants asked questions, yet many participants replied. It is concluded that AHTGs foster Social Capital in a different way when compared to a forum.


Introduction
In online learning networks it is imperative that participants are connected to each other. Being connected allows participants more readily to learn and develop. To take this point further, one can argue that for a Learning Network to build up and sustain its communities, it needs to foster its Social Capital (Fetter, Berlanga, & Sloep, 2010). Social Capital represents the relationships people have between them and the benefits that can be obtained by building on these relationships (Burt, 2005;Portes, 1998). To foster Social Capital in a Learning Network it is especially important to develop three pillars of Social Capital, namely: Relationship Characteristicshow are people connected, Sense of Connectednessdo people feel part of the network, and Mutual Supportare people using the established relationships . Sloep (2009) proposes to use a peer-support mechanism called Ad Hoc Transient Groups (AHTGs) to foster social capital. The idea behind AHTGs is to create many short-term moments of contact between different combinations of participants within a network (Berlanga, Sloep, Kester, Brouns, & Koper, 2008) in order to increase in number and decentralization of relationships between participants. It is also expected that by introducing AHTGs the sense of connectedness will increase and that the mutual support between participants be enhanced. In Ad Hoc Transient Groups (AHTGs) participants that have a request are helped by other participants in a dedicated private space ('ad-hoc') and for a limited amount of time only ('transience') ( Figure 1). The nature of the request depends on the participant's needs as well as the network.
Requests can be questions about factual knowledge or best practices. Yet requests might also be doing a short term class project together or reaching out to others purely from a socialization point of view. To date work on AHTGs has focused on relatively small groups (Van Rosmalen et al., 2006). However, we feel that these results cannot be directly translated to large scale networks.

Figure 1 -Ad Hoc Transient Groups
In this paper we report on a large-scale experiment where we introduced a peer-support service using AHTGs to teachers of the eTwinning Learning Network. The eTwinning network (http://www.etwinning.net) consists of more than 130,000 teachers (named eTwinners) from all over Europe. Through this network, eTwinners come together to complete projects ranging from providing their students opportunities to learn from each other to improving their own teaching practices.
This study tests the hypotheses that introducing AHTGs in a Learning Network will have the following main effects : 1. Improving Relationship Characteristics (Coleman, 1990).
a. The number of relationships increases.
b. The weak/strong relationships balance shifts towards the weak relationships.
c. The relationships spread out in a decentralized way, rather than in a typical star pattern.

Mutual Support increases
To measure whether the introduction of AHTGs had a measurable effect, a No-Intervention group was used. However, as this is the first time AHTGs are used in a existing network an additional group was included which had access to a Forum instead of the AHTGs.
As Forums are well known in eTwinning (and in general as a peer-support tool), we surmised this would provide a good benchmark. An interdisciplinary design was chosen as we agree with Molenaar (2010) in that interdisciplinary approaches are severely lacking in the research field on Social Capital.
In the following section we discuss our methodology, including participants and provide a short description of the prototype used. In addition, we explain which measurements have been taken. Thereafter, the results of the experiment are reported and discussed, including limitations.
Finally, general conclusions are drawn and future research is elaborated upon.

Participants
In preparation of this experiment, an eTwinning datadump was used to identify which eTwinners would be most appropriate to include in the experiment. This datadump was obtained through the TellNet project (http://www.tellnet.eun.org) and included the profiles and usage logging of all eTwinners over the previous five years. The new datadump was especially made for this experiment at the pre-and post-measuring times. Based on this datadump we made the decision to invite eTwinners who teach in one of the four main eTwinning teaching topics.
Invitations were sent out via a newsletter and were included on the eTwinning personal page.
The invitation included links to the pre-measurement questionnaire which was administered via Google Docs. All invitations and questionnaires were available in four languages (English, French, Spanish, and German). As explained in the invitation, completing the questionnaire was also an agreement to participate in our study and provided us the necessary authorization for linking the participants' email to the IDs as present in the datadump. This was needed for identification, measurements, and matching data for those in the AHTG group.
In total, 819 pre-questionnaires were completed. After matching the thus obtained email addresses with the eTwinning database and after the removal of duplicates, 691 participants were left (see Table 1). Participants who taught one of the four teaching topics mentioned above were randomly divided over the AHTG and Forum group until each group consisted of 230 participants. The remaining 231 participants were assigned to the No-Intervention group. Once assigned, participants in the AHTG and Forum groups received an invitation to start using the prototype. Participants in the No-Intervention group received an email explaining that they were in the No-Intervention group. It should be noted that participants in the No-Intervention group still had full access to eTwinning and its communication/collaboration tools. After the experiment, all participants received another email inviting them to fill in the final questionnaire.
For the AHTG and Forum group this included an evaluation of the software service as well. The post-questionnaire was filled in by 375 participants (AHTG=103, Forum=118, No-Intervention=154). All subsequent analyses that used data from the questionnaire only used participants who filled in both questionnaires.

Tools
In order to facilitate the use of AHTGs, a peer-support service was created (Fetter, Berlanga, & Sloep, 2009). In addition, a built-in Forum of our testing environment was used. All participants in both the AHTG and Forum groups were asked to complete a profile.

AHTG group
The service creates small groups of peers with similar interests and problems (i.e. an Ad-Hoc Transient Group). Before they used the service, participants were invited to follow a tutorial (made available via Slideshare) and respond to questions in their profile with some relevant information (months active in eTwinning, number of projects, languages spoken and teaching experience).Once the profile was filled in, the participant could go to the application and ask his or her question (Figure 2) or, if they were invited to do so, answer the question of someone else.

Figure 2 -Asking a question
To ask a question, participants had to fill in a form as shown in figure 2. Next to a short title and an explanation of the question, participants had to choose the subject of the question and indicate in which language the question was written. Regarding the subject, participants had six options to choose from, specifically: About eTwinning, Projects, and the four teaching topics participants were selected on. After completing the form, the service's matching system (see Figure 3) generates a list of participants who it estimates are likely to be able to help, including the matching scores that represent their suitability (see Figure 4). To find the most suitable participants, the matching system takes into account all the information derived from the eTwinning datadump (e.g. Language preferences, number of projects, time active in eTwinning), the subject and language of the question, and the profiles of others (see Figure 3 for the whole algorithm). The service then calculates sub-scores, which are accumulated into a single matching score. Sub-scores are provided based on the following categories (Fetter, Berlanga, & Sloep, 2012):  Availability: The number of satisfactory answers given by a particular participant relative to the average number of satisfactory answers given by all participants. Thus, the more requests a participant has accepted and answered, the lower his or her availability score is.
 Content competency: Participants whose expertise area is/are similar to the question category receive a positive sub-score.
 Similarity: A correlational score is calculated based on the participants' subject(s). The higher the correlation, the more similar they are.
 Language: Based on the language of the question, only those participants who have the same language selected in their profile are considered in the matching exercise. Figure 3 shows the matching algorithm. Once the overall matching score is calculated, the service creates a list with the top 25 best matches ( Figure 4). The participants can then choose from this list and invite others to help, or they can let the service do this for them. In the latter case, e-mail invitations are sent out to the first three participants who appear on the list. The invitation includes the description of the problem and a request to join a shared workspace. If the participants invited do not reply within three days, or if they decline the invitation, the service sends out invitations to the next three participants on the list. The process stops either when a sufficient number of participants accept or the list has been depleted and insufficient numbers of peers have indicated their willingness to help. In case participants chose the manual option, the invitational emails are only sent to those marked. In case of no response, no new invitations are sent.
It is important to stress that the addition of the manual invitation option was only included in the system after a stakeholder consultation in a workshop on the eTwinning Conference Sevilla in 2009. Participants in this workshop were well-connected eTwinners who were knowledgeable about the network and their fellow eTwinners. They stressed the importance of eTwinners having the ability to choose whom to invite to answer questions. Taking this feedback into account together with the fact that at the time the design for the tool assumed large numbers of participants, it was deemed worth the risk of inclusion. We argued that the chance of having known participants presented through the matching was sufficiently low when considering the size of the eTwinning network. Once accepted, these participants and the questioner thus form an AHTG. In the workspace, they discuss the question that was posted and interact with each other to clarify it.
Participants have access to AHTGs through the "My Questions and Answers" screen.
This is the main screen of the service. It gives access to: • the currently active question with its associated group space; • a list of recent activities (questions asked and answered); • the list of current collaborators; • a box through which one can ask new questions ( Figure 2); • general preferences for the service; • a flow chart as a reference for participants ( Figure 4).

Figure 5 -Flow chart
Once the question has been discussed to the questioner's satisfaction, he or she can flag the question as closed. If no answer was given, or the answer was unsatisfactory, the questioner can either resubmit the question or close it and flag it as answered unsatisfactorily. Upon closing the question, the questioner is asked to provide feedback to those who were involved and the participants who participated in the AHTG are added to the "people I worked with" list.

Forum group
The Forum group used the same Liferay environment as did the AHTGs group, except that they had a built-in forum instead of the AHTGs prototype. Separate tutorials were made using Slideshare. As a preparation, the Forum was filled with six threads corresponding with the topics participants could choose in the AHTGs group (no new topics could be added by participants). Participants were instructed to start a new post for every new question asked.

Relationship Characteristics
In order to test the hypotheses regarding the Relationship Characteristics, a Social Network Analysis (SNA) approach is taken. Through the use of SNA it is possible to measure and compare the spread of the relationships. In addition, these analyses can provide the number of relationships between participants and their strengths. Relationships (called 'degrees' in SNA terms) between the eTwinners were calculated for each group, based on a datadump provided by eTwinning before and after the experiment. For the AHTG and Forum groups, relationships gained through the service were added. The number of relationships was based on eTwinners' contacts, posts they left on each others' personal message wall, as well as with whom they worked together in a project. For each contact, post, and shared project the relationship between two participants was increased by 1. For example, an eTwinner who has someone in their contacts (+1), posted something on their wall (+1), and once completed a project with this eTwinner (+1), would have a relationship with this eTwinner of strength 3.  The second group of measurements is derived from the network using UCINET (https://sites.google.com/site/ucinetsoftware/home) and provides a more in depth look at the role of each participant. These measurements are participants' flow-betweenness, Clustering coefficient, and their level of constraint. Flow-betweenness is an adaptation of the classical betweenness measurement from Freeman (Freeman, 1978). Freeman's Betweenness is based on the shortest paths between participants in the network. It gives an estimate of how dependent the network is on some core participants with respect to the flow of information through the network. However, communication flow often does not follow the shortest path. Therefore the Flow-betweenness also takes into consideration alternative paths (Newman, 2005). Like Freeman's betweenness, this calculation produces a value for each participant, indicating overall the network's dependence on a small number of participants for the information to be able to flow throughout the network. The clustering coefficient reflects how well each participant's neighbors are connected. This gives an indirect insight into the growth of the network and the relationships established (Soffer & Vázquez, 2005). The constraint measurement indicates the importance of the participant's connections to others who are interconnected between each other as well (Burt, 1995). In other words, if a participant has a low constraint, this means that her connections are also connected to each other.
Finally, descriptive statistics and visualizations are utilized to represent our network. For this, NetDraw (as part of UCINET) was used. We analyze how many participants are connected to the largest component in the network and how this changes between pre-and postmeasurement. Using UCINET a k-core analysis is run which determines which participants belong to which groups (component) and how many participants are part of each component.
The main information gained from this analysis provides an insight into how many participants are connected to the heart of the network.

Sense of Connectedness
To measure the Sense of Connectedness participants experienced, we adapted the Classroom Community Scale developed by Rovai (2002). This questionnaire consists of two subscales: Sense of Connectedness; and, Sense of Learning. For our analyses only the Sense of Connectedness subscale was included. Questions were altered to fit the eTwinning network and the 5 point Likert scales were changed into 9 point scales (1-strongly disagree to 9-strongly agree) to get a more precise measurement and for the scales to be in line with the other questions asked. Questions were also added to the Sense of Connectedness questionnaire to measure participants' connectivity in the eTwinning network. These measurements are used as extra information for interpretation of the results, especially those of the SNA.

Mutual Support
Mutual support reflects the number of questions asked as well as the number of answers given and the general willingness to help. To measure mutual support we looked at (a) the number of questions asked, (b) the number of answers given, (c) the ratio between a and b, and (d) the number of unique participants asking / answering a question. Data is obtained through logging embedded in the AHTG tool and Forum.

Relationship Characteristics
Parametric tests Table 3 shows that there are multiple significant interactions between groups. For each significant interaction, a difference score was calculated. ANOVAs with post-hoc tests (Bonferroni corrected) were conducted to specify the exact differences between the groups.    Table 4 shows a significant increase in flow-betweenness and clustering coefficient. At the same time there was a significant decrease for the constraint variable. However, the follow-up tests revealed that only for the clustering coefficient was there a significant effect of group. Follow-up pairwise comparisons, Bonferroni corrected, reveal a significant increase in clustering coefficient for both the AHTG (p < 0.05) and Forum (p < 0.01) group when compared to the No-Intervention group. No difference in increase was found between the AHTG and Forum group.

Descriptive Results
The descriptive results are composed of a largest connected component analysis and network visualizations (figure 6). The largest connected component analysis was run in UCINET. The results (see Table 5) show that for all groups the number of components went down and the number of participants connected to the largest component increased. Yet, the results also indicate that where the proportion of participants connected to the largest component stays the same for the No-Intervention group, it increases for both the AHTG and Forum group.

Sense of Connectedness
The pre-and post-measure Sense of Connectedness questionnaire was filled in by 375 participants. According to our hypotheses, we expected the AHTG and Forum group to increase their Sense of Connectedness and the No-Intervention group to stay the same. However, a mixed ANOVA showed a negative significant difference over time for all groups F (-2.87) = 6.12, p = .01 (see Table 6). In addition, there was no significant difference between groups at pre-F (0.32) = 0.24, p = .78 or post-F (0.08) = 0.05, p = .95 measure. T-tests were run for each question separately in order to check whether there were differences between the groups on question level. No differences were found between groups. Further follow-up analyses showed that specifically participants with a higher initial sense of connectedness filled in the postquestionnaire F (6.64) = 3.61, p = .058. participants provided an answer in the AHTG group than in the Forum group.

Evaluation and Additional Questions Additional Questions
As there were no differences between groups with regard to the additional questions asked, results were pooled together in order to identify possible overall changes (see Table 8).
Interestingly, as can be seen from the results in Table 8, there was a general decline in contacting other eTwinners. Other interesting results are that our respondents seem content with the type and amount of contact they have with other eTwinners. Finally, eTwinners seem to prefer a mix of support (official eTwinning support and their peers) and are very willing to help other eTwinners.

Evaluation
The results of the evaluation can be seen in table 9 below. While in general positive about the AHTGs, the Forum was clearly evaluated better. Table 9. Mutual Support Results AHTG Forum Difference Overall, the support provided by the 'Ask anything' service is relevant to my eTwinning activities.
5.54 6.11 -0.57* Overall, I believe that the 'Ask anything' service provides adequate support.
6.07 6.69 -0.62* Overall, I believe that the 'Ask anything' service is useful.
6.04 6.41 -0.37 I did not have to wait long for an answer using the 'Ask anything' service. 5.03 6 -0.97** Overall, using the 'Ask anything' service requires significantly less mental effort to obtain answers than when using other eTwinning services.

5.4
5.99 -0.59* It is easy to learn how to use the 'Ask anything' service. 6.48 6.72 -0.24 It is easy to navigate through the 'Ask anything' service.
5.92 6.52 -0.6* I would like to use the service after the pilot.
6.23 6.74 -0.51* I would like to see an improved version integrated into eTwinning.
7.36 7.18 0.18 I would recommend the system to other eTwinners.

Relationship Characteristics
Based on the social network analyses carried out, three conclusions can be drawn: (1) Overall the AHTG group and the Forum group show signs of decentralization, (2) the Forum group has a significant increase in weak relationships, whereas the AHTG group only has a significant increase in strong relationships, and (3) the Forum group has a significant increase in relationships and unique relationships. Each of these main relationship characteristics results will be discussed below.
When looking at the visualizations (figure 6), the decentralization seems most apparent for the AHTG group, followed by the Forum group. Evidence of decentralization is found in the density increases. As the results show, both the AHTG and the Forum group had a significant increase in network density, whereas the No-Intervention group had not. In relation, the increase in clustering coefficient shows that participants either connected to other already connected participants, or that their existing neighbours increased their number of connections. In addition, proof of decentralization for the AHTGs is also found in the higher number of individual participants who asked a question. Finally, the decentralization hypothesis is strengthened by the fact that for the AHTG and Forum group the proportion of participants who belong to the largest network component increased, whereas this remained constant for the No-Intervention group.
Thus, the social network structure that emerged went in the opposite direction of the typical starpattern (Jones & Czerniewicz, 2011). This direction is theorized to be beneficial for the stability of the social network structure as it makes the network less dependent on a small group of participants and thus fosters Social Capital .
As expected, participants who were in the Forum group significantly increased their number of relationships as well as strengthened pre-existing ones. However, the increase in new relationships was not found in the AHTG group. Instead, there was a significant increase in the strengthening of existing relationships. This result is surprising as the use of a matching system was especially meant for participants to come into contact with unknown eTwinners.
The change in the ratio between weak and strong relationships is related to this. At the beginning of the experiment the ratio of weak and strong relationships was the same for all groups. Roughly speaking, this equalled a ratio of four to one, for each four weak relationships a participant had one strong relationship. However, this ratio clearly changed for both the Forum and the AHTG group. Whereas in the Forum group the number of weak relationships increased, in the AHTG group the number of strong relationships increased. A possible explanation for this unexpected result will be elaborated in the discussion on Mutual Support (below). For now, suffice to say, the large difference in the number of answers between the AHTG and Forum group has something to do with it.
While these findings need further research, the most likely explanation with regard to the AHTGs groups' increase in strong relationships seems to lie in the tool. As explained in the methodology section, we had included a separate step with regard to the matching where participants were able to choose peers from a list or let the system do it for them. Nearly half the time (44%), participants choose to manually select their peers. It seems there is a fair chance participants chose those they already knew, thus undermining the initial goal of the matching system. This seems likely as nearly all participants are clearly active, core eTwinners with many pre-existing contacts. Yet the tool was designed especially for those with little to no contact.
However, as we were only allowed to use the newsletter and eTwinners' personal eTwinning page for invitations, the chances of actually reaching those with little to no relationships was small at best. Together with a limited number of participants per group resulted in quite high chances that the matching list would contain at least some known participants. However, as was mentioned in the methodology chapter, we felt we had no choice but to include this manual option because of the results of the stakeholders evaluation at the eTwinning conference.
In addition, we had to conclude during the test-phase that the matching algorithm reacted too slowly. Participants' availability score was only adjusted after accepting a question, giving an answer, and the question being closed. As this process could take days, the matching algorithm would keep the same participants at the top of the matching list. So while the matching did provide different lists based on the categories, these lists were populated by the same participants as in the first days. While regrettable, this problem resolved itself to some extent over time as, questions were answered and closed.

Mutual Support
As mentioned above, one of the most eye-catching results is the staggering number of answers given in the Forum group, specifically when compared to the AHTG group. As the number of answers is of crucial importance in the social network analyses, this difference in answers has undoubtedly had a strong effect on these analyses. However, even with this difference in answers, the hypothesis regarding the increase in mutual support seems to be confirmed. Indeed, it is very clear from the results that participants in the AHTG group did not only ask more questions, they were also asked by more individuals. This finding is crucial for AHTGs as it shows that they enable participants to easily ask a question and, more importantly, has more participants join in. This is important, as once inactive participants become active, they tend to stay active (Neelen & Fetter, 2010) In itself the difference in answers would have been a result indicating that a Forum elicits more responses. Yet, in hindsight we see that the most likely difference between the groups comes from the choice of keeping the AHTGs private and the Forum open. To clarify, in the AHTGs, only those invited were able to even see the question, let alone answer it. On the other hand, in the Forum each question was available to all participants. We chose to keep AHTGs private, in order to lower the threshold for participants to ask questions. Yet, we had not anticipated that this would result in such a big difference in answers, especially from a Social Exchange Theory perspective which predicts that the larger the group (like the Forum) the higher the chance of diffusion of responsibility resulting in less response (Gleitman, Fridlund, & Reisberg, 1998).

Sense of Connectedness
The results on the sense of connectedness show a significant decline for all groups. This difference was uniform for all Sense of Connectedness questions. However, many participants did not fill in the second questionnaire, which might have had a distorting effect. It seems likely that some major influence in eTwinning itself is the reason for this negative change and could very well have snowed over any changes the Forum and AHTG group might have had. Indeed, we know that just before the experiment started, eTwinning was completely redesigned. As most eTwinners log in infrequently, the first exposure to the redesigned environment could easily have come only after our pre-measurement. At the same time, as we were only allowed to recruit participants through means normally only accessed by avid users of eTwinning, our sample typically did not have many participants with a low sense of connectedness. It is likely that a regression-effect occurred, that is, that our participants already had a high sense of connectedness which is unlikely to change significantly precisely because it already is at such a high level.

Evaluation and Additional Questions
As is clear from the results, the Forum group performed better than the AHTG group. Of special interest are the large differences between groups with regard to waiting times and satisfaction. The difference in waiting time scores strengthens the notion that the AHTGs matching algorithm worked too slowly as previously stated. The overall satisfaction difference is not surprising given how many questions in the AHTG group were unanswered. In addition, the lower satisfaction could also point to general user interface issues, which might have to be improved in future versions.

Conclusions
In this paper, we report on the first introduction of peer-support through Ad Hoc Transient Groups (AHTGs) in the existing large scale Learning Network eTwinning. The aim of introducing AHTGs was to foster the Learning Networks' Social Capital. Specifically, AHTGs were hypothesized to improve the relationship characteristics, raise the sense of belonging to the community, as well as increase overall mutual support. These hypotheses reflect the interdisciplinary approach taken in this experiment. Methodologies used range from Social Network Analyses to Sociology and Data Mining. To date work on AHTGs has focused on relatively small groups. We would argue that the results of such studies could not be simply translated to or directly compared in the context of a large-scale Learning Network such as this. Therefore, we used an experimental design which included next to AHTGs and a No-Intervention group also a Forum group. The inclusion of a Forum was seen as a good comparison, as this is a well-known, established method for peer-support.
Overall we can conclude that the introduction of AHTGs shows clear signs of fostering Social Capital with regard to the mutual support and the relationship characteristics. As we have shown, AHTGs stimulate more individual participants to ask a question and thus potentially get involved. This is important, as once inactive participants become active, they tend to stay active.
Furthermore, AHTGs had a decentralizing effect on the relationships between participants.
Thus, the social network structure that emerged went in the opposite direction of the typical starpattern. This direction is theorized to be beneficial for the stability of the social network structure and thus fosters Social Capital. In relation, AHTGs seem to foster Social Capital in different ways than the Forum. This leads to an interesting new hypothesis which suggests that Forums and AHTGs cater for different needs and might be most valuable in a Learning Network when presented together.
We suggest that future research should compare AHTGs and Forums directly, in order to gain a deeper understanding in what ways they both can benefit a Learning Network and how they work together. To this end, an experiment to address these issues has already commenced.
Furthermore, as we were unable to properly test the Sense of Connectedness hypotheses, future research might want to start with more small scale experiments. Finally, improvements can be made to the matching algorithm and the AHTGs peer-support service. These changes will also be implemented in the previously mentioned experiment.