Cohesion network analysis of CSCL participation

Mihai Dascalu, Danielle S. McNamara, Stefan Trausan-Matu, Laura Allen

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

The broad use of computer-supported collaborative- learning (CSCL) environments (e.g., instant messenger– chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.
Original languageEnglish
Pages (from-to)604-619
JournalBehavior Research Methods
Volume50
Issue number2
DOIs
Publication statusPublished - 2017
Externally publishedYes

Fingerprint

Learning
Blogging
Regression Analysis
Technology
Participation
Collaborative Learning
Cohesion
Network Analysis
Learning Environment
Grounded Theory
Power (Psychology)
Interaction
Instant
Modeling
Online Communities
Polyphony
Tutor
Dialogism
Rating
Evaluation

Keywords

  • Cohesion network analysis
  • Computer-supported collaborative learning
  • Cohesion-based discourse analysis
  • Participation evaluation
  • Dialogism
  • Polyphonic model

Cite this

Dascalu, M., McNamara, D. S., Trausan-Matu, S., & Allen, L. (2017). Cohesion network analysis of CSCL participation. Behavior Research Methods, 50(2), 604-619. https://doi.org/10.3758/s13428-017-0888-4
Dascalu, Mihai ; McNamara, Danielle S. ; Trausan-Matu, Stefan ; Allen, Laura. / Cohesion network analysis of CSCL participation. In: Behavior Research Methods. 2017 ; Vol. 50, No. 2. pp. 604-619.
@article{5e788253a92e43409cb59dc16167cdcb,
title = "Cohesion network analysis of CSCL participation",
abstract = "The broad use of computer-supported collaborative- learning (CSCL) environments (e.g., instant messenger– chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54{\%} of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.",
keywords = "Cohesion network analysis, Computer-supported collaborative learning, Cohesion-based discourse analysis, Participation evaluation, Dialogism, Polyphonic model",
author = "Mihai Dascalu and McNamara, {Danielle S.} and Stefan Trausan-Matu and Laura Allen",
year = "2017",
doi = "10.3758/s13428-017-0888-4",
language = "English",
volume = "50",
pages = "604--619",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer",
number = "2",

}

Dascalu, M, McNamara, DS, Trausan-Matu, S & Allen, L 2017, 'Cohesion network analysis of CSCL participation', Behavior Research Methods, vol. 50, no. 2, pp. 604-619. https://doi.org/10.3758/s13428-017-0888-4

Cohesion network analysis of CSCL participation. / Dascalu, Mihai; McNamara, Danielle S.; Trausan-Matu, Stefan; Allen, Laura.

In: Behavior Research Methods, Vol. 50, No. 2, 2017, p. 604-619.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Cohesion network analysis of CSCL participation

AU - Dascalu, Mihai

AU - McNamara, Danielle S.

AU - Trausan-Matu, Stefan

AU - Allen, Laura

PY - 2017

Y1 - 2017

N2 - The broad use of computer-supported collaborative- learning (CSCL) environments (e.g., instant messenger– chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.

AB - The broad use of computer-supported collaborative- learning (CSCL) environments (e.g., instant messenger– chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.

KW - Cohesion network analysis

KW - Computer-supported collaborative learning

KW - Cohesion-based discourse analysis

KW - Participation evaluation

KW - Dialogism

KW - Polyphonic model

U2 - 10.3758/s13428-017-0888-4

DO - 10.3758/s13428-017-0888-4

M3 - Article

VL - 50

SP - 604

EP - 619

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

IS - 2

ER -