Assessing User Expertise in Spoken Dialog System Interactions

Eugenio Ribeiro; Fernando Batista; Isabel Trancoso; Jose Lopes; Ricardo Ribeiro; David de Matos

doi:10.1007/978-3-319-49169-1_24

Assessing User Expertise in Spoken Dialog System Interactions

Eugenio Ribeiro, Fernando Batista, Isabel Trancoso, Jose Lopes, Ricardo Ribeiro, David de Matos

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

Abstract

Identifying the level of expertise of its users is important for a system since it can lead to a better interaction through adaptation techniques. Furthermore, this information can be used in offline processes of root cause analysis. However, not much effort has been put into automatically identifying the level of expertise of an user, especially in dialog-based interactions. In this paper we present an approach based on a specific set of task related features. Based on the distribution of the features among the two classes – Novice and Expert – we used Random Forests as a classification approach. Furthermore, we used a Support Vector Machine classifier, in order to perform a result comparison. By applying these approaches on data from a real system, Let’s Go, we obtained preliminary results that we consider positive, given the difficulty of the task and the lack of competing approaches for comparison.

Original language	English
Title of host publication	Advances in Speech and Language Technologies for Iberian Languages
Subtitle of host publication	Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings
Editors	Alberto Abad, Alfonso Ortega, António Teixeira, Carmen García Mateo, Carlos D. Martínez Hinarejos, Fernando Perdigão, Fernando Batista, Nuno Mamede
Publisher	Springer Nature Switzerland AG
Pages	245-254
ISBN (Electronic)	978-3-319-49169-1
ISBN (Print)	978-3-319-49168-4
DOIs	https://doi.org/10.1007/978-3-319-49169-1_24
Publication status	Published - Nov 2016
Externally published	Yes
Event	International Conference on Advances in Speech and Language Technologies for Iberian Languages - Lisbon, Portugal Duration: 23 Nov 2016 → 25 Nov 2016 https://link.springer.com/conference/iberspeech

Publication series

Series	Lecture Notes in Computer Science
Volume	10077
ISSN	0302-9743

Conference

Conference	International Conference on Advances in Speech and Language Technologies for Iberian Languages
Abbreviated title	IberSPEECH 2016
Country/Territory	Portugal
City	Lisbon
Period	23/11/16 → 25/11/16
Internet address	https://link.springer.com/conference/iberspeech

Keywords

User expertise
Let’s Go
SVM
Random Forest

Access to Document

10.1007/978-3-319-49169-1_24

1 Finished

Rage: Realising an Applied Gaming Eco-system
Westera, W., Georgiadis, K., Saveski, G., van Lankveld, G., Bahreini, K., van der Vegt, W., Berkhout, J., Nyamsuren, E., Kluijfhout, E. & Nadolski, R.
1/02/15 → 31/07/19
Project: Research

Cite this

Ribeiro, E., Batista, F., Trancoso, I., Lopes, J., Ribeiro, R., & de Matos, D. (2016). Assessing User Expertise in Spoken Dialog System Interactions. In A. Abad, A. Ortega, A. Teixeira, C. García Mateo, C. D. Martínez Hinarejos, F. Perdigão, F. Batista, & N. Mamede (Eds.), Advances in Speech and Language Technologies for Iberian Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings (pp. 245-254). Springer Nature Switzerland AG. Lecture Notes in Computer Science Vol. 10077 https://doi.org/10.1007/978-3-319-49169-1_24

Ribeiro, Eugenio ; Batista, Fernando ; Trancoso, Isabel ; Lopes, Jose ; Ribeiro, Ricardo ; de Matos, David. / Assessing User Expertise in Spoken Dialog System Interactions. Advances in Speech and Language Technologies for Iberian Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings. editor / Alberto Abad ; Alfonso Ortega ; António Teixeira ; Carmen García Mateo ; Carlos D. Martínez Hinarejos ; Fernando Perdigão ; Fernando Batista ; Nuno Mamede. Springer Nature Switzerland AG, 2016. pp. 245-254 (Lecture Notes in Computer Science, Vol. 10077).

@inproceedings{a6924e8a08324e3682a0cca3d9b04fe5,

title = "Assessing User Expertise in Spoken Dialog System Interactions",

abstract = "Identifying the level of expertise of its users is important for a system since it can lead to a better interaction through adaptation techniques. Furthermore, this information can be used in offline processes of root cause analysis. However, not much effort has been put into automatically identifying the level of expertise of an user, especially in dialog-based interactions. In this paper we present an approach based on a specific set of task related features. Based on the distribution of the features among the two classes – Novice and Expert – we used Random Forests as a classification approach. Furthermore, we used a Support Vector Machine classifier, in order to perform a result comparison. By applying these approaches on data from a real system, Let{\textquoteright}s Go, we obtained preliminary results that we consider positive, given the difficulty of the task and the lack of competing approaches for comparison.",

keywords = "User expertise, Let{\textquoteright}s Go, SVM, Random Forest",

author = "Eugenio Ribeiro and Fernando Batista and Isabel Trancoso and Jose Lopes and Ricardo Ribeiro and {de Matos}, David",

year = "2016",

month = nov,

doi = "10.1007/978-3-319-49169-1_24",

language = "English",

isbn = "978-3-319-49168-4",

series = "Lecture Notes in Computer Science",

publisher = "Springer Nature Switzerland AG",

pages = "245--254",

editor = "Abad, {Alberto } and Ortega, {Alfonso } and Teixeira, {Ant{\'o}nio } and {Garc{\'i}a Mateo}, {Carmen } and {D. Mart{\'i}nez Hinarejos}, {Carlos } and Perdig{\~a}o, {Fernando } and Batista, {Fernando } and Mamede, {Nuno }",

booktitle = "Advances in Speech and Language Technologies for Iberian Languages",

address = "Switzerland",

note = "International Conference on Advances in Speech and Language Technologies for Iberian Languages, IberSPEECH 2016 ; Conference date: 23-11-2016 Through 25-11-2016",

url = "https://link.springer.com/conference/iberspeech",

}

Ribeiro, E, Batista, F, Trancoso, I, Lopes, J, Ribeiro, R & de Matos, D 2016, Assessing User Expertise in Spoken Dialog System Interactions. in A Abad, A Ortega, A Teixeira, C García Mateo, C D. Martínez Hinarejos, F Perdigão, F Batista & N Mamede (eds), Advances in Speech and Language Technologies for Iberian Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings. Springer Nature Switzerland AG, Lecture Notes in Computer Science, vol. 10077, pp. 245-254, International Conference on Advances in Speech and Language Technologies for Iberian Languages, Lisbon, Portugal, 23/11/16. https://doi.org/10.1007/978-3-319-49169-1_24

Assessing User Expertise in Spoken Dialog System Interactions. / Ribeiro, Eugenio; Batista, Fernando; Trancoso, Isabel; Lopes, Jose; Ribeiro, Ricardo; de Matos, David.

Advances in Speech and Language Technologies for Iberian Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings. ed. / Alberto Abad; Alfonso Ortega; António Teixeira; Carmen García Mateo; Carlos D. Martínez Hinarejos; Fernando Perdigão; Fernando Batista; Nuno Mamede. Springer Nature Switzerland AG, 2016. p. 245-254 (Lecture Notes in Computer Science, Vol. 10077).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - Assessing User Expertise in Spoken Dialog System Interactions

AU - Ribeiro, Eugenio

AU - Batista, Fernando

AU - Trancoso, Isabel

AU - Lopes, Jose

AU - Ribeiro, Ricardo

AU - de Matos, David

PY - 2016/11

Y1 - 2016/11

N2 - Identifying the level of expertise of its users is important for a system since it can lead to a better interaction through adaptation techniques. Furthermore, this information can be used in offline processes of root cause analysis. However, not much effort has been put into automatically identifying the level of expertise of an user, especially in dialog-based interactions. In this paper we present an approach based on a specific set of task related features. Based on the distribution of the features among the two classes – Novice and Expert – we used Random Forests as a classification approach. Furthermore, we used a Support Vector Machine classifier, in order to perform a result comparison. By applying these approaches on data from a real system, Let’s Go, we obtained preliminary results that we consider positive, given the difficulty of the task and the lack of competing approaches for comparison.

AB - Identifying the level of expertise of its users is important for a system since it can lead to a better interaction through adaptation techniques. Furthermore, this information can be used in offline processes of root cause analysis. However, not much effort has been put into automatically identifying the level of expertise of an user, especially in dialog-based interactions. In this paper we present an approach based on a specific set of task related features. Based on the distribution of the features among the two classes – Novice and Expert – we used Random Forests as a classification approach. Furthermore, we used a Support Vector Machine classifier, in order to perform a result comparison. By applying these approaches on data from a real system, Let’s Go, we obtained preliminary results that we consider positive, given the difficulty of the task and the lack of competing approaches for comparison.

KW - User expertise

KW - Let’s Go

KW - SVM

KW - Random Forest

U2 - 10.1007/978-3-319-49169-1_24

DO - 10.1007/978-3-319-49169-1_24

M3 - Conference article in proceeding

SN - 978-3-319-49168-4

T3 - Lecture Notes in Computer Science

SP - 245

EP - 254

BT - Advances in Speech and Language Technologies for Iberian Languages

A2 - Abad, Alberto

A2 - Ortega, Alfonso

A2 - Teixeira, António

A2 - García Mateo, Carmen

A2 - D. Martínez Hinarejos, Carlos

A2 - Perdigão, Fernando

A2 - Batista, Fernando

A2 - Mamede, Nuno

PB - Springer Nature Switzerland AG

T2 - International Conference on Advances in Speech and Language Technologies for Iberian Languages

Y2 - 23 November 2016 through 25 November 2016

ER -

Ribeiro E, Batista F, Trancoso I, Lopes J, Ribeiro R, de Matos D. Assessing User Expertise in Spoken Dialog System Interactions. In Abad A, Ortega A, Teixeira A, García Mateo C, D. Martínez Hinarejos C, Perdigão F, Batista F, Mamede N, editors, Advances in Speech and Language Technologies for Iberian Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal, November 23-25, 2016, Proceedings. Springer Nature Switzerland AG. 2016. p. 245-254. (Lecture Notes in Computer Science, Vol. 10077). https://doi.org/10.1007/978-3-319-49169-1_24

Assessing User Expertise in Spoken Dialog System Interactions

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Projects

Rage: Realising an Applied Gaming Eco-system

Cite this