A machine learning-based usability evaluation method for eLearning systems

Asil Oztekin, Dursun Delen, Ali Turkyilmaz, Selim Zaim

Research output: Contribution to journalArticle

38 Citations (Scopus)

Abstract

The research presented in this paper proposes a new machine learning-based evaluation method for assessing the usability of eLearning systems. Three machine learning methods (support vector machines, neural networks and decision trees) along with multiple linear regression are used to develop prediction models in order to discover the underlying relationship between the overall eLearning system usability and its predictor factors. A subsequent sensitivity analysis is conducted to determine the rank-order importance of the predictors. Using both sensitivity values along with the usability scores, a metric (called severity index) is devised. By applying a Pareto-like analysis, the severity index values are ranked and the most important usability characteristics are identified. The case study results show that the proposed methodology enhances the determination of eLearning system problems by identifying the most pertinent usability factors. The proposed method could provide an invaluable guidance to the usability experts as to what measures should be improved in order to maximize the system usability for a targeted group of end-users of an eLearning system.

Original languageEnglish
Pages (from-to)63-73
Number of pages11
JournalDecision Support Systems
Volume56
Issue number1
DOIs
StatePublished - 1 Dec 2013
Externally publishedYes

Fingerprint

Learning systems
Decision trees
Linear regression
Sensitivity analysis
Support vector machines
Decision Trees
Neural networks
Linear Models
Research
Machine Learning
Evaluation method
Machine learning
Usability
Electronic learning
Electronic Learning
Evaluation
Support Vector Machine

Keywords

  • eLearning (web-based learning/distance learning)
  • Information fusion
  • Machine learning
  • Sensitivity analysis
  • Severity index
  • Usability engineering

Cite this

Oztekin, Asil ; Delen, Dursun ; Turkyilmaz, Ali ; Zaim, Selim. / A machine learning-based usability evaluation method for eLearning systems. In: Decision Support Systems. 2013 ; Vol. 56, No. 1. pp. 63-73.
@article{2901559839c841099ca10375168bf6c5,
title = "A machine learning-based usability evaluation method for eLearning systems",
abstract = "The research presented in this paper proposes a new machine learning-based evaluation method for assessing the usability of eLearning systems. Three machine learning methods (support vector machines, neural networks and decision trees) along with multiple linear regression are used to develop prediction models in order to discover the underlying relationship between the overall eLearning system usability and its predictor factors. A subsequent sensitivity analysis is conducted to determine the rank-order importance of the predictors. Using both sensitivity values along with the usability scores, a metric (called severity index) is devised. By applying a Pareto-like analysis, the severity index values are ranked and the most important usability characteristics are identified. The case study results show that the proposed methodology enhances the determination of eLearning system problems by identifying the most pertinent usability factors. The proposed method could provide an invaluable guidance to the usability experts as to what measures should be improved in order to maximize the system usability for a targeted group of end-users of an eLearning system.",
keywords = "eLearning (web-based learning/distance learning), Information fusion, Machine learning, Sensitivity analysis, Severity index, Usability engineering",
author = "Asil Oztekin and Dursun Delen and Ali Turkyilmaz and Selim Zaim",
year = "2013",
month = "12",
day = "1",
doi = "10.1016/j.dss.2013.05.003",
language = "English",
volume = "56",
pages = "63--73",
journal = "Decision Support Systems",
issn = "0167-9236",
publisher = "Elsevier",
number = "1",

}

A machine learning-based usability evaluation method for eLearning systems. / Oztekin, Asil; Delen, Dursun; Turkyilmaz, Ali; Zaim, Selim.

In: Decision Support Systems, Vol. 56, No. 1, 01.12.2013, p. 63-73.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A machine learning-based usability evaluation method for eLearning systems

AU - Oztekin, Asil

AU - Delen, Dursun

AU - Turkyilmaz, Ali

AU - Zaim, Selim

PY - 2013/12/1

Y1 - 2013/12/1

N2 - The research presented in this paper proposes a new machine learning-based evaluation method for assessing the usability of eLearning systems. Three machine learning methods (support vector machines, neural networks and decision trees) along with multiple linear regression are used to develop prediction models in order to discover the underlying relationship between the overall eLearning system usability and its predictor factors. A subsequent sensitivity analysis is conducted to determine the rank-order importance of the predictors. Using both sensitivity values along with the usability scores, a metric (called severity index) is devised. By applying a Pareto-like analysis, the severity index values are ranked and the most important usability characteristics are identified. The case study results show that the proposed methodology enhances the determination of eLearning system problems by identifying the most pertinent usability factors. The proposed method could provide an invaluable guidance to the usability experts as to what measures should be improved in order to maximize the system usability for a targeted group of end-users of an eLearning system.

AB - The research presented in this paper proposes a new machine learning-based evaluation method for assessing the usability of eLearning systems. Three machine learning methods (support vector machines, neural networks and decision trees) along with multiple linear regression are used to develop prediction models in order to discover the underlying relationship between the overall eLearning system usability and its predictor factors. A subsequent sensitivity analysis is conducted to determine the rank-order importance of the predictors. Using both sensitivity values along with the usability scores, a metric (called severity index) is devised. By applying a Pareto-like analysis, the severity index values are ranked and the most important usability characteristics are identified. The case study results show that the proposed methodology enhances the determination of eLearning system problems by identifying the most pertinent usability factors. The proposed method could provide an invaluable guidance to the usability experts as to what measures should be improved in order to maximize the system usability for a targeted group of end-users of an eLearning system.

KW - eLearning (web-based learning/distance learning)

KW - Information fusion

KW - Machine learning

KW - Sensitivity analysis

KW - Severity index

KW - Usability engineering

UR - http://www.scopus.com/inward/record.url?scp=84889088422&partnerID=8YFLogxK

U2 - 10.1016/j.dss.2013.05.003

DO - 10.1016/j.dss.2013.05.003

M3 - Article

AN - SCOPUS:84889088422

VL - 56

SP - 63

EP - 73

JO - Decision Support Systems

JF - Decision Support Systems

SN - 0167-9236

IS - 1

ER -