TY - GEN
T1 - Validating the reflective writing framework (RWF) for assessing reflective writing in computer science education through manual annotation
AU - Alrashidi, Huda
AU - Joy, Mike
AU - Ullmann, Thomas Daniel
AU - Almujally, Nouf
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - The accuracy of a framework for annotating reflective writing can be increased through the evaluation and revision of the annotation scheme to ensure the reliability and validity of the framework. To our knowledge, there is a lack of literature related to the accuracy of any reflective writing framework in Computer Science (CS) education. This paper describes a manual annotation scheme, applied during four pilot studies, to validate the authors’ novel Reflective Writing Framework (RWF) for CS education. The results show, through the pilot studies, that the accuracy of Inter-Rater Reliability (IRR) increases from 0.5 to 0.8, which was substantial and close to an almost perfect agreement. This paper contributes to CS education through the reliability and validity of the RWF that can be potentially used for generating an Intelligent Tutoring Systems (ITS) using machine learning algorithms.
AB - The accuracy of a framework for annotating reflective writing can be increased through the evaluation and revision of the annotation scheme to ensure the reliability and validity of the framework. To our knowledge, there is a lack of literature related to the accuracy of any reflective writing framework in Computer Science (CS) education. This paper describes a manual annotation scheme, applied during four pilot studies, to validate the authors’ novel Reflective Writing Framework (RWF) for CS education. The results show, through the pilot studies, that the accuracy of Inter-Rater Reliability (IRR) increases from 0.5 to 0.8, which was substantial and close to an almost perfect agreement. This paper contributes to CS education through the reliability and validity of the RWF that can be potentially used for generating an Intelligent Tutoring Systems (ITS) using machine learning algorithms.
KW - Assessment
KW - Computer science education
KW - Framework
KW - Intelligent Tutoring Systems
KW - Manual annotation
KW - Reflection
KW - Reflective writing
UR - http://www.scopus.com/inward/record.url?scp=85086232930&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-49663-0_38
DO - 10.1007/978-3-030-49663-0_38
M3 - Conference contribution
AN - SCOPUS:85086232930
SN - 9783030496623
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 323
EP - 326
BT - Intelligent Tutoring Systems - 16th International Conference, ITS 2020, Proceedings
A2 - Kumar, Vivekanandan
A2 - Troussas, Christos
PB - Springer
T2 - 16th International Conference on Intelligent Tutoring Systems, ITS 2020
Y2 - 8 June 2020 through 12 June 2020
ER -