Research Article
BibTex RIS Cite

A SENTIMENT ANALYSIS-BASED SMARTPHONE APPLICATION TO CONTINUOUSLY ASSESS STUDENTS’ FEEDBACK AND MONITOR THE QUALITY OF COURSES AND THE LEARNING EXPERIENCE IN EDUCATIONAL INSTITUTIONS

Year 2023, , 15 - 26, 04.05.2023
https://doi.org/10.18768/ijaedu.1256188

Abstract

The quality of education in a specific educational institution is directly reflected in the outcomes of their system. Higher-quality educational systems continue to deliver better learning experiences to enrolled students and better-developed skills and knowledge. To provide high-quality education, an institution must continually monitor its plans, update its courses’ topics and curriculum, and improve teaching facilities and different learning experiences. Students’ opinions and feedback regarding different aspects of a course and their personal learning experience, if properly gathered and analyzed, can be strong indicators of the quality of that course and help identify the areas of satisfaction and dissatisfaction with that course. Highlighting the strengths and weaknesses of each course helps faculty members put, execute, and evaluate a course quality improvement plan in the following semester. Such valuable students’ feedback and opinions about courses are scattered throughout different social media platforms and managed by different discussion groups, usually students. Thus, gathering honest and freely written comments and opinions in one place is challenging. Furthermore, extracting and analyzing courses’ quality and learning experience-related posts is not a trivial task. This study describes the process of designing and developing a smartphone application utilizing Sentiment Analysis techniques to address the problem of gathering, analyzing, and understanding students’ feedback and comments regarding different aspects of courses quality provided by an educational institution. The project’s primary goal is to benefit from student feedback regarding the institution’s courses to continuously assess and monitor the quality of the courses and the students’ learning experiences. A sample representative dataset of students’ unstructured free-text comments and answers to open-ended questions about five different courses over four consecutive semesters was collected, cleaned, and used to develop and test two sentiment analysis models: Naive Bayes in WEKA and a sentiment lexicon-based model named VADER. To further analyze and assess different aspects of the learning experience and courses along with its overall quality, answers to closed-end questions were also analyzed using the 5-point Likert scale. Preliminary results obtained from evaluating the sentiment analysis models show that the Naïve Bayes model achieved 68.7%, 68.8%, 68.8%, and 68.8%, while the VADER model achieved 72.12%, 72.82%, 72.12%, 71.87%, in terms of accuracy, precision, recall, and F1-score, respectively. Performance testing results of the application show that the maximum usage for the CPU is 44%, for the memory is 119 MB, for sending a request on the network 14.7 KB/s, for receiving a response is 226.5 KB/s, and the maximum energy usage is medium. For stress testing, obtained results show that the application can successfully deal with a maximum of 500 random, fast, and abnormal events. For user acceptance testing, users were surveyed to measure their level of satisfaction with the application using the system usability scale. The results show that 100% of users either agreed or strongly agreed that they would like to use the application and be more engaged in assessing the quality of courses. They also indicated that the application is easy to use, quick, and easy to learn. This paper also highlights various challenges and limitations developers face, along with important recommendations for further improvements and future work directions.

References

  • Abedin, N. F. Z., Taib, J. M., & Jamil, H. M. T. (2014). Comparative study on course evaluation process: Students’ and lecturers’ perceptions. Procedia - Social and Behavioral Sciences, 123, 380-388. doi:10.1016/j.sbspro.2014.01.1436
  • Android Profiler - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio/profile/android-profiler
  • Android Studio & App Tools - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio
  • Apache NetBeans [Computer Software]. (1996). Retrieved from https://netbeans.apache.org/
  • Beaumont, C., O'Doherty, M., & Shannon, L. (2011). Reconceptualising assessment feedback: A key to improving student learning? Studies in Higher Education (Dorchester-on-Thames), 36(6), 671-687. doi:10.1080/03075071003731135
  • Carron, G., & Chau, T. N. (1996). The quality of primary schools in different development contexts. Paris: Unesco.
  • Darling-Hammond, L. (1998). Doing what matters most. New York: National Commission on Teaching & America's Future.
  • UNICEF. (2000). Defining quality in education. FeedbackPanda: Student Feedback Templates for Online Teachers [Computer Software]. (2019). Retrieved from https://www.feedbackpanda.com/
  • Frank, E., Hall, M. A., Holmes, G., Kirkby, R., Pfahringer, B. & Witten, I. H. (2005). Weka: A machine learning workbench for data mining.. In O. Maimon & L. Rokach (eds.), Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers (pp. 1305--1314). Springer.
  • Fuller, B., Dellagnelo, L., et al. (1999). How to raise children's early literacy? Comparative Education Review, 43(1), 1-35. doi:10.1086/447543
  • Gottipati, S., Shankararaman, V., & Lin, J. R. (2018). Text analytics approach to extract course improvement suggestions from students’ feedback. Research and Practice in Technology Enhanced Learning, 13(1), 6-19. doi:10.1186/s41039-018-0073-0
  • Hutto, C., & Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the ... International AAAI Conference on Weblogs and Social Media, 8(1), 216-225. doi:10.1609/icwsm.v8i1.14550
  • John E. Mullens, John B. Willett, & Richard J. Murnane. (1996). The contribution of training and subject matter knowledge to teaching effectiveness: A multilevel analysis of longitudinal evidence from belize. Comparative Education Review, 40(2), 139-157. doi:10.1086/447369
  • Mwaura, P. A. M., & Marfo, K. (2011). Bridging culture, research, and practice in early childhood development: The madrasa resource centers in east africa. Child Development Perspectives, 5(2), 134-139. doi:10.1111/j.1750-8606.2011.00168.x
  • MySQL [Computer Software]. (1995). Retrieved from https://www.mysql.com/
  • Pennycuick, D. (1993). School effectiveness in developing countries - A summary of the research evidence PhpMyAdmin [Computer Software]. (1998). Retrieved from https://www.phpmyadmin.net/
  • PiHappiness [Computer Software]. (2018). Retrieved from https://www.pihappiness.com/
  • Simon, E. (2013). Teacher development in higher education (1. publ. ed.). New York [u.a.]: Routledge.
  • Taboada, M. (2016). Sentiment analysis: An overview from linguistics. Annual Review of Linguistics, 2(1), 325-347. doi:10.1146/annurev-linguistics-011415-040518
  • UI/Application Exerciser Monkey - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio/test/other-testing-tools/monkey
  • Vivi [Computer Software]. (2015) Retrieved from https://www.vivi.io/empower-your-teachers-to-deliver-personalized-learning/
  • Walczowski, L. T., Dimond, K. R., & Waller, W. A. J. (2000). A digital engineering curriculum for the new millennium. International Journal of Electrical Engineering & Education, 37(1), 108-117. doi:10.7227/IJEEE.37.1.10
  • Webb, G. I., Keogh, E., & Miikkulainen, R. (2010). Naïve bayes. Encyclopedia of Machine Learning, 15, 713-714. Willms, J. (2002). Standards of care: Investments to improve children's educational outcomes in latin America
  • Ziplet | The Easy Way To Check In With Your Students [Computer Software]. (2015). Retrieved from https://ziplet.com/
  • Zonka Feedback [Computer Software]. (2014). Retrieved from https://www.zonkafeedback.com/
Year 2023, , 15 - 26, 04.05.2023
https://doi.org/10.18768/ijaedu.1256188

Abstract

References

  • Abedin, N. F. Z., Taib, J. M., & Jamil, H. M. T. (2014). Comparative study on course evaluation process: Students’ and lecturers’ perceptions. Procedia - Social and Behavioral Sciences, 123, 380-388. doi:10.1016/j.sbspro.2014.01.1436
  • Android Profiler - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio/profile/android-profiler
  • Android Studio & App Tools - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio
  • Apache NetBeans [Computer Software]. (1996). Retrieved from https://netbeans.apache.org/
  • Beaumont, C., O'Doherty, M., & Shannon, L. (2011). Reconceptualising assessment feedback: A key to improving student learning? Studies in Higher Education (Dorchester-on-Thames), 36(6), 671-687. doi:10.1080/03075071003731135
  • Carron, G., & Chau, T. N. (1996). The quality of primary schools in different development contexts. Paris: Unesco.
  • Darling-Hammond, L. (1998). Doing what matters most. New York: National Commission on Teaching & America's Future.
  • UNICEF. (2000). Defining quality in education. FeedbackPanda: Student Feedback Templates for Online Teachers [Computer Software]. (2019). Retrieved from https://www.feedbackpanda.com/
  • Frank, E., Hall, M. A., Holmes, G., Kirkby, R., Pfahringer, B. & Witten, I. H. (2005). Weka: A machine learning workbench for data mining.. In O. Maimon & L. Rokach (eds.), Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers (pp. 1305--1314). Springer.
  • Fuller, B., Dellagnelo, L., et al. (1999). How to raise children's early literacy? Comparative Education Review, 43(1), 1-35. doi:10.1086/447543
  • Gottipati, S., Shankararaman, V., & Lin, J. R. (2018). Text analytics approach to extract course improvement suggestions from students’ feedback. Research and Practice in Technology Enhanced Learning, 13(1), 6-19. doi:10.1186/s41039-018-0073-0
  • Hutto, C., & Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the ... International AAAI Conference on Weblogs and Social Media, 8(1), 216-225. doi:10.1609/icwsm.v8i1.14550
  • John E. Mullens, John B. Willett, & Richard J. Murnane. (1996). The contribution of training and subject matter knowledge to teaching effectiveness: A multilevel analysis of longitudinal evidence from belize. Comparative Education Review, 40(2), 139-157. doi:10.1086/447369
  • Mwaura, P. A. M., & Marfo, K. (2011). Bridging culture, research, and practice in early childhood development: The madrasa resource centers in east africa. Child Development Perspectives, 5(2), 134-139. doi:10.1111/j.1750-8606.2011.00168.x
  • MySQL [Computer Software]. (1995). Retrieved from https://www.mysql.com/
  • Pennycuick, D. (1993). School effectiveness in developing countries - A summary of the research evidence PhpMyAdmin [Computer Software]. (1998). Retrieved from https://www.phpmyadmin.net/
  • PiHappiness [Computer Software]. (2018). Retrieved from https://www.pihappiness.com/
  • Simon, E. (2013). Teacher development in higher education (1. publ. ed.). New York [u.a.]: Routledge.
  • Taboada, M. (2016). Sentiment analysis: An overview from linguistics. Annual Review of Linguistics, 2(1), 325-347. doi:10.1146/annurev-linguistics-011415-040518
  • UI/Application Exerciser Monkey - Android Developers [Computer Software]. (2013). Retrieved from https://developer.android.com/studio/test/other-testing-tools/monkey
  • Vivi [Computer Software]. (2015) Retrieved from https://www.vivi.io/empower-your-teachers-to-deliver-personalized-learning/
  • Walczowski, L. T., Dimond, K. R., & Waller, W. A. J. (2000). A digital engineering curriculum for the new millennium. International Journal of Electrical Engineering & Education, 37(1), 108-117. doi:10.7227/IJEEE.37.1.10
  • Webb, G. I., Keogh, E., & Miikkulainen, R. (2010). Naïve bayes. Encyclopedia of Machine Learning, 15, 713-714. Willms, J. (2002). Standards of care: Investments to improve children's educational outcomes in latin America
  • Ziplet | The Easy Way To Check In With Your Students [Computer Software]. (2015). Retrieved from https://ziplet.com/
  • Zonka Feedback [Computer Software]. (2014). Retrieved from https://www.zonkafeedback.com/
There are 25 citations in total.

Details

Primary Language English
Subjects Other Fields of Education
Journal Section Articles
Authors

Sarah A. Alkhodair 0000-0001-8428-3092

Publication Date May 4, 2023
Submission Date February 24, 2023
Published in Issue Year 2023

Cite

EndNote Alkhodair SA (May 1, 2023) A SENTIMENT ANALYSIS-BASED SMARTPHONE APPLICATION TO CONTINUOUSLY ASSESS STUDENTS’ FEEDBACK AND MONITOR THE QUALITY OF COURSES AND THE LEARNING EXPERIENCE IN EDUCATIONAL INSTITUTIONS. IJAEDU- International E-Journal of Advances in Education 9 25 15–26.

 Published and Sponsored by OCERINT International © 2015 - 2023

Contact: ijaedujournal@hotmail.com

Creative Commons License

International E-Journal of Advances in Education by IJAEDU is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Permissions beyond the scope of this license may be available at http://ijaedu.ocerintjournals.org