Articles

Comparison of Performance between Paper-Based and Computer- Based Tests on the Speech-Language Pathologist National Examination

AUTHOR
Eun Kyoung Lee, Jin-Ae Bae, Seong Hee Choi
INFORMATION
2025, Vol. 10, Issue 2 / pp. 83-96
e-ISSN
2508-5948
p-ISSN

ABSTRACT

Purpose : Despite the expansion of computer-based testing (CBT) for all healthcare professions in Korea, little is known about the applicability of CBT to the National Examination for Speech-Language Pathologists. This study aimed to examine the equivalence and relationship of test performance between CBT and PBT by Korean speech-language pathologists.Methods : Ninety-three speech-language pathologists (41 first-level and 52 second-level) participated in this study, and all subjects performed both PBT and CBT. The test consisted of four periods, and students were randomly divided into Groups A and B. The two groups were crossover-designed to take PBT and CBT. All items were five-item, multiple-choice questions.Results : The total score was not significantly different between PBT and CBT in the first -level certificate test. In contrast, the total score in the second-level certificate test was significantly higher in CBT than in PBT. Item discrimination and difficulty did not differ substantially between the two test modes. CBT total scores were positively correlated with PBT (r = 0.852).Conclusions : In this study, there was no performance difference between the two test modes for the first-level certificate examination, and CBT was superior for the second-level certificate examination. Therefore, this study suggests that CBT can be comparable with PBT. CBT can help students reflect and improve their performance on the national examination and may provide a rationale for switching to CBT mode.

Keyword
PBT CBT Item discrimination Validity Difficulty Speech-language pathologists

REFERENCES

  1. Al-Amri S. Computer-based testing vs. Paper-based testing: establishing the comparability of reading tests through the revolution of reading tests a new comparability model in a Saudi EFL context [dissertation]. Colchester: University of Essex;
  2. AlKadi SZ, Madini AA. EFL learners' lexico-grammatical competence in paper-based vs. computer-based in genre writing. Arab World English Journal. 2019;5:154-75. https://doi.org/10.24093/awej/call5.12 [Crossref]
  3. Balogun AG, Olanrewaju AS. Role of computer self-efficacy and gender in computer-based test anxiety among undergraduates in Nigeria. Psychological Thought. 2016; 9(1):58-66. https://doi.org/10.5964/psyct.v9i1.160 [Crossref]
  4. Boevé AJ, Meijer RR, Albers CJ, Beetsma Y, Bosker RJ. Introducing computer-based testing in high-stakes exams in higher education: results of a field experiment. PLOS ONE. 2015;10(12):e0143616. https://doi.org/10.1371/journal.pone.0143616 [Crossref] [Pubmed]
  5. Boo J, Vispoel. W. Computer versus paper-and-pencil assessment of educational development: a comparison of psychometric features and examinee preferences. Psychol Rep. 2012;111(2):443-60. https://doi.org/10.2466/10.03.11.PR0.111.5.443-460 [Crossref] [Pubmed]
  6. Carter MD. The effects of computer-based simulations on speech-language pathology student performance. J Commun Disord. 2019;77:44-55. https://doi.org/10.1016/j.jcomdis.2018.12.006 [Crossref] [Pubmed]
  7. Chapelle CA. Technology and second language acquisition. Annu Rev Appl Lingui. 2007;27:98-114. https://doi.org/10.1017/S0267190508070050 [Crossref]
  8. Choi IC, Kim KS, Boo J. Comparability of a paper-based language test and a computer-based language test. Language Testing. 2003;20(3):295-320. https://doi.org/10.1191/0265532203lt258oa [Crossref]
  9. Goldberg AL, Pedulla J. Performance differences according to test mode and computer familiarity on a practice graduate record exam. Educ Psychol Meas. 2002;62(6): 1053-67. https://doi.org/10.1177/0013164402238092 [Crossref]
  10. Han C, Kim M, Hwang B, Jung M. Implementation plan and item development and evaluation of computer-based national examination for oriental medicine practitioners. Seoul (South Korea): Korea Healthcare Personnel National Testing Center; 2021.
  11. Han NY, Lee JY, Kwak HS, Lee BK, Lee SH, Yong CS, Kim JH, Oh JM. An expert survey for the selection of practical examination areas for the Pharmacist Board Examination. Korean J Clin Pharm. 2017;27(3):119-26. https://doi.org/10.24304/kjcp.2017.27.3.119 [Crossref]
  12. Hassanien MA, Al-Hayani A, Abu-Kamer R, Almazrooa A. A six step approach for developing computer based assessment in medical education. Med Teach. 2013;35(sup1): S15-9. https://doi.org/10.3109/0142159X.2013.765542 [Crossref] [Pubmed]
  13. Heo S. Computerized testing and question bank database construction in medical education. Journal of Medical Education. 2014;16(1):11-5. https://doi.org/10.17496/kmer.2014.16.1.011 [Crossref]
  14. Heo S, Lee DS, Lim JH, Jeong EY, Heo Y. Measurement review, item bank construction, and administrative and financial support for the introduction of computerized adaptive testing to the Korean National Healthcare Workers' Examination. Seoul (So
  15. Higgins CA, Gray G, Symeonidis P. Tsintsifas A. Automated assessment and experiences of teaching programming. Journal on Educational Resources in Computing. 2005; 5(3):5-es. https://doi.org/10.1145/1163405.1163410 [Crossref]
  16. Hochlehnert A, Brass K, Moeltner A, Juenger J. Does medical students' preference of test format (computerbased vs. paper-based) have an influence on performance? BMC Medical Education. 2011;11(1):1-6. https://doi.org/10.1186/1472-6920-11-89 [Crossref] [Pubmed]
  17. Hols-Elders W, Bloemendaal P, Bos N, Quaak M, Sijstermans R, De Jong P. Twelve tips for computer-based assessment in medical education. Med Teach. 2008;30(7):673-8. https://doi.org/10.1080/01421590802279595 [Crossref] [Pubmed]
  18. Hosseini M, Abidin MJZ, Baghdarnia M. Comparability of test results of computer based tests (CBT) and paper and pencil tests (PPT) among English language learners in Iran. 2014. Procedia - Soc Behav Sci. 2014;98:659-67. https://doi.org/10.1016/j.sbspro.2014.03.465 [Crossref]
  19. Hwang K, Shin S, Kim N, Ryu J, Lee C, Lee S, Hwang I. A study on multimedia item development for computerized test application of the national examination for dental technicians. Seoul (South Korea): Korea Healthcare Personnel National Examinat
  20. International Test Commission. International guidelines on computer-based and internet-delivered testing. Int J Test. 2006;6(2):143-71. https://doi.org/10.1207/s15327574ijt0602_4 [Crossref]
  21. Jamieson J. Trends in computer-based second language assessment. Annu Rev Appl Linguist. 2005;25:228-42. https://doi.org/10.1017/S0267190505000127 [Crossref]
  22. Jeong H. A comparative study of scores on computer-based tests and paper-based tests. Behaviour & Information Technology. 2014;33(4):410-22. https://doi.org/10.1080/0144929X.2012.710647 [Crossref]
  23. Karay Y, Schauber SK, Stosch C, Schüttpelz-Brauns K. Computer versus paper-does it make any difference in test performance? Teach Learn Med. 2015;27(1):57-62. https://doi.org/10.1080/10401334.2014.979175 [Crossref] [Pubmed]
  24. Khoshsima H, Toroujeni SM, Thompson N, Ebrahimi MR. Computer-based (CBT) vs. paper-based (PBT) testing: Mode effect, relationship between computer familiarity, attitudes, aversion and mode preference with CBT test scores in an Asian private EFL
  25. Kim HR, Bowles M, Yan X, Chung SJ. Examining the comparability between paper- and computer-based versions of an integrated writing placement test. Assessing Writing. 2018;36:49-62. https://doi.org/10.1016/j.asw.2018.03.006 [Crossref]
  26. Kim YT, Kim JO, Jeon HS, Choi HJ, Kim MJ, Tae W, Kang MK. A job analysis study of speech-language pathologists through a Delphi survey. Speech Therapy Research. 2014; 23(3):147-61. https://doi.org/10.15724/jslhd.2014.23.3.012 [Crossref]
  27. Kolen MJ. Threats to score comparability with applications to performance assessments and computerized adaptive tests. Educational Assessment, 1999;6(2):73-96. https://doi.org/10.1207/S15326977EA0602_01 [Crossref]
  28. Korea Manpower Institute. Industrial Engineer CBT test held for the first time, test takers highly satisfied. Press release. Daejeon (South Korea): Korea Manpower Institute; 2020.
  29. Lee JY, Kim KI, Kim KD, Kim MK, Kim EY, Kim JK, Kim HA, Moon Y, Ji Y, Jeong YS, Lee JY, Jung EK, Chun KH, Choi KH. A study on the development of various item types that can be implemented in computerized examinations. Seoul (South Korea): Korea
  30. Lim MK, Bo H, Jung K, Jun K. A study on the question management system of foreign healthcare licensure examinations. Seoul (South Korea): Korea Healthcare Personnel National Examination Service; 2015.
  31. Lim MK, Kim YH, Song KR, Kim HM, Lee HW, Lee SY. Basic research on computerized test administration. Seoul (South Korea): Korea Healthcare Personnel National Testing Center; 2006.
  32. Lim MK, Min H, Heo S. A study on the development of computer-based national healthcare personnel examination. Seoul (South Korea): Korea Healthcare Personnel National Examination Center; 2014.
  33. Mangen A, Walgermo BR, Brønnick K. Reading linear texts on paper versus computer screen: effects on reading comprehension. Int J Educ Res. 2013;58:61-8. https://doi.org/10.1016/j.ijer.2012.12.002 [Crossref]
  34. Mizrachi D. Undergraduates' academic reading format preferences and behaviors. J Acad Libr. 2015;41(3):301-11. https://doi.org/10.1016/j.acalib.2015.03.009 [Crossref]
  35. Mohammadi M, Barzgaran M. Comparability of computer-based and paper-based versions of writing section of PET in Iranian EFL context. J Foreign Lang Teach Transl Stud. 2012;1(2):1-20.
  36. Odo DM. Computer familiarity and test performance on a computer-based cloze ESL reading assessment. Teaching English with Technology. 2012;12(3):18-35.
  37. Öz H, Özturan T. Computer-based and paper-based testing: Does the test administration mode influence the reliability and validity of achievement tests? J Lang Linguist Stud. 2018;14(1):67-85.
  38. Park HK, Kim DH, Kang YJ, Bae SY, Moon YJ. A study of test-taker fatigue following computer-based testing (CBT). Seoul (South Korea): Korea Healthcare Personnel National Testing Center; 2021.
  39. Prisacari AA, Danielson J. Computer-based versus paper-based testing: investigating testing mode with cognitive load and scratch paper use. Comput Human Behav. Comparison of Performance between Paper-Based and Computer-Based Tests on the Speech https://doi.org/10.1016/j.chb.2017.07.044 [Crossref]
  40. Rudland JR, Schwartz P, Ali A. Moving a formative test from a paper-based to a computer-based format. A student viewpoint. Med Teach. 2011;33(9):738-43. https://doi.org/10.3109/0142159X.2011.577119 [Crossref] [Pubmed]
  41. Sung T. Contemporary Educational Assessment. 3rd ed. Seoul: Hakji; 2010.
  42. Washburn S, Herman J, Stewart R. Evaluation of performance and perceptions of electronic vs. multiplechoice paper exams. Adv Physiol Educ. 2017;41(4):548-55. https://doi.org/10.1152/advan.00138.2016 [Crossref] [Pubmed]
  43. Yoo SG, Won JS, Lee GJ, Choi HK, Han DK, Lee HS, Sung HJ, Jung IH, Eun J. A feasibility study on MAT item development by healthcare occupation and SBT national test system. Seoul (South Korea): Korea Healthcare Personnel National Testing Center;