تعداد نشریات | 25 |
تعداد شمارهها | 932 |
تعداد مقالات | 7,652 |
تعداد مشاهده مقاله | 12,494,425 |
تعداد دریافت فایل اصل مقاله | 8,885,789 |
On the Construct Validity of the Iranian Ministry of Health Language Exam (MHLE) | ||
Journal of Language Horizons | ||
دوره 4، شماره 2 - شماره پیاپی 8، مهر 2020، صفحه 9-36 اصل مقاله (519.85 K) | ||
نوع مقاله: Research article | ||
شناسه دیجیتال (DOI): 10.22051/lghor.2020.28036.1180 | ||
نویسندگان | ||
S. Susan Marandi1؛ Leila Tajik* 2؛ Leila Zohali3 | ||
1Department of English Language and Literature, Faculty of Literature, Alzahra University, Tehran, Iran | ||
2Department of English Language and Literature, Faculty of Literature, Alzahra University, Tehran, Iran | ||
3M.A. graduate, Department of English Language and Literature, Faculty of Literature, Alzahra University, Tehran, Iran | ||
چکیده | ||
Considering validity as a unitary concept, this study investigated the construct validity of the Iranian Ministry of Health Language Exam (MHLE). To meet this objective, we first conducted item analysis and reliability analysis, and verified KR20-if-item-deleted indices on the scores of 987 MHLE test takers before running factor analysis. Though the test was found to enjoy a high level of reliability, it suffered from 28 problematic items flagged through item analysis and KR20-if-item-deleted indices. Next, we ran factor analysis on the data, screened through item analysis, by implementing Horn’s parallel analysis and Velicer’s minimum average partial (MAP) tests. Parallel analysis resulted in overfactoring. The MAP test, however, produced results with two to seven factors. Though the 4-factor result of the MAP test seemed to be more logical at first glance, the overall results were rather disappointing. Nineteen items did not load significantly on any factor and a clear pattern of item loading was not found for many items. These findings can be viewed as evidence detracting from the validity of MHLE. | ||
کلیدواژهها | ||
language assessment؛ language testing؛ proficiency test؛ item analysis؛ reliability analysis؛ factor analysis؛ MHLE | ||
عنوان مقاله [English] | ||
بررسی اعتبار سازهای آزمون زبان انگلیسی وزارت بهداشت | ||
نویسندگان [English] | ||
sسیده سوسن مرندی1؛ لیلا تاجیک2؛ لیلا زحلی3 | ||
1دانشیار، گروه زبان و ادبیات انگلیسی، دانشکدۀ ادبیات، دانشگاه الزهرا (س)، تهران، ایران | ||
2استادیار، گروه زبان و ادبیات انگلیسی، دانشکدۀ ادبیات، دانشگاه الزهرا (س)، تهران، ایران | ||
3دانش آموختۀ کارشناسی ارشد، گروه زبان و ادبیات انگلیسی، دانشکدۀ ادبیات، دانشگاه الزهرا (س)، تهران، ایران. | ||
چکیده [English] | ||
با در نظرگرفتن اعتبار بهعنوان یک مفهوم واحد، این تحقیق اعتبار سازهای آزمون زبان انگلیسی وزارت بهداشت را مورد بررسی قرار داد. به این منظور، ابتدا تحلیل پرسش، تحلیل پایایی، همبستگی پرسش به کل آزمون پیش از انجام تحلیل عامل بر روی نمرات 987 شرکتکننده در آزمون انجامگرفت. با اینکه دریافتیم آزمون از پایایی بالایی برخوردار است، اما تحلیل پرسش و همبستگی پرسش به کل آزمون نشان داد که تعداد 28 سوال دارای ایراد هستند. در مرحلۀ بعد، با اجرای دو آزمون تحلیل موازی هُرن و روش جزئی حداقل میانگین وِلیسر برروی دادههایی که ازطریق تحلیل پرسش مورد غربالگری قرار گرفته بودند، تحلیل عاملی انجام شد. تحلیل موازی، تعداد مولفهها را بیش از حد معمول نشان داد. اما آزمون جزئی حداقل میانگین نتایجی با 2 تا 7 مولفه را نشان داد. اگرچه نتیجۀ چهار مولفهایِ آزمون جزئی حداقل میانگین در نگاه اول منطقیتر بهنظر میرسید، اما نتایج کلی چندان راضیکننده نبود. نوزده پرسش بر هیچ مولفهای بارگذاری نکردند و الگوی واضحی از بارگذاری برروی مولفهها برای بسیاری از پرسشها یافت نشد. این نتایج را میتوان شاهدی بر کاستی اعتبار سازهای آزمون زبان انگلیسی وزارت بهداشت دانست. | ||
کلیدواژهها [English] | ||
ارزیابی زبان, آزمون مهارت زبان انگلیسی, اعتبار سازهای, تحلیل عاملی, آزمون زبان انگلیسی وزارت بهداشت | ||
مراجع | ||
Alavi, M. (1997). An investigation of the construct validity of reading comprehension test in an academic context: Using rhetorical structure theory. Paper presented in BALEAP Conference, University of Wales, Swansea, UK.
Alavi, S. M., & Ghaemi, H. (2011). Application of structural equation modeling in EFL testing: A report of two Iranian studies. Language Testing in Asia, 1(3), 22-35. https://doi.org/10.1186/2229-0443-1-3-22.
Alavi, S. M., Kaivanpanah, S., & Masjedlou, A.P. (2018) Validity of the listening module of international English language testing system: Multiple sources of evidence. Language Testing Asia, 8. https://doi.org/10.1186/s40468-018-0057-4
Alderson, C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Brooks/Cole.
Anastasi, A. (1986). Evolving concepts of test validation. Annual Reviews of Psychology, 37(1), 1-15.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
Beglar, D., & Hunt, A. (1999). Revising and validating the 2000 word level and university word level vocabulary test. Language Testing, 16(2), 131-162.
Brindley, G. (1998). Assessing listening abilities. Annual Review of Applied Linguistics, 18, 171-191.
Brooks, G. P., & Johanson, G. A. (2003). TAP: Test analysis program. Applied Psychological Measurement, 27(4),303-304.
Brown, J. D. (2005). Testing in language programs. McGraw Hill.
Brown, T. (2010). Construct validity: A unitary concept for occupational therapy assessment and measurement. HKJOT, 20(1), 30-42.
Bryman, A., & Cramer, D. (1990). Quantitative data analysis for social scientists. Routledge.
Buck, G. (2001). Assessing listening. Cambridge University Press.
Colliver, J. A., Conlee, M. J., & Verhulst, S. J. (2012). From test validity to construct validity… and back? Medical Education, 46(4), 366-371.
Cox, T. L., & Malone, M. E. (2018). A validity argument to support the ACTFL Assessment of Performance Toward Proficiency in Languages (AAPPL). Foreign Language Annals, 51(3), 548-574.
Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart, and Winston.
Downing, S. M., & Haladyna, T. M. (Eds.) (2006). Handbook of test development. Lawrence Erlbaum Associates Publishers.
Falvey, P., Holbrook, J., & Coniam, D. (1994). Assessing students. Longman.
Field, A (2009). Discovering statistics using SPSS. Sage Publications.
Freedle, R., & Kostin, I. (1999). Does the text matter in a multiple-choice test of comprehension? The case for the construct validity of TOEFL's minitalks. Language Testing, 16(1), 2-32.
Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. Routledge.
Hale, G. A., Rock, D. A., & Jirele, T. (1989). Confirmatory factor analysis of the Test of English as a Foreign Language (TOEFL Research Report No. 32). Educational Testing Service.
Hale, G. A., Stansfield, C. W., Rock, D. A., Hicks, M. M., Butler, F. A., & Oller, Jr., J. W. (1988). The relation of multiple-choice cloze items to the Test of English as a Foreign Language. Language Testing, 6(1), 47-76.
Hatch, E., & Farhady, H. (1982). Research design and statistics for applied linguistics. Newbury House.
Hinton, P. R., Brownlow, C., McMurray, I., & Cozens, B. (2004). SPSS explained. Taylor & Francis.
Hughes, A. (2003). Testing for language teachers. Cambridge University Press.
In'nami, Y, & Koizumi, R. (2011). Structural equation modelling in language testing and learning research: A review. Language Assessment Quarterly, 8(3), 250-273. https://doi.org/10.1080/15434303.2011.582203.
Jackson, T. R., Draugalis, J. R., Slack, M. K., Zachry, W. M., & D'Agpstino, J. (2002). Validation of authentic performance assessment: A process suited for Rasch modeling. American Journal of Pharmaceutical Education, 66(3), 233-243.
Kerlinger, N. F. (1979). Behavioral research: A conceptual approach. Holt, Rinehart and Winston.
Khine, M. S. (2013). Application of structural equation modeling in educational research and practice. Sense Publishers.
Kim, Y. M., & Kim, M. (2017). Validations of an English placement test for a general English language program at the tertiary level. JLTA (Japan Language Testing Association) Journal, 20, 17-34.
Kyle, K., Crossley, S. A., & McNamara, D. S. (2016). Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 33(3), 319-340.
Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(10), 12-27.
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5-11.
Messick, S. (1990). Validity of test interpretation and use (Research Report No. 90.11). Educational Testing Service.
Messick, S. (2005). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5-8.
Meyers, L. S., Gamst, G., & Guarino, A. J. (2006). Applied multivariate research: Design and interpretation. Sage Publications.
Moses, M. S., & Nanna, M. J. (2007). The testing culture and the persistence of high stakes testing reforms. Education and Culture, 23(1), 55-72.
Ockey, G, & Choi, I. (2015). Structural equation modeling reporting practices for language assessment. Language Assessment Quarterly, 12(3), 305-319. https://doi.org/10.1080/15434303.2015.1050101.
Reyment, R., & Joreskog, K. G. (1993). Applied factor analysis in the natural sciences. Cambridge University Press.
Roever, C. (2001). Web-based language testing. Language Learning and Technology, 5(2), 84-94.
Saito, K. (2019). To what extent does long-term foreign language education improve spoken second language lexical proficiency? TESOL Quarterly,53(1), 82-101.
Salehi, M. (2011). On the construct validity of the reading section of the University of Tehran English Proficiency Test. Journal of English Language Teaching and Learning, 222, 129-159.
Sawaki, Y (2012). Factor analysis: The encyclopedia of applied linguistics. Blackwell Publishing Ltd. https://doi.org/10.1002/9781405198431.wbeal0407.
Sawaki, Y., Stricker, L. J., & Oranje, A. H. (2009). Factor structure of the TOEFL Internet-based test. Language Testing, 26(1), 5-30. Schmitt, T. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psycho-educational Assessment, 29(4), 304-321. https://doi.org/10.1177/0734282911406653. Shaw, S. D., & Weir, C. J. (2007). Examining writing: Research and practice in assessing second language writing. Cambridge University Press.
Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405-450.
Spolsky, B. (1995). Measured words: The development of objective language testing. Oxford University Press.
Stapleton. C. D. (1997, January). Basic concepts in exploratory factor analysis (EFA) as a tool to evaluate score validity: A right-brained approach [Paper presentation]. The annual meeting of the Southwest Educational Research Association, Austin. Stoker, H. W., & Impara, J. C. (1995). Basic psychometric issues in licensure testing. In J. C. Impara (Ed.), Licensure testing: Purposes, procedures, and practices (pp. 167-186). Nebraska Series on Measurement and Testing. Stricker, L. J., Rock, D. A., & Lee, Y.-W. (2005). Factor structure of the language test across language groups (TOEFL Monograph Series MS-32). Educational Testing Service. Tabachnick, B. G., & Fidell, L. S. (2008). Using multivariate statistics (5th Ed.). Pearson.
Van der Walt, J. L., & Steyn, F. (2008). The validation of language tests. Stellenbosch Papers in Linguistics, 38, 191-204.
Weir, C. J. (2005). Language testing and validation. Palgrave McMillan. | ||
آمار تعداد مشاهده مقاله: 762 تعداد دریافت فایل اصل مقاله: 660 |