Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale

T. Oguz Basokcu; Tuncay Ogretmen

doi:doi:10.11648/j.ajtas.20140302.11

| Peer-Reviewed

Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale

T. Oguz Basokcu, Tuncay Ogretmen

Published in American Journal of Theoretical and Applied Statistics (Volume 3, Issue 2)

Received: 31 December 2013 Published: 20 March 2014

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.

Published in	American Journal of Theoretical and Applied Statistics (Volume 3, Issue 2)
DOI	10.11648/j.ajtas.20140302.11
Page(s)	31-38
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2014. Published by Science Publishing Group

Keywords

Differential Item Functioning,Item Response Theory, Nonparametric Differential Item Functioning, Nonparametric IRT

References

[1]	Adams, R. J., & Rowe, K. J. (1988). Item Bias In J. P. Keeves (Ed.), Educational research, methodology, and measurement:An international handbook. (pp. 398-403). Oxford: Pergamon Press.
[2]	Adams, R. J., & Rowe, K. J. (1988). Item Bias In J. P. Keeves (Ed.), Educational research, methodology, and measurement:An international handbook. (pp. 398-403). Oxford: Pergamon Press.
[3]	Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory: Brooks/Cole Pub.
[4]	Carter, N. T., &Zickar, M. J. (2011). A Comparison of the LR and DFIT Frameworks of Differential Functioning Applied to the Generalized Graded Unfolding Model. Applied Psychological Measurement, 35(8), 623-642. doi: 10.1177/0146621611427898
[5]	Cohen, A. S., Kim, S.-H., & Baker, F. B. (1993). Detection of Differential Item Functioning in the Graded Response Model. Applied Psychological Measurement, 17(4), 335-350. doi: 10.1177/014662169301700402
[6]	Devine, P. J., &Raju, N. S. (1982). Extent of Overlap among Four Item Bias Methods. Educational and Psychological Measurement, 42(4), 1049-1066. doi: 10.1177/001316448204200412
[7]	Douglas, J. (1997). Joint consistency of nonparametric item characteristic curve and ability estimation. Psychometrika, 62(1), 7-28. doi: 10.1007/bf02294778
[8]	Finch, H. (2005). The MIMIC Model as a Method for Detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT Likelihood Ratio. Applied Psychological Measurement, 29(4), 278-295. doi: 10.1177/0146621605275728
[9]	Glickman, M., Seal, P., &Eisen, S. (2009). A non-parametric Bayesian diagnostic for detecting differential item functioning in IRT models. Health Services and Outcomes Research Methodology, 9(3), 145-161. doi: 10.1007/s10742-009-0052-4
[10]	Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory: SAGE Publications.
[11]	Holland, P. W., & Thayer, D. T. (1986). Differential item performance and the Mantel-Haenszel procedure Technical Report (Vol. 86-69). Princeton, NJ: Educational Testing Service.
[12]	Holland, P. W., Wainer, H., & Service, E. T. (1993). Differential Item Functioning: Taylor & Francis.
[13]	Lord, F. M. (1977). A Broad-Range Tailored Test of Verbal Ability. Applied Psychological Measurement, 1(1), 95-100. doi: 10.1177/014662167700100115
[14]	Lord, F. M. (1980). Applications of item response theory to practical testing problems: Erlbaum Associates.
[15]	Mellenberg, G. J. (1983). Conditional Item Bias Methods. In S. H. Irvine & W. J. Barry (Eds.), Human Assesment and Cultural Factors (pp. 123-144). Newyork: Plenum Pres.
[16]	Murphy, K. R., &Davidshofer, C. O. (1998). Psychological testing: principles and applications: Prentice Hall.
[17]	Nozawa, Y. (2008). Comparison of Parametric and Nonparametric IRT Equating Methods Under The Common-Item Nonequıvalent Groups Desing. Doctor of Philosopy, The University of Iowa.
[18]	Raju, N. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495-502. doi: 10.1007/bf02294403
[19]	Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611-630. doi: 10.1007/bf02294494
[20]	Ramsay, J. O. (2000). TestGraf A Program for the Graphical Analysis of Multiple Choice Test and Questionnaire Data Canada, Montreal, Quebec:McGill University
[21]	Rodney, G. L., &Drasgow, F. (1990). Evaluation of Two Methods for Estimating Item Response Theory Parameters When Assessing Differential Item Functioning. . Journal of Applied Psychology, 75(2), 164-174.
[22]	Roju, N. S., van der Linden, W. J., & Fleer, P. F. (1995). IRT-Based Internal Measures of Differential Functioning of Items and Tests. Applied Psychological Measurement, 19(4), 353-368. doi: 10.1177/014662169501900405
[23]	Roussos, L. A., & Stout, W. F. (1996). Simulation Studies of the Effects of Small Sample Size and Studied Item Parameters on SIBTEST and Mantel-Haenszel Type I Error Performance. Journal of Educational Measurement, 33(2), 215-230. doi: 10.1111/j.1745-3984.1996.tb00490.x
[24]	Rudner, L. M., Getson, P. R., & Knight, D. L. (1980). Biased Item Detection Techniques. Journal of Educational Statistics, 5(3), 213-233. doi: 10.2307/1164965
[25]	Shepard, L. A., Camilli, G., & Williams, D. M. (1985). Validity of Approximation Techniques for Detecting Item Bias. Journal of Educational Measurement, 22(2), 77-105. doi: 10.2307/1435009
[26]	Sijtsma, K., &Molenaar, I. W. (2002). Introduction to Nonparametric Item Response Theory: SAGE Publications.
[27]	Torgerson, W. S. (1958). Theory and methods of scaling: Wiley.
[28]	Wand, P., & Jones, C. (1994). Kernel Smoothing: Taylor & Francis.
[29]	Wang, W., Tay, L., &Drasgow, F. (2013). Detecting Differential Item Functioning of Polytomous Items for an Ideal Point Response Process. Applied Psychological Measurement, 37(4), 316-335. doi: 10.1177/0146621613476156
[30]	Zumbo, B. D., &Hubley, A. M. (2003). Item Bias. Encyclopedia of Psychological Assessment. SAGE Publications Ltd. London: SAGE Publications Ltd.

Cite This Article

Plain Text BibTeX RIS

APA Style

T. Oguz Basokcu, Tuncay Ogretmen. (2014). Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. American Journal of Theoretical and Applied Statistics, 3(2), 31-38. https://doi.org/10.11648/j.ajtas.20140302.11

Copy | Download

ACS Style

T. Oguz Basokcu; Tuncay Ogretmen. Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. Am. J. Theor. Appl. Stat. 2014, 3(2), 31-38. doi: 10.11648/j.ajtas.20140302.11

Copy | Download

AMA Style

T. Oguz Basokcu, Tuncay Ogretmen. Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. Am J Theor Appl Stat. 2014;3(2):31-38. doi: 10.11648/j.ajtas.20140302.11

Copy | Download

@article{10.11648/j.ajtas.20140302.11,
  author = {T. Oguz Basokcu and Tuncay Ogretmen},
  title = {Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale},
  journal = {American Journal of Theoretical and Applied Statistics},
  volume = {3},
  number = {2},
  pages = {31-38},
  doi = {10.11648/j.ajtas.20140302.11},
  url = {https://doi.org/10.11648/j.ajtas.20140302.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajtas.20140302.11},
  abstract = {This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.},
 year = {2014}
}

Copy | Download

TY  - JOUR
T1  - Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale
AU  - T. Oguz Basokcu
AU  - Tuncay Ogretmen
Y1  - 2014/03/20
PY  - 2014
N1  - https://doi.org/10.11648/j.ajtas.20140302.11
DO  - 10.11648/j.ajtas.20140302.11
T2  - American Journal of Theoretical and Applied Statistics
JF  - American Journal of Theoretical and Applied Statistics
JO  - American Journal of Theoretical and Applied Statistics
SP  - 31
EP  - 38
PB  - Science Publishing Group
SN  - 2326-9006
UR  - https://doi.org/10.11648/j.ajtas.20140302.11
AB  - This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.
VL  - 3
IS  - 2
ER  -

Copy | Download

Author Information

T. Oguz Basokcu

Department of Assessment and Evaluation in Education, Ege University, ?zmir, Turkey
Tuncay Ogretmen

Department of Assessment and Evaluation in Education, Ege University, ?zmir, Turkey

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

T. Oguz Basokcu, Tuncay Ogretmen. (2014). Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. American Journal of Theoretical and Applied Statistics, 3(2), 31-38. https://doi.org/10.11648/j.ajtas.20140302.11

Copy | Download

ACS Style

T. Oguz Basokcu; Tuncay Ogretmen. Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. Am. J. Theor. Appl. Stat. 2014, 3(2), 31-38. doi: 10.11648/j.ajtas.20140302.11

Copy | Download

AMA Style

T. Oguz Basokcu, Tuncay Ogretmen. Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale. Am J Theor Appl Stat. 2014;3(2):31-38. doi: 10.11648/j.ajtas.20140302.11

Copy | Download

@article{10.11648/j.ajtas.20140302.11,
  author = {T. Oguz Basokcu and Tuncay Ogretmen},
  title = {Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale},
  journal = {American Journal of Theoretical and Applied Statistics},
  volume = {3},
  number = {2},
  pages = {31-38},
  doi = {10.11648/j.ajtas.20140302.11},
  url = {https://doi.org/10.11648/j.ajtas.20140302.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajtas.20140302.11},
  abstract = {This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.},
 year = {2014}
}

Copy | Download

TY  - JOUR
T1  - Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale
AU  - T. Oguz Basokcu
AU  - Tuncay Ogretmen
Y1  - 2014/03/20
PY  - 2014
N1  - https://doi.org/10.11648/j.ajtas.20140302.11
DO  - 10.11648/j.ajtas.20140302.11
T2  - American Journal of Theoretical and Applied Statistics
JF  - American Journal of Theoretical and Applied Statistics
JO  - American Journal of Theoretical and Applied Statistics
SP  - 31
EP  - 38
PB  - Science Publishing Group
SN  - 2326-9006
UR  - https://doi.org/10.11648/j.ajtas.20140302.11
AB  - This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.
VL  - 3
IS  - 2
ER  -

Copy | Download