Statistical Power Simulation for Testing the Rasch Model

Description

Statistical power simulation for testing the Rasch Model based on a three-way analysis of variance design with mixed classification.

Author(s)

Takuya Yanagida [aut,cre] takuya.yanagida@univie.ac.at, Jan Steinfeld [aut] jan.steinfeld@univie.ac.at, Thomas Kiefer [ctb]

Maintainer: Takuya Yanagida <takuya.yanagida@univie.ac.at>

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

Verhelst, N. D. (2008). An efficient MCMC algorithm to sample binary matrices with fixed marginals. Psychometrika, 73(4), 705-728.

Verhelst, N., Hatzinger, R., & Mair, P. (2007). The Rasch sampler. Journal of Statistical Software, 20(4), 1-14.

See Also

aov.rasch, pwr.rasch

Sample of test data from subtest 2 of the Adaptive Intelligence Diagnosticum (AID3; Kubinger \& Holocher-Ertl, 2014)

Description

A dataset containing the test data of 300 childen (drawn randomly from the original dataset). The variables are as follows:

Usage

aid_st2

Format

A data frame with 300 rows and 28 variables:

• ID: ID variable of each testee

• age_in_month: the age of the testperson in month

• sex: gender of the testee

• country: country of the testee

• stage: stage of the data collection

• it1...it18: items of the subtest 2

Three-Way Analysis of Variance with Mixed Classification for Testing the Rasch Model

Description

This function applies the three-way analysis of variance with mixed classification for testing the Rasch model.

Usage

aov.rasch(data, group = "group", person = "person", item = "item",
  response = "response", output = TRUE)

Arguments

Details

The F-test in a three-way analysis of variance design (A > B) x C with mixed classification (fixed factor A = subgroup, random factor B = testees, and fixed factor C = items) is used to test the Rasch model. Rasch model fitting means that there is no interaction A x C. A statistically significant interaction A x C indicates differential item functioning (DIF) of the items with respect of the two groups of testees Note, if a main effect of A (subgroup) exists, an artificially high type I risk of the A x C interaction F-test results - that is, the approach works as long as no statistically significant main effect of A occurs. Note that in case of unbalanced groups computation can take a long time.

Value

Returns an ANOVA table

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

See Also

reshape.rasch, pwr.rasch

Examples

## Not run: 

# simulate Rasch model based data
# 100 persons, 20 items,
dat <- simul.rasch(100, items = seq(-3, 3, length.out = 20))
# reshape simulated data into 'long' format with balanced assignment
# of testees into two subgroups
dat.long <- reshape.rasch(dat, group = rep(0:1, each = nrow(dat) / 2))
# apply three-way analysis of variance with mixed classification for testing the Rasch model
aov.rasch(dat.long)

# extract variable names of items
vnames <- grep("it", names(aid_st2), value = TRUE)
# reshape aid subtest 2 data into 'long' format with split criterium sex
aid_long.sex <- reshape.rasch(aid_st2[, vnames], group = aid_st2[, "sex"])
# apply three-way analysis of variance with mixed classification for testing the Rasch model
aov.rasch(aid_long.sex)

## End(Not run)

Summary of DIF items

Description

This function builds a table of DIF items specified in the pwrrasch object

Usage

itemtable(object, all = FALSE, digits = 2)

Arguments

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

Examples

## Not run: 

# item parameters
ipar2 <- ipar1 <- seq(-3, 3, length.out = 20)
# model differential item function (DIF)
ipar2[10] <- ipar1[11]
ipar2[11] <- ipar1[10]
# simulation for b = 100
simres <- pwr.rasch(100, ipar = list(ipar1, ipar2))
itemtable(simres)

## End(Not run)

Plot Statistical Power Curve

Description

Generic plot function for the pwrrasch object, which plots the statistical power curve relating statistical power to sample size

Usage

## S3 method for class 'pwrrasch'
plot(x, plot.sig.level = TRUE, type = c("b", "b"),
  pch = c(19, 17), lty = c(1, 3), lwd = c(1, 1), legend = "topleft",
  bty = "o", ...)

Arguments

Details

Graphical parameters are:

• type The following values are possible: "p" for points, "l" for lines, "b" for both point and lines

• pch see points

• lty Line types can be specified as an integer (0 = blank, 1 = solid, 2 = dashed, 3 = dotted, 4 = dotdash, 5 = longdash, 6 = twodash)

• lwd Positive numbers indicating line widths

• legend Either the x and y coordinates to be used to position the legend or keyword from the list "bottomright", "bottom", "bottomleft", "left", "topleft", "top", "topright", "right" and "center"

• bty Allowed values are "o" (draw box around legend) and "n" (do not draw box around legend).

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

Examples

## Not run: 

# item parameters
ipar2 <- ipar1 <- seq(-3, 3, length.out = 20)
# model differential item function (DIF)
ipar2[10] <- ipar1[11]
ipar2[11] <- ipar1[10]
# simulation for b = 100, 200, 300, 400, 500
simres <- pwr.rasch(seq(100, 500, by = 100), ipar = list(ipar1, ipar2))
plot(simres)

## End(Not run)

Simulation to Estimate Statistical Power of a Rasch Model Test

Description

This function conducts a simulation to estimate statistical power of a Rasch model test for user-specified item and person parameters.

Usage

pwr.rasch(b, ipar = list(), ppar = list("rnorm(b, mean = 0, sd = 1.5)",
  "rnorm(b, mean = 0, sd = 1.5)"), runs = 1000, H0 = TRUE,
  sig.level = 0.05, method = c("loop", "vectorized"), output = TRUE)

Arguments

Details

The F-test in a three-way analysis of variance design (A \succ \mathbf{B}) x C(A > B) x C with mixed classification (fixed factor A = subgroup, random factor B = testee, and fixed factor C = items) is used to simulate statistical power of a Rasch model test. This approach using a F-distributed statistic, where the sample size directly affects the degree of freedom enables determination of the sample size according to a given type I and type II risk, and according to a certain effect of model misfit which is of practical relevance. Note, that this approach works as long as there exists no main effect of A (subgroup). Otherwise an artificially high type I risk of the A x C interaction F-test results - that is, the approach works as long as no statistically significant main effect of A occurs.

Value

Returns a list with following entries:

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

See Also

aov.rasch

Examples

## Not run: 

# item parameters
ipar2 <- ipar1 <- seq(-3, 3, length.out = 20)
# model differential item function (DIF)
ipar2[10] <- ipar1[11]
ipar2[11] <- ipar1[10]
# simulation for b = 200
pwr.rasch(200, ipar = list(ipar1, ipar2))

# simulation for b = 100, 200, 300, 400, 500
pwr.rasch(seq(100, 500, by = 100), ipar = list(ipar1, ipar2))

# simulation for b = 100, 200, 300, 400, 500
# uniform distribution [-3, 3] of person parameters
pwr.rasch(200, ipar = list(ipar1, ipar2), ppar = list("runif(b, -3, 3)", "runif(b, -3, 3)"))

## End(Not run)

Reshape data frame in wide format into a long format

Description

This function reshapes a matrix from 'wide' into a 'long' format. This is necessary for the three-way analysis of variance with mixed classification for testing the Rasch model.

Usage

reshape.rasch(data, group)

Arguments

Details

In order to apply the three-way analysis of variance with mixed classification for testing the Rasch model, data need to be in 'long' format. That is, Rasch model data design is interpreted as a analysis of variance design (A > B) x C, where items are levels of a fixed factor C and the testees are levels of a random factor B, nested within a fixed factor A of different subgroups.

Value

Returns a data frame with following entries:

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

See Also

aov.rasch

Examples

## Not run: 

# simulate Rasch model based data
# 100 persons, 20 items,
dat <- simul.rasch(100, items = seq(-3, 3, length.out = 20))
# reshape simulated data into 'long' format with balanced assignment
# of testees into two subgroups.
dat.long <- reshape.rasch(dat, group = rep(0:1, each = nrow(dat) / 2))
head(dat.long)

# extract variable names of items
vnames <- grep("it", names(aid_st2), value = TRUE)
# reshape aid subtest 2 data into 'long' format with split criterium sex
aid_long.sex <- reshape.rasch(aid_st2[, vnames], group = aid_st2[, "sex"])

## End(Not run)

Simulate data according to the Rasch model

Description

This function simulates data according to the Rasch model based on user-specified item and person parameters.

Usage

simul.rasch(persons, items, sum0 = TRUE)

Arguments

Details

If persons is an integer value, the corresponding parameter vector is drawn from N(0, 1.5). If items is an integer value, the corresponding parameter vector is equally spaced between [-3, 3]. Note that item parameters need to be normalized to sum-0. This precondition can be overruled using argument sum0 = FALSE.

Value

Returns a 0-1 matrix according to the Rasch model.

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

References

Kubinger, K. D., Rasch, D., & Yanagida, T. (2009). On designing data-sampling for Rasch model calibrating an achievement test. Psychology Science Quarterly, 51, 370-384.

Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). A new approach for testing the Rasch model. Educational Research and Evaluation, 17, 321-333.

See Also

aov.rasch, pwr.rasch

Examples

## Not run: 

# simulate Rasch model based data
# 100 persons, 20 items,
# person parameter drawn from a normal distribution: N(0,1.5)
# item parameters equally spaced between [-3, 3]
simul.rasch(100, items = 20)

# simulate Rasch model based data
# 100 persons, 17 items
# person parameter drawn from a uniform distribution: U[-4, 4]
# item parameters: [-4.0, -3.5, -3.0, ... , 3.0, 3.5, 4.0]
simul.rasch(runif(100, -4, 4), items = seq(-4, 4, by = 0.5))

## End(Not run)

Object Summary

Description

Generic summary function for the aovrasch object

Usage

## S3 method for class 'aovrasch'
summary(object, ...)

Arguments

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

Examples

## Not run: 

# simulate Rasch model based data
# 100 persons, 20 items,
dat <- simul.rasch(100, items = seq(-3, 3, length.out = 20))
# reshape simulated data into 'long' format with balanced assignment
# of examinees into two subgroups.
dat.long <- reshape.rasch(dat, group = rep(0:1, each = nrow(dat) / 2))
# apply three-way analysis of variance with mixed classification for testing the Rasch model.
res <- aov.rasch(dat.long)
summary(res)

## End(Not run)

Object Summary

Description

Generic summary function for the pwrrasch object

Usage

## S3 method for class 'pwrrasch'
summary(object, ...)

Arguments

Author(s)

Takuya Yanagida takuya.yanagida@univie.ac.at, Jan Steinfeld jan.steinfeld@univie.ac.at

Examples

## Not run: 

# item parameters
ipar2 <- ipar1 <- seq(-3, 3, length.out = 20)
# model differential item function (DIF)
ipar2[9] <- ipar1[12]
ipar2[12] <- ipar1[9]
# simulation for b = 100
simres <- pwr.rasch(100, ipar = list(ipar1, ipar2))
summary(simres)

# item parameters
ipar2 <- ipar1 <- seq(-3, 3, length.out = 20)
# model differential item function (DIF)
ipar2[10] <- ipar1[11]
ipar2[11] <- ipar1[10]
# simulation for b = 100, 200, 300, 400, 500
simres <- pwr.rasch(seq(100, 500, by = 100), ipar = list(ipar1, ipar2))
summary(simres)

## End(Not run)