11.10 Assignment II

  1. Why is model identifiability important and what are the possible consequences of interpreting results from a non-identified model?

  2. What are model parameters for a CDM when marginalized maximum likelihood estimation (MMLE) is used? Why are person attribute profiles not considered as model parameters?

  3. Discuss how person’s attribute profiles can be estimated when MMLE is used. What are the relationship between different estimation methods for person’s attribute profiles?

  4. Why is it important to assess model-data fit, if all models are deemed wrong?

  5. How should absolute and relative fit measures be used together in practice?

  6. If the classification accuracy for an attribute is estimated to be .8, What does it mean? Can we say a model with higher classification accuracy is better than a model producing lower classificationa accuracy?

  7. Please analyze data2 and answer the following questions:

  • Fit the G-DINA model and DINA model to data2.

  • Based on information criteria and likelihood ratio test, which model can fit data better?

  • Check if the better-fit model can fit data adequately in an absolute sense.

  • Please use Wald test to see if the G-DINA model can be simplified at item level and report your absolute and relative fit results related to the Wald-selected models.

  • Using the most appropriate model, which could be the G-DINA model, DINA model or the models selected by the Wald test, (1) report and interpret the classification accuracy, (2) report the proportion of students with each attribute profile, (3) report which attribute is the hardest and which one is the easiest and (4) find the estimated attribute profiles for the first 10 students in the sample.

  1. (Optional) Please conduct a simulation study to evaluate how misspecification of models affects the estimation of classification accuracy.
  • Simulate data from the DINA model, with true guessing and slip parameters being fixed at .15 for all items.
  • After generating data from the DINA model, fit your data using both DINA, DINO and G-DINA model.
  • Estimate the classification accuracy using CA function from the GDINA package and compare the classification accuracy of three models.
  • Since the true attribute profiles are known, please calculate the classification accuracy by comparing the estimated and true attributes.
  • Compare the classification accuracy calculated from the previous steps.