6.1 Global Identifiability

A model is (globally) identifiable if the parameter values uniquely determine the probability distribution of the data and the probability distribution of the data uniquely determines the parameter values. (Everitt & Howell, 2005)

6.1.1 Parameters in CDMs

In CDMs, the model parameters are item parameters (denoted by $\Theta$, e.g., guess and slip for the DINA model) and population proportion parameters (denoted by $p$).

Person parameters (attribute profiles) are not model parameters in random-effect CDMs.

6.1.2 Global identifiability in CDMs

A CDM is identifiable if all item parameters and population proportion parameters are identified. More formally,

Definition 1 (G. Xu, 2019)

$(\Theta ,p)$ is said to be identifiable if for any $(\overline{\Theta },\overline{p})\ne (\Theta ,p)$, there exists at least one response pattern $x$ such that

$$P(x|Q,\overline{\Theta },\overline{p})\ne P(x|Q,\Theta ,p)$$

Exercise 6.1 Suppose a test with three items measures two attributes and the Q-matrix is given below:

$$\left[\begin{array}{cc}1& 0\\ 1& 0\\ 1& 1\end{array}\right]$$

For simplicity, suppose all items have the same slip parameters, denoted by $s$ and the same guessing parameters, denoted by $g$.

Please show that $p\left(00\right)$ and $p\left(01\right)$ are not identifiable.

Click for Answer

It can be shown that the conditional probability of all response vectors for $00$ and $01$ are identical. For example, $$\begin{array}{cc}P(X=(000)|\alpha =(00\left)\right)& =g^3\\ P(X=(000)|\alpha =(01\left)\right)& =g^3\end{array}$$ This suggests that from a response vector, there is no way to distinguish $00$ from $01$, and therefore, $p\left(00\right)$ and $p\left(01\right)$ are not identifiable.

More formally, $$\begin{array}{cc}P(x|Q,\Theta ,p)=& \sum _{c}P(x|{\alpha }_c,Q,\Theta )p\left({\alpha }_c\right)\\ =& P(x|\alpha =(00),Q,\Theta )p(\alpha =(00\left)\right)+\\ & P(x|\alpha =(10),Q,\Theta )p(\alpha =(10\left)\right)+\\ & P(x|\alpha =(01),Q,\Theta )p(\alpha =(01\left)\right)+\\ & P(x|\alpha =(11),Q,\Theta )p(\alpha =(11\left)\right)\end{array}$$

Because $P(x|\alpha =(00),Q,\Theta )=P(x|\alpha =(01),Q,\Theta )$ for all $x$, $p(\alpha =(00\left)\right)$ and $p(\alpha =(01\left)\right)$ cannot be uniquely determined.

References

Everitt, B., & Howell, D. C. (Eds.). (2005). Encyclopedia of statistics in behavioral science. John Wiley & Sons.

Xu, G. (2019). Identifiability and Cognitive Diagnosis Models (M. von Davier & Y.-S. Lee, Eds.; pp. 333–357). Springer International Publishing. http://link.springer.com/10.1007/978-3-030-05584-4_16