6.1 Global Identifiability
A model is (globally) identifiable if the parameter values uniquely determine the probability distribution of the data and the probability distribution of the data uniquely determines the parameter values. (Everitt & Howell, 2005)
6.1.1 Parameters in CDMs
In CDMs, the model parameters are item parameters (denoted by \(\Theta\), e.g., guess and slip for the DINA model) and population proportion parameters (denoted by \(\mathbf{p}\)).
Person parameters (attribute profiles) are not model parameters in random-effect CDMs.
6.1.2 Global identifiability in CDMs
A CDM is identifiable if all item parameters and population proportion parameters are identified. More formally,
- Definition 1 (G. Xu, 2019)
-
\((\Theta,\mathbf{p})\) is said to be identifiable if for any \((\bar{\Theta},\bar{\mathbf{p}})\neq ({\Theta},{\mathbf{p}})\), there exists at least one response pattern \(\mathbf{x}\) such that
\[ P(\mathbf{x}|Q,\bar{\Theta},\bar{\mathbf{p}})\neq P(\mathbf{x}|Q,{\Theta},{\mathbf{p}}) \]
Exercise 6.1 Suppose a test with three items measures two attributes and the Q-matrix is given below:
\[ \begin{bmatrix} 1 & 0 \\ 1 & 0 \\ 1 & 1 \end{bmatrix} \]
For simplicity, suppose all items have the same slip parameters, denoted by \(s\) and the same guessing parameters, denoted by \(g\).
Please show that \(p(00)\) and \(p(01)\) are not identifiable.
Click for Answer
It can be shown that the conditional probability of all response vectors for \(00\) and \(01\) are identical. For example, \[\begin{align} P(\mathbf{X}=(000)|\alpha=(00)) &= g^3 \\ P(\mathbf{X}=(000)|\alpha=(01)) &= g^3 \\ \end{align}\] This suggests that from a response vector, there is no way to distinguish \(00\) from \(01\), and therefore, \(p(00)\) and \(p(01)\) are not identifiable.
More formally, \[\begin{align} P(\mathbf{x}|Q,{\Theta},{\mathbf{p}}) =&\sum_c P(\mathbf{x}|\alpha_c,Q,{\Theta}) p(\alpha_c)\\ =&P(\mathbf{x}|\alpha=(00),Q,{\Theta}) p(\alpha=(00))+\\ &P(\mathbf{x}|\alpha=(10),Q,{\Theta}) p(\alpha=(10))+\\ &P(\mathbf{x}|\alpha=(01),Q,{\Theta}) p(\alpha=(01))+\\ &P(\mathbf{x}|\alpha=(11),Q,{\Theta}) p(\alpha=(11)) \end{align}\]
Because \(P(\mathbf{x}|\alpha=(00),Q,{\Theta})=P(\mathbf{x}|\alpha=(01),Q,{\Theta})\) for all \(\mathbf{x}\), \(p(\alpha=(00))\) and \(p(\alpha=(01))\) cannot be uniquely determined.