7.3 Relative Model-Data Fit at Test Level (Cont’d)

Let $P$ be the number of model parameters, several information criteria can be defined:

Akaike (1974) Information Criterion (AIC) adjusts the -2 log likelihood by twice the number of parameters in the model: $$AIC=-2\log L\left(Y\right)+2P$$ Schwarz (1978) Bayesian Criterion (BIC) has a stronger penalty than the AIC for overparametrized models, and adjusts the -2 log likelihood by the number of parameters times the log of the number of cases. It is also known as the Bayesian Information Criterion. $$BIC=-2\log L\left(Y\right)+P\log \left(N\right)$$ Bozdogan (1987) Consistent Akaike’s Information Criterion (CAIC) has a stronger penalty than the AIC for overparametrized models, and adjusts the -2 log likelihood by the number of parameters times one plus the log of the number of cases. As the sample size increases, the CAIC converges to the BIC. $$CAIC=-2\log L\left(Y\right)+P[\log \left(N\right)+1]$$ The sample-size-adjusted BIC (SABIC) is proposed by Sclove (1987) to reduce the penalty in BIC. $$SABIC=-2\log L\left(Y\right)+P[\log \left(\frac{N+2}{24}\right)]$$

References

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705

Bozdogan, H. (1987). Model selection and akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370. https://doi.org/10.1007/BF02294361

Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136

Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333–343. https://doi.org/10.1007/BF02294360