Comparison of strategies for validating binary logistic regression models
Comparison of strategies for validating binary logistic regression models - Online sex
a number of variable information gain criteria may be more useful than discrimination criteria.(1) There is an extensive literature on why one should prefer full models to restricted/parsimonious models.
However, larger models may not be feasible for many clinical applications.This is a toy example; however, I am typically asked to build models with 8-12 variables, and removing variables that do not contribute to a model is a primary interest for me.Variable 17 just appears to mean very little to the model as a whole (in terms of predictability), yet the likelihood ratio test tells us there is a significant difference between the two models (likely due to large n rather than an actual difference in these two models).As such, I am hoping to find a way to compare these two models (finding a method that does not indicate a difference between these two models)(1) not sure I quite understand what you're looking for.But in medicine the issue with using discrimination like c-statistic is well established, c-statics can be unchanged even with addition of significant variable, and has lead to development of reclassification indices (circ.ahajournals.org/content/121/15/1768.full) (2) are the AIC/BIC similar?They may be less well calibrated than larger models, external/internal validity may be reduced.
(4) The c statistic may not be optimal in assessing models that predict future risk or stratify individuals into risk categories.
In this setting, calibration is as important to the accurate assessment of risk.
(2) As far as I know, Discrimination/Discrimination indexes aren’t (?
should not be) used as a model/variable selection parameter.
They aren’t intended for this use and as a result there may not be much of a literature on why they shouldn’t be used for model building.
(3) Parsimonious models may have limitations that aren’t readily apparent.