Palisade Knowledge Base

HomeTechniques and Tips@RISK Distribution FittingInterpreting AIC Statistics

4.12. Interpreting AIC Statistics

Applies to: @RISK 6.x/7.x, Professional and Industrial Editions

@RISK gives me several candidate distributions. How can I interpret the AIC statistics? How much of a difference in AIC is significant?

The answer uses the idea of evidence ratios, derived from David R. Anderson's Model Based Inference in the Life Sciences: A Primer on Evidence (Springer, 2008), pages 89-91. The idea is that each fit has a delta, which is the difference between its AICc and the lowest of all the AICc values. (@RISK actually displays AICc, though the column heading is AIC; see Discrepancy in AIC Calculation?)

Example: suppose that the normal fit has the lowest AICc, AICc = –110, and a triangular fit has AICc = –106. Then the delta for the triangular fit is (–106) – (–110) = 4.

The delta for a proposed fit can be converted to an evidence ratio. Anderson gives a table, which can also be found on the Web. One place is page 26 of Burnham, Anderson, Huyvaert's "AIC model selection and multimodel inference in behavioral ecology", Behav Ecol Sociobiol (2011) 65:23–35 (PDF, accessed 2014-07-11). In the table, a delta of 4 corresponds to an evidence ratio of 7.4, meaning that the normal fit is 7.4 times as likely as the triangular fit to be the right fit. If you had to choose between those two only, there's a 7.4/8.4 = 88% chance that the normal is right, and a 1/8.4 = 12% chance that the triangular is right. But of course you usually have more than two fits to choose from.

To give you a further idea, delta = 2 corresponds to an evidence ratio of 2.7, and delta = 8 to an evidence ratio of 54.6.

So how high does delta need to be before you reject a proposed fit as unlikely? Anderson cautions, "Evidence is continuous and arbitrary cutoff points ... should not be imposed or recognized." Yes, the higher deltas correspond to higher evidence ratios, so you can think of them as higher evidence against the lower-ranking fit, but you can never reject a fit with complete certainty. And of course if all the fits are poor then the best of them is still not a good fit. One other argument against relying solely on mechanical tests: A model that is a poorer overall fit may nonetheless be better in the region you care most about, or vice versa. It's always advisable to look at the fitted curves against the histogram of the data when making your final decision.

Last edited: 2015-06-19

This page was: Helpful | Not Helpful