Home → Troubleshooting → @RISK for Excel: Simulation → Change in Output Mean Inconsistent with Sensitivity Tornado
Applies to: @RISK 6.x/7.x
Some of my inputs don't appear on the tornado diagram that shows regression coefficients. That makes sense because they have little or no impact on the output. But when I make a Change in Output Mean graph on that output, those inputs do show up with bars. What is wrong?
Most likely there's nothing wrong, and this is just normal statistical variability.
The Change in Output Statistic tornado does not measure the influence of any input on the output. The numbers are purely statistics, based on two parts of the simulation. One end of the bar is the mean of the output values from the iterations with the lowest 10% of input values, and the other end is the mean of the output values from the iterations with the highest 10% of input values. (For more on how the numbers are computed, please see Interpreting Change in Output Statistic in Tornado Graphs.)
Think of those two groups of iterations as two smaller samples from the simulation. Even if this input has zero impact on the output, you'd expect some difference between the means of two sub-samples from the same simulation. If you look at the standard error of the output (standard deviation divided by the square root of the number of iterations), you should see that the size of these unexpected bars is within a couple of standard errors, meaning that it can be explained by random chance and is not significant. If the difference is more than a couple of standard errors, then it is significant and may indicate impact of that input on the output.
My input does have some impact on the output, but it is minor. The numbers on the two ends of the bars are too far apart to be explained by statistical variation.
Remember, the Change in Output mean tornado doesn't measure impact. (For that, you want the regression coefficients tornado; see Interpreting Regression Coefficients in Tornado Graphs.) This input could have low impact on the output, as you say, but if it's correlated with other inputs, the output would be affected to a greater degree by the combination of those inputs.
See also: All Articles about Tornado Charts
Last edited: 2016-08-29