HomeTechniques and Tips@RISK Simulation: Graphical ResultsInterpreting Change in Output Statistic in Tornado Graphs

7.8. Interpreting Change in Output Statistic in Tornado Graphs

Applies to: @RISK 6.x/7.x

How do I interpret the double-sided tornado graphs in Quick Reports, Browse Results, and Sensitivity Analysis? What's the default behavior, and how can I change it?

Let's talk first about the default behavior for Change in Output Mean, which is the default statistic, and then we can go into the variations. We'll suppose that you have 2500 iterations in your simulation.

The baseline is the overall simulated mean of that output.

The double-sided tornado has one bar for each selected input, and each bar has numbers at its edges. Each bar is prepared by considering one input and ignoring everything else but the output. (The other inputs are not held constant; their values from the simulation are simply not used in the computation.)

Here's how the numbers on each bar are computed for Change in Output Mean:

  1. @RISK puts all the iterations in order by ascending values of that input.
  2. @RISK divides those ordered iterations into 10 bins. With 2500 iterations, the first bin contains the 250 iterations with the 250 lowest values of this input; the second bin contains the 250 iterations with the 251st to 500th lowest values of this input; and so on to the last bin, which contains the 250 iterations with the 250 highest values of this input.
    Note: The bins all have the same number of iterations. For a uniform distribution that means they all have the same width, but for most distributions the bins will have different widths so that they all have the same number of iterations. Another way to look at it is that the bins have equal probability and the same number of iterations, but most likely not equal width based on the shape of the distribution.
  3. @RISK computes the mean of the output values within each bin.
  4. @RISK looks at the ten output means from the ten bins. The lowest of the ten output means becomes the number at the left edge of the bar for this input, and the highest of the ten output means becomes the number at the right edge of the bar.

The key here is that the inputs are first sorted in ascending order and binned in that order, then an output mean is computed for just the iterations in each bin.

Different shading, beginning with @RISK 7.5, shows you which end of each bar represents high input values and which represents low input values. Thus you can easily tell which inputs have positive impact on this output (high inputs at the right) and which have negative impact (high inputs at the left).

In @RISK 6.0–7.0, there's no way to see from the graph which bin produced which output mean. For instance, if a Change in Output Mean bar goes from 1500 to 4980, you don't know whether that output mean of 1500 came from the bin with the 250 lowest input values, or the bin with the 250 highest input values, or a bin with intermediate input values. This is where correlation sensitivities or a scatter plot can help, to tell you whether increasing values of an input tend to associate with increasing values of the output, with decreasing values of the output, or with some more complicated trend.

Note: The change in output values does not necessarily indicate any influence of that input on the output. For more, see Change in Output Mean Inconsistent with Sensitivity Tornado.

Variation: number of bins
If you increase the number of bins, @RISK will have more output means, each representing a smaller number of inputs. For most models, that translates to a greater range of output means. For a very simplified example, please have a look at the attached workbook. (You don't want so many bins that each one has only a few iterations; see next paragraph.)

Variation: number of iterations
With more iterations, from one simulation to the next you'll see less variability in Change in Output Mean, just as with any other output statistic. In other words, output statistics are more stable with more iterations. With fewer iterations, you'll see more variability in all your output statistics. However, from one simulation to the next, output statistics should vary only within normal statistical variability for the number of iterations.

Variation: choice of statistic
You can display a change in output percentile rather than a change in output mean. In this case, the computation is similar but instead of an output mean for each bin @RISK computes an output percentile for each bin. For example, with 2500 iterations and 10 bins, if you select Change in 90th Percentile then @RISK will compute the 90th percentile of the output values within each of the 10 bins (within each group of 250 iterations sorted by input values), and the edges of the bar will be the smallest and largest of those 90th percentiles. The baseline of the output becomes the overall 90th percentile of the simulation.

Setting preferences for double-sided tornado
To set default number of bars for double-sided tornado in Browse Results, Sensitivity Analysis, and Quick Reports, select Utilities » Application Settings » Sensitivity Defaults. If necessary, set Preferred Calculation Method to Change in Output Statistic. You can then change the number of bars and your preferred statistic. (The Change in Output Statistic tornado will never show more than 10 bars. If you set a maximum greater than 10, it applies only to the correlation and regression sensitivity tornado graphs. However, you can still use a worksheet function to retrieve the change in output statistic for lower-ranked inputs.)

To set the preferences for a particular Browse Outputs graph, click the tornado icon at the bottom of the graph window and select Settings.

Last edited: 2017-04-19


This page was: Helpful | Not Helpful