HomeEnd User SetupConfiguring WindowsData Execution Prevention

3.6. Data Execution Prevention

How should I set DEP (Data Execution Prevention) for compatibility with Palisade software?

The simple answer is: don't. Windows default settings for DEP are just fine with Palisade software, but our software will not work with certain non-default settings. DEP policy can also be manipulated independently of the Windows policy, for example by using Microsoft's Enhanced Mitigation Experience Toolkit (EMET), an optional Windows component. We don't recommend changing DEP policy via EMET either.

Please read on for more about the compatibility of Palisade software with different Windows DEP policies and EMET policies.

A. Windows DEP Settings: WMIC and BCDEDIT

This command will check the DEP setting:

wmic OS Get DataExecutionPrevention_SupportPolicy

Either open a command prompt and type or paste that command, or open a Start » Run window and type or paste

cmd /k wmic OS Get DataExecutionPrevention_SupportPolicy

It will return one of these values:

2 OptIn (the Windows default) Only Windows system components and services have DEP applied.

3 OptOut DEP is enabled for all processes. Administrators can create a list of specific applications that will not have DEP applied.

1 AlwaysOn DEP is enabled for all processes.

0 AlwaysOff DEP is not enabled for any processes.

Our software should work with the default value 2 or the non-default values 0 and 3.

Only the value 1 is a problem. The value 1 turns DEP on for all processes. Palisade software can run with DEP on when the policy is set to 3, but the value 1 also turns off the setting called "ATL Thunk Emulation" for all processes. If Data Execution Prevention is On and ATL Thunk Emulation is Off, Palisade software will not run.

If the value 1 is returned, open an administrative command prompt. Enter this command:

bcdedit.exe /set {current} nx OptIn

Restart Windows — not just logging off and on, but a system restart — and use the wmic command again to verify that the value is now 2.

B. DEP Policy in EMET or Another Tool

DEP settings via EMET or another system tool can be a problem, even if wmic returns the desired value of 2. To test for this problem, download and run the attached program, StatusOfDataExecutionPrevention.exe.

Any result is fine, except for Data Execution Prevention = On with ATL Thunk Emulation = Off. That result indicates settings incompatible with our software. The bcdedit command in Part A should have fixed this. If it did not, then DEP has been set via a policy independent of the Windows settings, most likely via the EMET tool.

The possible DEP policies in EMET are similar to the Windows policies. If DEP is set to "Always On" in EMET, Palisade software will not run, just like with the Windows AlwaysOn policy (wmic value 1). However, EMET's "Application Opt Out" policy is different from the Windows OptOut policy (wmic value 3). Both of them turn DEP on for processes that are not on the list of exceptions. The difference lies in how they handle "ATL Thunk Emulation", which breaks Palisade software if set to off. The Windows OptOut policy doesn't turn ATL Thunk Emulation off, but EMET's "Application Opt Out" does turn ATL Thunk Emulation off on Windows 8.1 and higher.

Therefore, if DEP is set via EMET, and the EMET policy needs to be set to "Application Opt Out" for some reason, you will need to add some Palisade processes to the list of exceptions. You can find these processes in subfolders under your Palisade install folder, which by default is C:\Program Files (x86)\Palisade or C:\Program Files\Palisade. Under Palisade, look at the folder for each of your installed products, such as RISK7 or StatTools6. You want to disable DEP for each "OutOfProcess" executable file in each product folder. If you have the DecisionTools Suite, look in the folders for all the components: BigPicture (DecisionTools Suite 7 only), Evolver, NeuralTools, PrecisionTree, RISK, StatTools, and TopRank.

Last edited: 2017-01-11

Downloads

This page was: Helpful | Not Helpful