Detecting a Heap Corruption

Describes the Micro Focus tools that can help you to detect a heap corruption.

When memory is allocated using a Micro Focus product, additional bytes known as guard bytes are allocated before and after the allocated block of memory. These guard bytes are populated with a known value so that unintentional overwrites can be detected.

The heap memory can be checked for corruption by calling the CBL_MEM_VALIDATE API (or the identical api PLI_MEM_VALIDATE). It checks both currently allocated memory and the free chains (if applicable), looking to see if any of the guard bytes have been overwritten.

Optionally, you can use the following COBOL run-time tunables:

memory_strategy - automatically check allocated memory and the free chains at every free and allocate
memory_free_check_size - determines how much freed memory to monitor

Important: The greater the number of checks you turn on, the greater the affect on performance. Open PL/I uses dynamic memory heavily for things like building up string concatenations, etc. so having all of these checks done all the time can get expensive.

In addition, Open PL/I assigns a type to each memory allocation. You can use the assigned type to determine which type of memory has been corrupted. Memory allocated by the Open PL/I run-time system has a type between 65350 and 65399. The most common type is 65398. (65350-65397 are not used.)

When a heap corruption occurs, it is reported as follows:

Default

By default, with no COBOL memory_strategy run-time tuneable in place, limited heap corruption detection is enabled for a program running under Enterprise Server. However, a PLIDUMP reports heap corruption occurrences at the top of the report because Open PL/I always triggers the CBL_MEM_VALIDATE API and reports its results before processing the remainder of a PLIDUMP. The following is a snippet from the top of a PLIDUMP showing a heap corruption:

Memory Strategy: 00000001
Corruption at: 0E2A776C Flags:0000000F Type: 0 Size: 0

This message indicates that the corrupted memory is at address 0E2A776C (32-bit program) and the type allocated to the memory is 0, meaning that the corrupted memory was not allocated by PL/I. The Flags value shows what type of memory corruption was detected (as opposed to the allocated type), who detected it, etc. See CBL_MEM_VALIDATE for a full description.

COBOL run-time system (RTS) errors and events

The JCL job log reports memory corruption as a COBOL RTS 252 error.

CTF trace reports memory corruption as COBOL RTS 249 and 252 events. The information is similar to what is displayed in the PLIDUMP showing the corruption address, type, size, etc. The following is an example of a CTF Trace RTS 249 event:

21:59:33.009 5892 MF.RTS 249 3 : 0X14025F10 65398 1 0X140362A2

This message contains several bits of useful information:

`65398`	The corrupted memory was allocated by PL/I. It could be a CONTROLLED or BASED variable, or even a temporary variable created by PL/I to handle an intermediate result.
`1`	The problem was a Run-Out of user data, indicating that something wrote PAST the end of the allocated block.
`0X140362A2`	The address of the check bytes that were hammered.

The mf.rts.xml in the %ProgramFiles(x86)%\Micro Focus\Enterprise Developer\etc\mftrace\annotations (Windows) or /etc/mftrace/annotations (UNIX) directory provides the following additional information for the 249 event:

      <Event Id="249" Description="Memory Stomp">
        <Arguments>
          <Argument Index="0" Description="Data Address" />
          <Argument Index="1" Description="Memory Type" />
          <Argument Index="2" Description="">
            <ArgumentCase Index="2" Value="0" Description="Run-In (Header)"/>
            <ArgumentCase Index="2" Value="1" Description="Run-Out (User Data)"/>
            <ArgumentCase Index="2" Value="2" Description="Bad Header"/>
            <ArgumentCase Index="2" Value="3" Description="Freed Overwrite"/>
            <ArgumentCase Index="2" Value="4" Description="Bad Header User"/>
            <ArgumentCase Index="2" Value="5" Description="Failed OS Check"/>
            <ArgumentCase Index="2" Value="6" Description="Run-In (User Data)"/>
            <ArgumentCase Index="2" Value="7" Description="Bad Track Data"/>
            <ArgumentCase Index="2" Value="8" Description="Bad Link"/>
          </Argument>
          <Argument Index="3" Description="">
            <ArgumentCase Index="2" Value="0" Description="Check Bytes Address"/>
            <ArgumentCase Index="2" Value="1" Description="Check Bytes Address"/>
            <ArgumentCase Index="2" Value="2" Description="Header Address"/>
            <ArgumentCase Index="2" Value="3" Description="Overwrite Address"/>
            <ArgumentCase Index="2" Value="6" Description="Check Bytes Address"/>
            <ArgumentCase Index="2" Value="7" Description="Track Data Address"/>
            <ArgumentCase Index="2" Value="8" Description="Header Address"/>
          </Argument>
        </Arguments>
      </Event>