Configuring and Tuning Event Integrity Checks

To validate that the event information in your database matches the content sent from SmartConnectors, run an Event Integrity Check. When you run the check, Recon searches the database for verification events received within the specified date range, then runs a series of checks to compare content in the database with information supplied by the verification event. The results of an the Event Integrity Check help you identify whether event data might be compromised. In addition to reviewing the raw event data received from SmartConnectors, you can enable Transformation Hub to generate more than 20 parsed fields to include in the check.

For more information about verification events and running integrity checks, see the Help for Recon.

 

Configuring a SmartConnector to Include a Verification Event for Raw Events

For a SmartConnector to support event integrity checks, you must enable it include a verification event for each batch of events. This configuration ensures that the connector generates a verification event for the Raw Event field in an event at the moment that your environment captures the event.

For this setting... Enter...
Preserve Raw Event

Yes

NOTE: When you enable this setting, the size of each event increases, which will require more storage space in your database.
Event Integrity Algorithm SHA-256
Check Event Integrity Method Recon

For more information about configuring SmartConnectors, see the following documentation:

 

Enabling Transformation Hub to Generate Verification Events for Parsed Fields

The Event Integrity Check can verify the integrity of multiple fields within an event. You must enable Transformation Hub to generate verification events for the parsed fields received from the SmartConnectors. You can configure this setting as you deploy Transformation Hub or at any time after deployment.

  1. Log in to the Management Portal.

  2. Navigate to Transformation Hub > Stream Processors and Routers.

  3. Enable Generate verification events for parsed field integrity checks.

  4. For Verification event batch size, specify the number of events that you want to be associated with a verification event.

    A lower value indicates fewer associated events need to be included in the batch for integrity checks. However, a lower value will also result in higher resource consumption by generating more verification events.

 

Improving the Performance of Event Integrity Checks

If Event Integrity Checks consistently fail with notifications about insufficient resources or disk space, your system might not have enough memory or disk space to run the check. To mitigate this issue, you can adjust the event integrity settings.

In the CDF Management Portal, select Fusion > Event Integrity Check Configuration. Change the following settings as needed:

Event Integrity Task Count – Lower the Value
This setting specifies the number of tasks that can run in parallel during an Event Integrity Check. For load balancing, we recommend that you set the number of tasks proportional to the number of nodes in your cluster. For example, three tasks for a three-node cluster. Note that allowing more tasks increases CPU and memory usage but also enables the check to complete in a shorter amount of time.
Event Integrity Chunk Size – Lower the Value
This setting specifies the number of events within each data chunk that the Event Integrity Check processes at a time. To reduce the amount of memory and disk space needed to processes data, you can change the data chunks to a smaller size. However, a smaller chunk size can cause the check to take longer to complete.
Use Event Integrity Resource Pool – Set to TRUE

By default, the system uses the general resource pool for event integrity queries. However, if you change this setting to True, the system uses the dedicated event integrity resource pool. By moving all queries for the check to a dedicated pool, event integrity queries do not affect other queries and vice versa. For additional tuning of resource pools, it allows detailed tuning furthermore. (add link for db-installer)