Configuring Event Integrity Checks

To validate that the event information in your database matches the content sent from SmartConnectors, run an Event Integrity Check. When you run the check, Recon searches the database for verification events received within the specified date range, then runs a series of checks to compare content in the database with information supplied by the verification event. The results of an Event Integrity Check help you identify whether event data might be compromised. In addition to reviewing the raw event data received from SmartConnectors, you can enable Transformation Hub to generate more than 20 parsed fields to include in the check.

For more information about verification events and running integrity checks, see the Help.

 

Configuring a SmartConnector to Include a Verification Event for Raw Events

For a SmartConnector to support event integrity checks, you must enable it to include a verification event for each batch of events. This configuration ensures that the connector generates a verification event for the Raw Event field in an event at the moment that your environment captures it.

For this setting... Enter...
Preserve Raw Event

Yes

NOTE: When you enable this setting, the size of each event increases, which will require more storage space in your database.
Event Integrity Algorithm MD5, SHA-1, or SHA-256
Check Event Integrity Method Recon

For more information about configuring SmartConnectors, see the following topics:

 

Enabling Transformation Hub to Generate Verification Events for Parsed Fields

The Event Integrity Check can verify the integrity of multiple fields within an event. You must enable Transformation Hub to generate verification events for the parsed fields received from the SmartConnectors. You can configure this setting as you deploy Transformation Hub or at any time after deployment, such as an upgrade.

It’s important to tune the number of partitions of the enrichment stream processor source topic before enabling Transformation Hub to generate verification events for parsed fields. If you change the number of partitions of the source topic after enabling it, you must browse to Kafka Manager’s Topics section and do the following:
1. Adjust and match the number of partitions of the Integrity events Enrichment changelog with the source topic number of partitions. The internal topic is named with the following format and pattern: com.arcsight.th.AVRO_ENRICHMENT_1-integrityMessageStore-changelog.
2. Restart the TH Web services pod by running the following command:
kubectl delete pod th-web-service-xxxxxxxxx-yyyyy -n arcsight-installer-yyyyy

  1. Log in to the Management Portal.

  2. Navigate to Transformation Hub > Stream Processors and Routers.

  3. Enable Generate verification events for parsed field integrity checks. Default value is false. If true, a verification event is generated that accompanies a batch of events for checking the integrity of parsed fields in each event. Recon uses this verification event to check event integrity. If true, then specify a value for Verification event batch size as described below.

  4. For Verification event batch size, specify the number of events that you want to be associated with a verification event. Default value is 256. A lower value indicates fewer associated events need to be included in the batch for integrity checks. However, a lower value will also result in higher resource consumption by generating more verification events.

This process generates an internal topic named with the following format and pattern com.arcsight.th.AVRO_ENRICHMENT_1-integrityMessageStore-changelog. The setting “# of replicas assigned to each Kafka Topic” setting also applies to it.
If the flow of events is not consistent, and there are long intervals between the reception of events, the feature will check every hour (60 mins) for a summary event that hasn’t reached the verification event batch size. If it hasn’t been sent for more than 4 hours (240 mins), then it will be sent with the aggregated info of the previous number of events, regardless of whether it reached the verification event batch size.