Configuring ESM as a Transformation Hub Consumer – Non-FIPS Mode

This procedure uses the CA certificate that is embedded in Transformation Hub.

To complete the configuration, complete the following tasks:

  1. Obtain the Transformation Hub CA certificate.
  2. On the ESM server, configure ESM to consume from Transformation Hub.

The steps for each task are outlined below.

Obtaining the Transformation Hub CA Certificate

Transformation Hub maintains its own certificate authority (CA) to issue certificates for individual nodes in the Transformation Hub cluster. ESM needs that CA certificate in its truststore so that it will trust connections to Transformation Hub. For information about obtaining the certificate, see the information about viewing and changing the certificate authority. You might need to contact the Transformation Hub administrator to obtain the CA certificate if you do not have sufficient privileges to access the Transformation Hub cluster.

If you have root access to the Transformation Hub cluster (version 3.x or later), you can obtain the CA certificate as follows:

master=<master node host name or IP address>
ssh root@$master env K8S_HOME=/opt/arcsight/kubernetes /opt/arcsight/kubernetes/scripts/cdf-updateRE.sh > /opt/arcsight/manager/th.ca.crt

The command copies the CA certificate to the file /opt/arcsight/manager/th.ca.crt.

Configuring ESM to Consume from Transformation Hub

  1. Run the following command:

    /opt/arcsight/manager/bin/arcsight managersetup -i console

  2. In the wizard, press Enter until the wizard asks whether you want to read events from Transformation Hub. Select Yes, then provide the following information:

    1. Host name and port information for the worker nodes in Transformation Hub. Use a comma-separated list (for example: <host>:<port>,<host>:<port>) and specify the FQDN of the worker nodes.

      Note: You must specify the host name and not the IP address.

      Transformation Hub can only accept IPv4 connections from ESM.

      If the Kafka cluster is configured to use SASL/PLAIN authentication, ensure that you specify the port configured in the cluster for the SASL_SSL listener.

    2. Topics in Transformation Hub from which you want to read. These topics determine the data source.

      Note: You can specify up to 25 topics using a comma-separated list (for example: topic1,topic2). ESM will read Avro-format events from any topic where the name contains "avro" in lower case. For example, th-arcsight-avro.

    3. Path to the Transformation Hub root certificate (/opt/arcsight/manager/th.ca.crt).

    4. Leave the authentication type as None.
    5. Leave the user name and password as empty.

    6. If you specified an Avro topic, specify the host name and port for connecting to the Schema Registry in the format <host name:port>.

      Note: The default port for connecting to the Schema Registry is 32081.

      Transformation Hub runs a Confluent Schema Registry that producers and consumers use to manage compatibility of Avro-format events.

      The wizard uses this information to connect to the Schema Registry, read the Avro schemas for the Avro topic that you specified, and verify that the topic contains Avro events that are compatible with ESM. If ESM cannot retrieve the Avro schemas for the Avro topic that you specified and compare it to the event schema that is packaged with ESM, or if incompatible schemas are detected, the wizard generates warning messages but allows you to continue. In some cases, you might already know that Transformation Hub will use a compatible schema when the Manager is running.

    7. If you choose to configure the Forwarding Connector to forward CEF events to Transformation Hub and then configure Transformation Hub to filter Avro events, use filters to ensure that ESM does not receive duplicate events. You might want to use filters to accomplish the following:

      • Filter out desired events from Connectors so that ESM does not process them.

      • Filter out ESM's correlation events that were forwarded (CEF events that the Forwarding Connector sent to th-cef) so that ESM does not re-process its own events.

        If you do not configure filtering, ESM must consume from the th-arcsight-avro topic. If you configure filtering, ESM must consume from the mf-event-avro-esmfiltered topic. For more information, see configuring filters and local and global event enrichment.

    After providing the information, specify Yes and complete the remaining sections of the wizard.

  3. After you complete the wizard, restart the Manager services:

    In compact mode: 

    /etc/init.d/arcsight_services stop manager
    /etc/init.d/arcsight_services start manager

    In distributed mode: 

    /etc/init.d/arcsight_services stop all
    /etc/init.d/arcsight_services start all

  4. Verify that the connection was successful:

    grep -rnw '/opt/arcsight/var/logs/manager/' -e 'Transformation Hub service is initialized' -e 'Started kafka readers'

    The output should be similar to the following:

    /opt/arcsight/var/logs/manager/default/server.std.log:5036:2021-07-13 09:51:36 =====> Transformation Hub service is initialized (49 s) <=====

    /opt/arcsight/var/logs/manager/default/server.log:11664:[2021-07-13 09:51:36,656][INFO ][default.com.arcsight.common.messaging.events.aa] Started kafka readers in PT0.115S

    /opt/arcsight/var/logs/manager/default/server.log:11665:[2021-07-13 09:51:36,657][INFO ][default.com.arcsight.server.NGServer] Transformation Hub service is initialized (49 s)