Configuring the Deployed Capabilities

For guidance on configuring your deployment, see "System Performance Benchmarks for Sizing and Tuning" in the Technical Requirements for ArcSight Platform for your workload. It might specify additional settings beyond what is described below.

You are now ready to deploy and then configure your deployed capabilities. The Pre-Deployment Configuration page displays instructions to configure the products and capabilities chosen at the start of the installation process. This section explains the process of configuring deployed capabilities on a supported platform for both on-premises and cloud deployments.

Understanding the Parameters in the Example Config Files
Reviewing Settings that Must Be Configured During Deployment
Transformation Hub
Fusion
ArcSight Database
Intelligence

Understanding the Parameters in the Example Config Files

The following parameters are mentioned in one or more of the example install config files.

Transformation Hub

For the Transformation Hub yaml, see the following:

Name	Description
routing-processor1-replicas	Specifies the number of Routing Stream Processor Instances to start for the Group 1 Stream Processor. Routing Stream Processors convert incoming CEF events based on predefined rules associated with a unique source Topics. Group numbers are dynamically assigned by Transformation Hub. Tune the number of instances based on throughput requirements.
th-init-noOfTopicPartitions	For newly created Kafka Topics, specifies the number of partitions assigned to each Topic. Default is 6. A Partition is the unit of parallelism in Kafka, enabling write operations on both the Producer and Broker to be performed concurrently. This is a key tuning property.
transform-processor-replicas	Specifies the number of CEF-to-Avro Stream Processor Instances to start. CEF-to-Avro Stream Processors convert incoming CEF events from th-cef topic to Avro format and route these events to th-arcsight-avro topic.
th-init-kafkaRetentionBytesForVertica	Specifies the size, in gigabytes, of the retention log for th-arcsight-avro and mf-event-avro-enriched Topics (Avro primary Topics). Default is 60 GB. This is a key tuning property. This log is associated with Avro processing. It is uncompressed and might require up to 7 times more space than compressed data. When this log size is exceeded, event data will be dropped.
th-init-kafkaRetentionBytes	Specifies the size, in gigabytes, of the retention log for each Kafka Topic. Default is 60 GB. This is a key tuning property. When the retention log exceeds the size limit, event data will be dropped.
enrichment-processor1-replicas	Specifies the number of Enrichment Stream Processor Group Instances to start. Enrichment Stream Processors transform incoming events based on the set of enabled event enrichment features, and route these events to one or more destination Topics. Enrichment examples include adding Global Event IDs and event integrity checking. Tune the number of instances based on throughput requirements.
th-enrichment-processor-group1-source-topic	Specifies the source Topic to be used by the Enrichment Stream Processor Group.
th-enrichment-processor-integrity-enabled	Indicates whether to generate a verification event that accompanies a batch of events for checking the integrity of parsed fields in each event. Recon uses this verification event to check event integrity. Also, specify a value for ‘Verification event batch size’.
th-enrichment-processor-integrity-batch-size	Specifies the number of events to be associated with a verification event. A lower value indicates fewer associated events need to be included in the batch for integrity checks; however, it will also result in higher resource consumption by generating more verification events.

Recon

For the Recon YAML file, see the following:

Name	Description
interset-elasticsearch-data-instances	Specifies the number of Elasticsearch data processing instances.
interset-elasticsearch-index-replicas-count	Specifies the number of replicas for each Elasticsearch index. 0 means no copy, only use that value when having no HA/Production requirement.
interset-logstash-event-buffering	Specifies the internal queuing model to use for event buffering. Specify memory for legacy in-memory based queuing; persisted for disk-based queuing.
interset-logstash-instances	Specifies the number of Logstash instances.
recon-enable	Indicates whether to explore events in Recon in addition to Intelligence.

Reviewing Settings that Must Be Configured During Deployment

This section describes configuration settings that must be set during deployment. Additional settings can be modified after deployment by logging in to the OMT Management Portal.

For more information on a setting, hover over the setting to display the setting tooltip, then set the values accordingly.

The following products require configuration settings to be set during deployment.

Transformation Hub
Fusion
ArcSight Database
Intelligence

Transformation Hub

If you deployed Transformation Hub, in the Transformation Hub tab, ensure the following are set to the number of Kafka worker nodes in your deployment or what is specified in the Technical Requirements for ArcSight Platform for your workload.

# of Kafka broker nodes in the Kafka cluster (th-kafka-count)
# of ZooKeeper nodes in the ZooKeeper cluster (th-zookeeper-count)
# of replicas assigned to each Kafka Topic (th-init-topicReplicationFactor) (This setting must be set to 1 for a single worker deployment, and 2 for a 3-node environment.)

On the Transformation Hub tab, configure the following security settings based on how you planned to secure communications as described in the Securing Communication Among Micro Focus Components section.

FIPS, Client Authentication, and Allow Plain Text connections to Kafka settings are available during installation and deployment only.

Allow plain text (non-TLS) connections to Kafka (th-kafka-allow-plaintext)
Enable FIPS 140-2 Mode (th-init-fips)
Connection to Kafka uses TLS Client Authentication (th-init-client-auth)
# of message replicas for the __consumer_offsets Topic (th-init-kafkaOffsetsTopicReplicationFactor)
Schema Registry nodes in the cluster (th-schema-registry-count)

If you are deploying ESM, configure your Enrichment Stream Processor Group source Topic according to the scope for which you want to leverage ESM's event enrichment capability. For more information, refer to Enrichment Stream Processors.

Fusion

If you deployed Fusion, on the Fusion tab:

Single Sign-on Configuration: Modify the Client ID (sso-client-id) and Client Secret (sso-client-secret) to a unique value for your environment.
If you are deploying Transformation Hub and configured # of Enrichment Stream Processor Group instances to start (enrichment-processor1-replicas) with a value greater than zero (default is 2), which means Enrichment Stream Processor will be enabled, the Fusion ArcMC Generator ID Manager must be enabled with a sufficient range of IDs because the Enrichment Stream Processor automatically requests generator IDs from the Fusion ArcMC in the same cluster as needed for its processing. To enable the Fusion ArcMC Generator ID Manager, configure Enable Generator ID Manager (arcmc-generator-id-enable) to True (default is True) and set the values of Generator ID Range Start (arcmc-generator-id-start) and Generator ID Range End (arcmc-generator-id-end) to provide a range of at least 100 between them. A range of 100 should be sufficient for common scenarios with a comfortable buffer, but you could also make the range larger if you have configured a large number of Enrichment Stream Processor instances or other components that use Generator IDs from this Fusion ArcMC instance.

It is important to choose a range that does not overlap with the Generator ID Manager range configured in any other ArcMC instances in your organization, otherwise different events with duplicate Globally Unique Event IDs could be created.

Maximum Search Results: This value specifies number of results that a search can return. Maximum limit is 10 million events.
Maximum Number of Searches: This value specifies the maximum number of searches that can exist in the system at any point. The default maximum search limit is 1,000, but you can change it to any number between 100 and 10,000. Any value above 10,000 or below 100 will display the following error message: "The value should be a number in the range of >=100 and <=10000."
- To change the maximum search limit:
  1. Log in to the Management Portal.
  2. Click Deployment.
  3. Select Deployments.
  4. Click the Three Dots icon (Browse) on the right side of the screen. Then, select Reconfigure.
  5. Select Fusion.
  6. Scroll down to the Search Configuration section.
  7. Change the value in the Maximum Number of Searches field to any number between 100-10,000.

The higher the number of searches, the more storage space will be consumed.

Event Integrity Auto-tuned Parameter Settings: By default, Event Integrity Auto-tuned parameter setting is enabled. Auto-tuning shares resources with other tasks. This might cause insufficient resources for other tasks that start after Event Integrity Check begins running. To have more control over system resources, disable auto-tuning and reduce values for Task Count and Chunk Size so that future tasks can freely run.

ArcSight Database

If you deployed the ArcSight Database and you configure SmartConnectors to use the CEF format when you send events to the Transformation Hub, in the Transformation Hub tab, ensure the # of CEF-to-Avro Stream Processor instances to start is set to at least 1 or what is specified in Technical Requirements for ArcSight Platform for your workload.

On the Database Configuration, ensure that you set these configuration settings for your environment:

Enable Database
Use SSL for Database Connections

Leave SSL disabled (the default) but it must be enabled in a later step.
Database Host

The host list of the database node's IP, that is node1-IP, node2-IP,..., upto nodeN-IP.

Database Application Admin User Name
Database Application Admin User Password
Search User Name
Search User Password
Database Certificate(s)

Leave the Database Certificate(s) field empty.

Database Host Name(s)

Intelligence

If you deployed Intelligence, on the Intelligence tab, ensure you set these configuration settings for your environment:

HDFS NameNode (interset-hdfs-namenode)
Elasticsearch Index Replicas Count (interset-elasticsearch-index-replicas-count)

Ensure you change default passwords to have a unique value in your environment.
H2 Password (interset-h2-password)

You can set this password only at the time of deployment
KeyStore Password
Elasticsearch Password (interset-elasticsearch-password)
If you have a non-collocated database cluster and Enable Secure Data Transfer with HDFS Cluster is enabled, perform the following steps:
1. Execute the following command in the master node:
```
/opt/arcsight/kubernetes/scripts/cdf-updateRE.sh > /tmp/re_ca.cert.pem
```
2. Execute the following commands in each database node:
```
scp root@<master_node_FQDN>:/tmp/re_ca.cert.pem /etc/pki/ca-trust/source/anchors/
```
```
update-ca-trust
```
3. Execute the following command to verify that there is a trust relationship with the CA from each database node:
```
curl https://<WORKER_RUNNING_HDFS_NAMENODE>:30071
```
  You should not encounter any certificate errors after executing the above command.
The Enable Secure Data Transfer with HDFS Cluster field is enabled by default to encrypt communication between the HDFS cluster and the database. However, this increases the run time of the analytics jobs.

Consider the following:

If the topic name specified for the Avro Event Topic field is not the default topic, then use Transformation Hub's Avro routing rules using ArcMC 2.96 or later to filter Avro events from the default topic. Create a routing rule with the source topic as mf-event-avro-enriched and destination topic as the topic name you have provided in the Avro Event Topic field. For more information, see Creating a Route.
For Analytics Configuration-Spark, set the values based on the data load. For information about the values for Spark, see System Hardware Sizing and Tuning Guidelines in the Technical Requirements for ArcSight Platform for your workload.
For the Data Identifiers to Identify Machine Users field, if you need to consider only human users for licensing, ensure that you provide appropriate values to identify and filter out the machine users from licensing. For more information, contact Micro Focus Customer Support.

If you are specifying details under the Hadoop File System (HDFS) Security section, consider the following:

If you are enabling Kerberos Authentication, then, before selecting kerberos in Enable Authentication with HDFS Cluster, ensure you configure the Kerberos Authentication. For more information, see Enabling and Configuring Kerberos Authentication.
The Kerberos details that you provide in Kerberos Domain Controller Server, Kerberos Domain Controller Admin Server, Kerberos Domain Controller Domain, and Default Kerberos Domain Controller Realm will be considered only if you select kerberos in Enable Authentication with HDFS Cluster. They are not valid if you select simple.