33.4 Settings in Elasticsearch for Secure Cluster Communication

Sentinel 8.4.0.0 and above comes with out of the box enhanced security features, for which some post-install/upgrade configurations are needed. Starting from 8.4.0.0, Sentinel communicates with Elasticsearch in a secure manner (over SSL) and has the X-Pack plug-in of Elasticsearch bundled in it by default. This will give a Sentinel administrator the ability to configure all node-to-node Elasticsearch communications securely and over SSL. This will open up possibilities of data getting stored across the Elasticsearch nodes over geographies and still enabling data to be passed around and viewed securely by a Sentinel server. Using this feature, a user can now join all his Elasticsearch clusters spread across the globe and still will be able to view and accumulate the results securely from one single search console of Sentinel.

IMPORTANT:For the upgrade process to finish, executing the below steps are mandatory. The details on this page are applicable only if the Event Visualization feature is enabled before the upgrade to versions Sentinel 8.4 or Sentinel 8.5 from an older version of Sentinel.

If you are upgrading from Sentinel 8.4 to Sentinel 8.5 then, the below steps should not be executed.

Without performing the steps below, the upgrade to Sentinel 8.4.0.0 or above from an older version will be incomplete and will have the following problems:

Elasticsearch will not start automatically.
If Elasticsearch is not manually restarted, alerts and events present in it will not reflect correctly while searching for it in Sentinel.

Enabling Secure Communication between Sentinel Server and Pre-bundled Elasticsearch when there is no External Elasticsearch Cluster Setup

This section is needed for cases where you do not have an external Elasticsearch cluster associated with Sentinel. In such a case, you only need to enable secure communication between Sentinel and the pre-bundled Elasticsearch.

Stop the internal Elasticsearch service using below command:
```
rcsentinel stopES
```
Switch to the novell user:
```
su novell
```
Perform steps 3 and 4, if the java version is 292. To find the java version at the OS level, run java -version at the command prompt.
(Conditional) Set the JAVA_HOME to Sentinel JDK bundled:
```
JAVA_HOME=/opt/novell/sentinel/jdk
```
(Conditional) Set the PATH for java to Sentinel JDK location:
```
PATH=$JAVA_HOME/bin:$PATH
```
Generate a Certificate Authority (CA) for your cluster in the Sentinel node. Run the following command in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-certutil ca
```
You are prompted for the file name and a password of the CA certificate. Here the default file name is elastic-stack-ca.p12.
Generate the certificates and private keys for the pre-bundled Elasticsearch node of Sentinel. For this, run the following command in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-certutil cert --ca <CA certificate filename>.p12 --out config/certs/node-1.p12
```
You are prompted to enter the password for your CA certificate. You are also prompted to create a password for the generated certificate.
Add the following settings in the <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch/config/elasticsearch.yml file in the Sentinel node:
- xpack.security.transport.ssl.enabled: true
- xpack.security.transport.ssl.keystore.path: certs/node-1.p12
- xpack.security.transport.ssl.truststore.path: certs/node-1.p12
- xpack.security.transport.ssl.verification_mode: certificate
Store the password of the truststore and keystore certificate file generated above in the Elasticsearch keystore. For this, run the following commands in the Elasticsearch home directory: <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
```
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
```
Start the Elasticsearch service using the below command:
```
rcsentinel startES
```

Enabling Secure Communication between External Elasticsearch Nodes as well as between Sentinel and Elasticsearch Cluster if there is an External Elasticsearch Cluster Setup

The latest release of Sentinel enables secure communication between the Sentinel server and external Elasticsearch cluster as well as between different nodes of the Elasticsearch cluster. This section explains the steps on how to enable these secure settings for cases where you have an external Elasticsearch cluster connected to the Sentinel server.

Steps to be followed for securing intra cluster communication between Elasticsearch nodes:
1. Stop the Elasticsearch on all the nodes.
2. Switch to the novell user:
```
su novell
```
  Perform steps 3 and 4, if the java version is 292. To find the java version at the OS level, run java -version at the command prompt.
3. (Conditional) Set the JAVA_HOME to Sentinel JDK bundled:
```
JAVA_HOME=/opt/novell/sentinel/jdk
```
4. (Conditional) Set the PATH for java to Sentinel JDK location:
```
PATH=$JAVA_HOME/bin:$PATH
```
5. Generate a Certificate Authority (CA) for your cluster in the Sentinel node. Run the following command in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-certutil ca
```
  You are prompted for the file name and a password of the CA certificate. Here the default file name is elastic-stack-ca.p12.
6. Generate the certificates and private keys for the pre-bundled Elasticsearch node of Sentinel. For this, run the following command in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-certutil cert --ca <CA certificate filename>.p12 --out config/certs/node-1.p12
```
  You are prompted to enter the password for your CA certificate. You are also prompted to create a password for the generated certificate.
7. Add the following settings in the <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch/config/elasticsearch.yml file in the Sentinel node:
  - xpack.security.transport.ssl.enabled: true
  - xpack.security.transport.ssl.keystore.path: certs/node-1.p12
  - xpack.security.transport.ssl.truststore.path: certs/node-1.p12
  - xpack.security.transport.ssl.verification_mode: certificate
8. Store the password of the truststore and keystore certificate file generated above in the Elasticsearch keystore. For this, run the following commands in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
```
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
```
9. Generate the certificates for all the external Elasticsearch nodes in the cluster. You can first create all the external Elasticsearch certificates in the Sentinel node itself and can then copy them to the respective Elasticsearch nodes. For this, first run the following command in the Elasticsearch home directory <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch of the Sentinel:
```
./bin/elasticsearch-certutil cert --ca <CA certificate filename>.p12 --out config/certs/newNode.p12
```
  You are prompted to enter the password for your CA certificate. You are also prompted to create a password for the generated certificate.
10. Copy the certificates to the respective external Elasticsearch nodes. For example, copy the newNode.p12 file into the /etc/elasticsearch/certs/ directory of the newNode of the external Elasticsearch cluster. Provide read-write permissions to the certificates on the new machines using the chmod command.
  
  NOTE:If the certs directory is not present, you need to create the same.
11. After generating and copying the certificates to all the external Elasticsearch nodes, add the following settings in the /etc/elasticsearch/elasticsearch.yml file of all the external Elasticsearch nodes:
  - xpack.security.enabled: true
  - xpack.security.transport.ssl.enabled: true
  - xpack.security.transport.ssl.keystore.path: certs/newNode.p12
  - xpack.security.transport.ssl.truststore.path: certs/newNode.p12
  - xpack.security.transport.ssl.verification_mode: certificate
12. On each of the external Elasticsearch nodes, store the password for the generated keystore and truststore certificate file in the Elasticsearch keystore. For this, run the following commands in the Elasticsearch home directory /usr/share/elasticsearch of all the external Elasticsearch nodes:
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
```
```
./bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
```
Steps to be followed for securing Sentinel to Elasticsearch cluster communications:
1. Switch to the novell user:
```
su novell
```
2. Run the following command, to generate a http certificate for an external Elasticsearch node from the Sentinel machine:
```
<sentinel_installation_path>/opt/novell/sentinel/bin/javacert.sh --generateES <provide path where the http certificate should be generated, example /opt/http.pks> <http certificate password> <keyalias>
```
3. Copy the http certificate to the Elasticsearch node. For example, copy the http.pks file into the ES_PATH_CONF/certs/ directory on the Elasticsearch node. Provide read-write permissions to the certificates on the new machines.
  
  NOTE:If the certs directory is not present, you need to create the same.
4. Add the following settings in the ES_PATH_CONF/elasticsearch.yml file in all the external Elasticsearch nodes:
  - xpack.security.http.ssl.enabled: true
  - xpack.security.http.ssl.keystore.path: certs/http.pks
5. Run the following command in the Elasticsearch home directory /usr/share/elasticsearch of all the external Elasticsearch nodes to save the password of the http certificate to the Elasticsearch keystore:
```
./bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
```
6. Start Elasticsearch service in each of the external Elasticsearch nodes:
```
/etc/init.d/elasticsearch start
```
(Conditional) If you are in FIPS mode, after performing the above two steps, you need to perform the below steps:
1. Add the internal Elasticsearch http certificate generated during the Sentinel install to the Sentinel server's FIPS keystore using the command:
```
./convert_to_fips.sh -i <sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch/config/http.pks
```
2. After the above step, there will be a prompt to restart Sentinel. Select No.
3. Copy the http certificates of all the external Elasticsearch nodes generated in Step 2 and add it to the FIPS keystore of the Sentinel server using the below command:
```
./convert_to_fips.sh -i <location of the copied http certificate>/<name of the certificate>
```
4. Make sure that all the http certificates of the external Elasticsearch nodes are present in the Sentinel server's FIPS keystore by running the command:
```
certutil -L -d sql:<sentinel_installation_path>/etc/opt/novell/sentinel/3rdparty/nss
```
5. Copy the internal Elasticsearch http certificate (<sentinel_installation_path>/opt/novell/sentinel/3rdparty/elasticsearch/config/http.pks in the Sentinel server) generated during the Sentinel install and add it to all the Remote Collector Manager’s (RCM) FIPS keystore using the command:
```
./convert_to_fips.sh -i <location of the copied http certificate>/http.pks
```
6. After the above step, there will be a prompt to restart Sentinel. Select No.
7. Copy the http certificates of all the external Elasticsearch nodes generated in Step 2 and add it to the FIPS keystore of all the RCMs using the below command:
```
./convert_to_fips.sh -i <location of the copied http certificate>/<name of the certificate>
```
8. Make sure that all the http certificates of the external Elasticsearch nodes are present in the RCM's FIPS keystore by running the following command:
```
certutil -L -d sql:<rcm_installation_path>/etc/opt/novell/sentinel/3rdparty/nss
```
Restart the Sentinel and all the RCMs:
```
rcsentinel restart
```