5.13.1 Configuring Clustering

To configure your system for high availability, and minimal downtime, we recommend that you create a cluster of at least three MSS server installations. If one of the clustered servers goes down, another server can seamlessly provide the replicated data and reduce the chance of downtime.

NOTE:Check that all ports for each system that will join the cluster are not being blocked by any firewall configuration.

To create, configure, and adjust a cluster:

Establish the cluster

After installation, an MSS server is not aware of any other MSS servers. Use Clustering to create a cluster of connected MSS servers that replicate data to all of the clustered servers.

First decide which MSS servers you want to include in the cluster.

Identify the MSS server that contains the configuration settings that you want all of the clustered servers to inherit. Consider this server to be the primary one.

CAUTION:The selection is important because when another server joins the cluster, it loses all of its current configuration settings before acquiring the cluster’s settings (replicated from the primary server).
Log on to the MSS server you identified as the primary one.
From the Administrative Console, click Configure Settings - Clustering.

In the list of nodes, this MSS server is designated as MASTER in the cluster (MASTER = true).
Select (or confirm) the Database Network Adapter.

The drop-down list shows each network adapter and its associated IP address. All other database nodes in the cluster must be able to connect to the selected IP address.

If your system has a single network adapter, that adapter is automatically selected.
To add other MSS servers, you need to log on to each of those servers and JOIN them to the cluster. They are not added from the MASTER server interface.

NOTE:This approach is different from configuring Replication in earlier releases of MSS.

Add an MSS server to the cluster

Now that a cluster is established, other MSS servers can be added one at a time.

Log in to one of the MSS servers that you want to include in the cluster.
From the Administrative Console, click Configure Settings - Clustering.
Click JOIN CLUSTER to add this server to the cluster.
Enter the address (including port if necessary) of the remote server you want to cluster with.

If this server is joining a collection of servers that are already clustered, you can enter the address of any of those clustered servers. Click Next.

NOTE:When you add a server to a cluster, the server being added will lose all of its current configuration settings. After the JOIN process, the added server inherits the configuration settings of the specified server, which is the same for all servers that are already in the cluster.
Verify that the certificate presented matches the server you just entered. Click Next.
Enter the username and password of a user with administrative rights on the server entered in step 5. Click Next.
The Clustering Progress dialog displays updates as each step is completed.

The full clustering process can take some time to complete. During this time, you will not be able to interact with the Administrative Console. See About the Clustering Process.
Once the clustering process is complete, click OK to dismiss the dialog.

All servers in the cluster are displayed in the Management Server Nodes table, and the one you are logged in to is listed first (at the top).
NOTE:Each server in the cluster is added as a node. Data is replicated to all of the nodes.
- One server is designated as the MASTER (true) in the Management Server Nodes table. The MASTER ensures that all nodes in the cluster receive all configuration changes, regardless of which node the change was initiated on.
- Any changes made to the certificate stores (+ IMPORT or DELETE certificates) will be replicated to the other MSS servers in the cluster. You do not need to repeat the process on each MSS server.
To add another server to this cluster, repeat steps 1 - 9.

About the Clustering Process

During the clustering process, certain services provided by the servers in the cluster are automatically restarted. As a result, some MSS functionality is temporarily interrupted during the clustering process.
Clustered MSS servers and services communicate over a secure TLS channel, which requires server certificates to be exchanged among all the servers in the cluster. This certificate exchange is handled automatically during the clustering process.
If an error occurs during the clustering process, the progress dialog will note the error. Refer to Troubleshooting Clustering for assistance.

When the cluster is configured, you can adjust the settings to monitor, promote, or remove a server in the cluster.

Once the cluster is established, the servers will synchronize with replicated data.

Monitor Cluster Servers’ Status

For troubleshooting and general information, you may want to monitor the status of the clustered servers. To view the servers’ status:

Log in to the Administrative Console for any server in the existing cluster.
Click Configure Settings - Clustering.
The Management Server Nodes table displays all of the servers in the cluster. Note that the server you are logged in to is listed first.

The current status for each server is specified in three columns:
- Server Status
  
  UP indicates that the management server instance itself is running and pingable. The UP link opens the Administrative Console login screen for that server in a new browser tab. (The server you are currently logged into does not provide an UP link).
  
  DOWN indicates the server is not currently running.
- Service Registry Status
  
  UP indicates that the service registry process for that server is running and pingable. The UP link opens the dashboard screen for that service registry instance in a new browser tab.
  
  DOWN indicates the service process for that server is not currently running.
- Database Status
  
  UP indicates that the database node associated with that server is currently running. The UP link opens the database cluster information in JSON format in a new browser tab. The database cluster information includes some general information about each database node as well as more specific information about that particular database node.
  
  DOWN indicates that the database node is not currently running.
NOTE:If the database node for a certain server is not currently running, you will not be able to view the Clustering panel in the Administrative Console on that server.

If you are viewing the Clustering panel, and the database node associated with that server goes DOWN, you will no longer be able to interact with the Clustering UI, but monitoring will continue. In either case, refer to Troubleshooting Clustering for assistance.

Remove an MSS server from a cluster

At times, you may need or want to remove an MSS server instance from a cluster, such as when a server is DOWN.

Note: You cannot remove the designated MASTER server from the cluster. If you need to remove the current MASTER server, you must first promote a different server to MASTER, and then remove the previous MASTER server. (See Promote an MSS server to MASTER.)

To remove an MSS server from a cluster:

Log into the Administrative Console for any server in the existing cluster.
Click Configure Settings - Clustering.
In the Management Server Nodes table, select the server to be removed.
Click REMOVE (above the Management Server Nodes table).
Click OK in the confirmation dialog.
A progress screen displays updates as each step of the removal process is completed.

The removal process can take some time to complete. During this time, you will not be able to interaction with the Administrative Console.

When a node is removed, its certificates are removed from the cluster.
Once the removal process is complete, click OK to dismiss the dialog.

The Management Server Nodes table reflects the server removal. If you are currently logged into the Administrative Console of the server that was removed, the Management Server Nodes table shows only a single entry of that server.

If you are logged into the Administrative Console of a different server in the cluster, the Management Server Nodes table no longer shows the server that was removed.

NOTE:

During the removal process, certain services provided by servers in the cluster are automatically restarted. As a result, some aspects of MSS functionality is temporarily interrupted during the removal process.
A server designated as MASTER in the Management Server Nodes table cannot be removed from a cluster. You must first promote another server in the cluster to be the MASTER server. See Promote an MSS server to MASTER.
Server certificate cleanup is handled automatically during the removal process, which breaks the trust relationship between the removed server and the rest of the servers in the cluster.
A server with a Server Status of DOWN in the Management Server Nodes table can be removed from a cluster, but be aware that this server is NO LONGER EXPECTED TO BE USED after removal.

For best results ALL servers in the cluster should be in the UP state when performing a server removal.

Promote an MSS server to MASTER

The MSS cluster configuration requires that one server in the cluster be designated as MASTER. The designated server appears in the Management Server Nodes table with a true entry in the MASTER column.

The role of the MASTER is to ensure all of the clustered nodes receive all configuration changes, no matter which node initiated the change. Any of the clustered servers could be the MASTER.

One reason to promote a server to MASTER is to be able to remove the currently-designated MASTER. For instance, if the current MASTER server goes DOWN and you choose to remove it from the cluster, you must first promote a different server to MASTER to enable the REMOVE button.

To change which server is designated as MASTER in an MSS cluster:

Log into the Administrative Console for any server in the existing cluster.
Click Configure Settings - Clustering.
In the Management Server Nodes table, select the server you wish to promote to MASTER.
Click PROMOTE (above the table).
Click OK in the confirmation dialog.
The selected server becomes the MASTER, identified with the true entry in the Management Server Nodes table. The previous master server simply becomes a node in the cluster, and could be removed, if needed.

NOTE:A DOWN server cannot be promoted to MASTER. Either start the server before promoting it, or select a different server in the cluster to promote to MASTER.

Change the database network adapter

At times, you may need to change the database network adapter used by a given server in an MSS cluster.

To change the database network adapter:

Log into the Administrative Console of the server whose database network adapter you want to change.
Select Configure Settings - Clustering.
Select the desired Database Network Adapter (with its associated IP address) from the drop-down list.
Click APPLY (at the bottom of the screen).

The database node is restarted with the desired network adapter.

NOTE:If the database node is not currently running, you will not be able to change the database network adapter in the Administrative Console’s Clustering view. Refer to Troubleshooting Clustering for assistance.