Configuring Management Servers for Failover

When you set up a management server failover configuration if one of the management servers fails or is offline, another management server in the cluster can provide services. Failover configuration provides fault tolerance for production environments.

Configuring Failover

You establish failover support by installing multiple management servers, adding them to a single management cluster, and configuring your DNS (Domain Name System) server to associate all management servers in the cluster with one symbolic name.

Things to remember

Management server and session server components have separate failover mechanisms. Session server failover and load distribution is addressed separately in Technical Note 10108.
In a multi-server failover environment, all servers (running management server or session server components) must be able to communicate with each other, and the clients (running connector API) must be able to communicate with all servers.
Each management server (and each session server) can belong to only one cluster or installation
Management Server failover is based on standard IP name resolution functionality, which allows an alias name to be mapped to multiple IP addresses.

How does it work?

If you have 3 different management servers installed on 3 different machines called machine1, machine2, and machine3 and have added them to a single cluster, then you can setup a DNS entry with the symbolic name my-management-cluster and in that DNS entry you can add machine1, machine2, and machine3.

When a client wants to connect to the management cluster, instead of providing the address of a management server in the cluster (for example, machine1), the client provides the DNS symbolic name, my-management-cluster. Because the DNS entry contains more than one machine, the DNS service provides the list addresses configured for the entry. If a management server is not running, then the client can connect to another server configured for the DNS entry, and failover support is achieved.

You can also configure the DNS service to achieve load balancing support. Since data replication automatically occurs between all servers in a management cluster the client will have the same experience regardless of the management service to which they are connected.

Setting Up Management Servers for Failover Support

Management server clusters can be set up during installation or anytime using the Administrative Console Management Server Explorer.

During installation, after you install Host Integrator and the management server component on the first server, select Join an existing installation and use the host name or IP address of the first server when installing on subsequent servers.
Create a single DNS alias (common name) for all the IP addresses of management servers in the installation environment. Each management server IP address must be assigned to the same common name.
- Recommended method: For ease of maintenance, configure you DNS server to setup failover support. The DNS server can return results in round-robin or random order for load distribution, as the default management server configuration (ManagementServer/conf/container.conf file) has DNS caching disabled (-Dsun.net.inetaddr.ttl=0).
- Alternative method: If you cannot configure your DNS server, you can edit the hosts file on each system with Host Integrator server or client components installed (including connectors used by client applications).
Example:
```
# Hosts file on system "workhorse01"
# Verastream Host Integrator production environment in Seattle

# First entry is the unique name for the local system
10.0.0.1 workhorse01

# Common name for the management server cluster
10.0.0.1 vhi-prod-sea
10.0.0.2 vhi-prod-sea
10.0.0.3 vhi-prod-sea
```
If you use this method, update the hosts file on all systems in the environment. All servers (running management server or session server components) must be able to communicate with each other, and the clients (running connector API) must be able to communicate with all servers.

The local unique system name is listed separately first to avoid problems with reverse DNS lookups on some platforms. This is followed by lines for the cluster common name (one line for each system running the Management Server service).

For systems that have multiple network interfaces, all IP addresses should be listed. On Windows systems, the hosts file is typically located in the C:\Windows\System32\drivers\etc folder. On UNIX/Linux systems, the hosts file is located in the /etc directory.

Networking protocols mandate that the hostname contain only ASCII letters a through z (case insensitive), digits 0 through 9, and the hyphen character (-). Comments beginning with the # character (through the end of the line) are ignored.
In Administrative Console, it is a good idea to change the name of the management cluster (Perspective > Management > Servers > Management Cluster > Properties). The default cluster name is the system host name where management server was first installed, but you can change it to the cluster DNS alias for your installation environment. The cluster name displays in the Administrative Console status bar (lower right) when connected.
To achieve failover capability, enter the cluster DNS alias name whenever you are required to provide a management server address:
- In your client application, when calling ConnectToSessionViaDomain or ConnectToModelViaDomain method in the connector API.
- In your Web Builder project properties, when "Connect to model via domain" or "Connect to session pool via domain" is selected for the model connection.
- In the embedded web service functionality, if you are configuring it to use a load distribution domain (see Technical Note 10092).
- In Administrative Console, when prompted to connect.
However, when you deploy models (using either the activatemodel and deactivatemodel commands or Design Tool), use the specific individual session server system names (not the cluster common alias name).

What happens at runtime?

When establishing a connection to the management server alias name, IP name resolution returns the list of IP addresses. An attempt is made to contact the first address in the list. If no response is received, the next address is tried, and so forth.

When deploying models, the session server contacts the management server for authentication and authorization. If you see the following errors when deploying a model, you may have an incorrect configuration in step 2 above.

[VHI 3852] Deployment of model failed: Cannot establish management session<br> [VHI 3852] Deployment of model failed: Token binding is invalid

More information