8.2 Planning for OES Cluster Services

Cluster Installation Administrator

Typically, a tree administrator user installs and sets up the first cluster in a tree, which allows the schema to be extended. However, the tree administrator can extend the schema separately, and then set up the necessary permissions for a container administrator to install and configure the cluster.

NCS Proxy User

During the cluster configuration, you must specify an NCS Proxy User. This is the user name and password that Cluster Services uses when the cluster management tools exchange information with eDirectory.

Cluster Services supports the OES Common Proxy User enablement feature of eDirectory. The proxy user is represented in eDirectory as a User object named OESCommonProxy_<server_name>.<context>. If the OES Common Proxy user is enabled in eDirectory when you configure a node for the cluster, the default NCS Proxy User is set to use the server’s OES Common Proxy User. You can alternatively specify the LDAP Admin user or another administrator user.

The specified NCS Proxy User for the node is automatically assigned as a member in the <cluster_name>_MGT_GRP.<context> group that resides in the Cluster object container, such as clus1_MGT_GRP.ncs.novell. The group accommodates the server-specific NCS Proxy Users that you assign when you configure each node for the cluster. Members are added to the group as you configure each node for a cluster. Each member of the group has the necessary rights for configuring the cluster and cluster resources and for exchanging information with eDirectory.

For example, Figure 8-2 shows that an OES Common Proxy User has been assigned as the NCS Proxy User for each node in a four-node cluster. The nodes are named avalon, camelot, excalibur, and cornwall. The context is novell.

Figure 8-2 Members of the NCS Management Group for a Cluster

Consider the following caveats for the three proxy user options:

Cluster Administrator or Administrator-Equivalent User

After the install, you can add other users (such as the tree administrator) as administrator equivalent accounts for the cluster by configuring the following for the user account:

Open Enterprise Server 23.4

Cluster Services for Linux supports OES 23.4. OES Cluster Services is one of the OES Services patterns.

We recommend having uniform nodes in the cluster. The same release version of OES must be installed and running on each node in the cluster.

Mixed-mode clusters with different operating system platforms are supported during rolling cluster upgrades or conversions for the following scenarios:

Cluster Services

Cluster Services is required for creating and managing clusters and shared resources on your OES servers. OES Cluster Services is one of the OES Services patterns on OES 23.4.

NetIQ eDirectory

NetIQ eDirectory is required for managing the Cluster object and Cluster Node objects for Cluster Services. eDirectory must be installed and running in the same tree where you create the cluster. eDirectory can be installed on any node in the cluster, on a separate server, or in a separate cluster. You can install an eDirectory master replica or replica in the cluster, but it is not required to do so for Cluster Services.

For information about using eDirectory, see NetIQ eDirectory Administration Guide.

In addition, ensure that your eDirectory configuration meets the following requirements:

Cluster Object Container

We recommend that the Cluster object and all of its member Server objects and Storage objects be located in the same OU context. Multiple Cluster objects can co-exist in the same eDirectory container. In iManager, use Directory Administration > Create Object to create a container for the cluster before you configure the cluster.

For example, Figure 8-3 shows an example where all clusters are configured in the ncs organizational unit. Within the container, each cluster is in its own organizational unit, and the Server objects for the nodes are in the same container as the Cluster object:

Figure 8-3 Same Container for Cluster Object and Server Objects

If the servers in the cluster are in separate eDirectory containers, the user that administers the cluster must have rights to the cluster server containers and to the containers where any cluster-enabled pool objects are stored. You can do this by adding trustee assignments for the cluster administrator to a parent container of the containers where the cluster server objects reside. For more information, see eDirectory Rights in the NetIQ eDirectory Administration Guide.

Renaming a pool involves changing information in the Pool object in eDirectory. If Server objects for the cluster nodes are in different containers, you must ensure that the shared pool is active on a cluster node that has its NCP server object in the same context as the Pool object of the pool you are going to rename. For information about renaming a shared pool, see Renaming a Clustered NSS Pool.

Cluster Objects Stored in eDirectory

Table 8-1 shows the cluster objects that are automatically created and stored in eDirectory under the Cluster object () after you create a cluster:

Table 8-1 Cluster Objects

Icon	eDirectory Object
	Master_IP_Address_Resource
	Cluster Node object (servername)
	Resource Template objects. There are 11 default templates: AV_Template DHCP_Template DNS_Template Generic_FS_Template Generic_IP_Service iPrint_Template MySQL_Template Samba_Template Xen_Template XenLive_Template

For example, Figure 8-4 shows the 13 default eDirectory objects that are created in the Cluster container as viewed from the Tree view in iManager:

Figure 8-4 Tree View of the Default eDirectory Objects in the Cluster

Table 8-2 shows the cluster objects that are added to eDirectory when you add nodes or create cluster resources:

Table 8-2 Cluster Resource Objects

Icon	eDirectory Object
	Cluster Node object (servername)
	NSS Pool Resource object (poolname_SERVER)
	Resource object

Table 8-3 shows the cluster objects that are added to eDirectory when you add nodes or create cluster resources in a Business Continuity Cluster, which is made up of OES Cluster Services clusters:

Table 8-3 BCC Cluster Resource Objects

Icon	eDirectory Object
	BCC NSS Pool Resource object
	BCC Resource Template object
	BCC Resource object

SLP

SLP (Service Location Protocol) is a required component for Cluster Services on Linux when you are using NCP to access file systems on cluster resources. NCP requires SLP for the ncpcon bind and ncpcon unbind commands in the cluster load and unload scripts. For example, NCP is needed for NSS volumes and for NCP volumes on Linux POSIX file systems.

SLP is not automatically installed when you select Cluster Services. SLP is installed as part of the eDirectory configuration during the OES installation. You can enable and configure SLP on the eDirectory Configuration - NTP & SLP page. For information, see Specifying SLP Configuration Options in the OES 23.4: Installation Guide.

When the SLP daemon (slpd) is not installed and running on a cluster node, any cluster resource that contains the ncpcon bind command goes comatose when it is migrated or failed over to the node because the bind cannot be executed without SLP.

The SLP daemon (slpd) must also be installed and running on all nodes in the cluster when you manage the cluster or cluster resources.

NCP Server re-registers cluster resource virtual NCP servers with SLP based on the setting for the eDirectory advertise-life-time (n4u.nds.advertise-life-time) parameter. The parameter is set by default to 3600 seconds (1 hour) and has a valid range of 1 to 65535 seconds.

You can use the ndsconfig set command to set the n4u.nds.advertise-life-time parameter. To reset the parameter in a cluster, perform the following tasks on each node of the cluster:

OpenSLP stores the registration information in cache. You can configure the SLP Directory Agents to preserve a copy of the database when the SLP daemon (slpd) is stopped or restarted. This allows SLP to know about registrations immediately when it starts.

For more information about configuring and managing SLP, see Configuring OpenSLP for eDirectory in the NetIQ eDirectory Administration Guide.

iManager 3.2.6

iManager is required for configuring and managing clusters on OES.

iManager must be installed on at least one computer in the same tree as the cluster. It can be installed in the cluster or not in the cluster. For information about using iManager, see the iManager documentation website.

For SFCB (Small Footprint CIM Broker) and CIMOM requirements, see SFCB and CIMOM.

For browser configuration requirements, see Web Browser.

Clusters Plug-in for iManager

The Clusters plug-in for iManager provides the Clusters role where you can manage clusters and cluster resources with OES Cluster Services. The plug-in can be used on all operating systems supported by iManager and iManager Workstation.

The following components must be installed in iManager:

If iManager is also installed on the server, these files are automatically installed in iManager when you install OES Cluster Services.

The Clusters plug-in also provides an integrated management interface for OES Business Continuity Clustering (BCC). The additional interface is present only if BCC is installed on the server. See the following table for information about the versions of BCC that are supported. BCC is sold separately from OES. For purchasing information, see the BCC product page.

Storage-Related Plug-Ins for iManager

In OES 11 and later, the following storage-related plug-ins for iManager share code in common in the storagemgmt.rpm file:

These additional plug-ins are needed when working with the NSS file system. Ensure that you include the common storagemgmt.rpm plug-in module when installing any of these storage-related plug-ins.

The plug-in files are included on the installation disk. The latest storage-related plug-ins can be downloaded as a single zipped download file from the OpenText Downloads website. For information about installing plug-ins in iManager, see Downloading and Installing Plug-in Modules in the NetIQ iManager Administration Guide.

For information about working with storage-related plug-ins for iManager, see Understanding Storage-Related Plug-Ins in the OES 23.4: NSS File System Administration Guide for Linux.

SFCB and CIMOM

The Small Footprint CIM Broker (SFCB) replaces OpenWBEM for CIMOM activities in OES 11 and later. SFCB provides the default CIMOM and CIM clients. When you install any OES components that depend on WBEM, SFCB and all of its corresponding packages are installed with the components. For more information, see Section N.0, Small Footprint CIM Broker (SFCB).

Port 5989 is the default setting for Secure HTTP (HTTPS) communications. If you are using a firewall, the port must be opened for CIMOM communications. Ensure that the CIMOM broker daemon is listening on port 5989. Log in as the root user on the cluster master node, open a terminal console, then enter the following at the command prompt:

netstat -an |grep -i5989

The Clusters plug-in (and all other storage-related plug-ins) for iManager require CIMOM connections for tasks that transmit sensitive information (such as a user name and password) between iManager and the _admin volume on the OES server that you are managing. Typically, CIMOM is running, so this should be the normal condition when using the server. CIMOM connections use Secure HTTP (HTTPS) for transferring data, and this ensures that sensitive data is not exposed.

You can use the following commands to start, stop, or restart SFCB:

For more information, see Web Based Enterprise Management using SFCB in the SUSE Linux Enterprise Server 12 Administration Guide.

OES Credential Store (OCS)

Cluster Services requires OES Credential Store to be installed and running on each node in the cluster.

To check whether the OCS is running properly, do the following:

Web Browser

For information about supported web browsers for iManager, see System Requirements for iManager Server in the NetIQ iManager Installation Guide.

The Clusters plug-in for iManager might not operate properly if the highest priority Language setting for your web browser is set to a language other than one of the supported languages in iManager. To view a list of supported languages and codes in iManager, select the Preferences tab, click Language. The language codes are Unicode (UTF-8) compliant.

To avoid display problems, in your web browser, select Tools > Options > Languages, and then set the first language preference in the list to a supported language. You must also ensure the Character Encoding setting for the browser is set to Unicode (UTF-8) or ISO 8859-1 (Western, Western European, West European).

NCP Server for Linux

NCP Server for Linux is required in order to create virtual server names (NCS:NCP Server objects) for cluster resources. This includes storage and service cluster resources. To install NCP Server, select the NCP Server and Dynamic Storage Technology option during the install.

NCP Server for Linux also allows you to provide authenticated access to data by using the OES Trustee model. The NCP Server component must be installed and running before you can cluster-enable the following storage resources:

NCP Server for Linux is not required when running Cluster Services on a Xen-based virtual machine (VM) host server (Dom0) for the purpose of cluster-enabling an LVM volume group that holds the configuration files for Xen-based VMs. Users do not directly access these VM files.

For information about configuring and managing NCP Server for Linux, see the OES 23.4: NCP Server for Linux Administration Guide.

For information about creating and cluster-enabling NCP volumes on Linux POSIX file systems, see Configuring NCP Volumes with OES Cluster Services in the OES 23.4: NCP Server for Linux Administration Guide.

Storage Services File System for Linux

Storage Services (NSS) file system on Linux provides the following capabilities used by OES Cluster Services:

The NSS pool configuration and NCS pool cluster resource configuration provide integrated configuration options for the following advertising protocols:

LVM Volume Groups and Linux POSIX File Systems

Cluster Services supports creating shared cluster resources on Linux Logical Volume Manager (LVM) volume groups. You can configure Linux POSIX file systems on the LVM volume group, such as Ext3, XFS, and Ext4. LVM and Linux POSIX file systems are automatically installed as part of the OES installation.

After the cluster is configured, you can create LVM volume group cluster resources as described in Configuring and Managing Cluster Resources for Shared LVM Volume Groups.

NCP Server is required if you want to create a virtual server name (NCS:NCP Server object) for the cluster resource. You can add an NCP volume (an NCP share) on the Linux POSIX file system to give users NCP access to the data. See NCP Server for Linux.

NCP Volumes on Linux POSIX File Systems

After you cluster-enable an LVM volume group, Cluster Services supports creating NCP volumes on the volume group’s Linux POSIX file systems. NCP Server is required. See NCP Server for Linux.

For information about creating and cluster-enabling NCP volumes, see Configuring NCP Volumes with OES Cluster Services in the OES 23.4: NCP Server for Linux Administration Guide.

Dynamic Storage Technology Shadow Volume Pairs

Cluster Services supports clustering for Dynamic Storage Technology (DST) shadow volume pairs on OES 11 and later. DST is installed automatically when you install NCP Server for Linux. To use cluster-enabled DST volume pairs, select the NCP Server and Dynamic Storage Technology option during the install.

For information about creating and cluster-enabling Dynamic Storage Technology volumes on Linux, see Configuring DST Shadow Volume Pairs with OES Cluster Services in the OES 23.4: Dynamic Storage Technology Administration Guide.

NCP File Access

Cluster Services requires NCP file access to be enabled for cluster-enabled NSS volumes, NCP volumes, and DST volumes, even if users do not access files via NCP. This is required to support access control via the OES Trustee model. See NCP Server for Linux.

CIFS

Cluster Services supports using CIFS as an advertising protocol for cluster-enabled NSS pools and volumes.

CIFS is not required to be installed when you install OES Cluster Services, but it must be installed and running in order for the CIFS Virtual Server Name field and the CIFS check box to be available. Select the check box to enable CIFS as an advertising protocol for the NSS pool cluster resource. A default CIFS Virtual Server Name is suggested, but you can modify it. The CIFS daemon should also be running before you bring resources online that have CIFS enabled.

To install CIFS, select the OES CIFS option from the OES Services list during the install. For information about configuring and managing the CIFS service, see the OES 23.4: OES CIFS for Linux Administration Guide.

Domain Services for Windows

Cluster Services supports using clusters in Domain Services for Windows (DSfW) contexts. If Domain Services for Windows is installed in the eDirectory tree, the nodes in a given cluster can be in the same or different DSfW subdomains. Port 1636 is used for DSfW communications. This port must be opened in the firewall.

For information using Domain Services for Windows, see the OES 23.4: Domain Services for Windows Administration Guide.

Shared Devices

Cluster Services supports the following shared disks:

Before configuring Cluster Services, the shared disk system must be properly set up and functional according to the manufacturer's instructions.

Prior to installation, verify that all the drives in your shared disk system are recognized by Linux by viewing a list of the devices on each server that you intend to add to your cluster. If any of the drives in the shared disk system do not show up in the list, consult the OES documentation or the shared disk system documentation for troubleshooting information.

Prepare the device for use in a cluster resource:

If this is a new cluster, connect the shared disk system to the first server so that the SBD cluster partition can be created during the Cluster Services install. See SBD Partitions.

SBD Partitions

If your cluster uses physically shared storage resources, you must create an SBD (split-brain detector) partition for the cluster. You can create an SBD partition in YaST as part of the first node setup, or by using the SBD Utility (sbdutil) before you add a second node to the cluster. Both the YaST new cluster setup and the SBD Utility (sbdutil) support mirroring the SBD partition.

An SBD must be created before you attempt to create storage objects like pools or volumes for file system cluster resources, and before you configure a second node in the cluster. NLVM and other NSS management tools need the SBD to detect if a node is a member of the cluster and to get exclusive locks on physically shared storage.

For information about how SBD partitions work and how to create an SBD partition for an existing cluster, see Creating or Deleting Cluster SBD Partitions.

Shared iSCSI Devices

If you are using iSCSI for shared disk system access, ensure that you have installed and configured the iSCSI initiators and targets (LUNs) and that they are working properly. The iSCSI target devices must be mounted on the server before the cluster resources are brought online.

Shared RAID Devices

We recommend that you use hardware RAID in the shared disk subsystem to add fault tolerance to the shared disk system.

Consider the following when using software RAIDs:

Path Failover Settings for Device Mapper Multipath

When you use Device Mapper Multipath (DM-MP) with Cluster Services, ensure that you set the path failover settings so that the paths fail when path I/O errors occur.

The default setting in DM-MP is to queue I/O if one or more HBA paths is lost. Cluster Services does not migrate resources from a node set to the Queue mode because of data corruption issues that can be caused by double mounts if the HBA path is recovered before a reboot.

Change the Retry setting in the /etc/modprobe.d/99-local.conf and /etc/multipath.conf files so that DM-MP works correctly with Cluster Services. See Modifying the Port Down Retry Setting in the modprobe.conf.local File and Modifying the Polling Interval, No Path Retry, and Failback Settings in the multipath.conf File.

Also consider changes as needed for the retry settings in the HBA BIOS. See Modifying the Port Down Retry and Link Down Retry Settings for an HBA BIOS.

Modifying the Port Down Retry Setting in the modprobe.conf.local File

The port_down_retry setting specifies the number of times to attempt to reconnect to a port if it is down when using multipath I/O in a cluster. Ensure that you have installed the latest HBA drivers from your HBA vendor. Refer to the HBA vendor’s documentation to understand the preferred settings for the device, then make any changes in the /etc/modprobe.conf.local file.

For example, for QLogic HBAs, ensure that you have installed the latest qla-driver. Ensure that you verify the vendor’s preferred settings before making the changes.

Modifying the Polling Interval, No Path Retry, and Failback Settings in the multipath.conf File

The goal of multipath I/O is to provide connectivity fault tolerance between the storage system and the server. When you configure multipath I/O for a stand-alone server, the retry setting protects the server operating system from receiving I/O errors as long as possible. It queues messages until a multipath failover occurs and provides a healthy connection. However, when connectivity errors occur for a cluster node, you want to report the I/O failure in order to trigger the resource failover instead of waiting for a multipath failover to be resolved. In cluster environments, you must modify the retry setting so that the cluster node receives an I/O error in relation to the cluster SBD verification process (recommended to be 50% of the heartbeat tolerance) if the connection is lost to the storage system. In addition, you want the multipath I/O fail back to be set to manual in order to avoid a ping-pong of resources because of path failures.

Use the guidance in the following sections to configure the polling interval, no path retry and failback settings in the /etc/multipath.conf file:

features "0"
no_path_retry fail 

failback "manual"

defaults
{
    polling_interval    5
#    no_path_retry        0
    user_friendly_names     yes
    features 0
}

devices {
  device {
        vendor "DGC"
        product ".*"
        product_blacklist "LUNZ"
        path_grouping_policy "group_by_prio"
        path_checker "emc_clariion"
        features "0"
        hardware_handler "1 emc"
        prio "emc"
        failback "manual"
        no_path_retry fail    #Set MP for failed I/O mode, any other non-zero values sets the HBAs for Blocked I/O mode
  }
} 

Modifying the Port Down Retry and Link Down Retry Settings for an HBA BIOS

In the HBA BIOS, the default settings for the Port Down Retry and Link Down Retry values are typically set too high for a cluster environment. For example, there might be a delay of more than 30 seconds after a fault occurs before I/O resumes on the remaining HBAs. Reduce the delay time for the HBA retry so that its timing is compatible with the other timeout settings in your cluster.

For example, you can change the Port Down Retry and Link Down Retry settings to 5 seconds in the QLogic HBA BIOS:

Port Down Retry=5
Link Down Retry=5