Introducing Databridge Client

Databridge is a combination of host and (optional) client software that provides automated replication of DMSII databases and flat files. All replications occur while the DMSII database is active. After the initial clone, Databridge updates the secondary database, copying only the DMSII data changes from the audit trail.

Choosing the Client Manager Service vs. the Command-Line Client

The Databridge Client provides two modes of operation. One mode lets you configure and run the Client from the Administrative Console, where the Client Manager service launches the Client and automates much of the replication process. The other mode uses a command prompt session (or terminal session in the case of UNIX) to run the command-line Client (dbutility). For a comparison of these modes of operation, see Comparing the Databridge Client Manager Service to Command-Line Operations.

While this guide can be used with either the command-line or service-controlled Client, it is intended primarily for command-line Client operations. For information specific to the service-controlled Client, see the following topics or refer to the Help in the Databridge Administrative Console:

Using the Administrative Console and the Service

The Administrative Console is an easy-to-use browser-based graphical interface that lets you access Clients on different platforms. You can view multiple data sources (defined by Databridge Server or Enterprise Server) and monitor all Client activity via onscreen messages and status updates.

The Administrative Console communicates directly with the Client Manager service, which starts Client runs in the background and completely automates Client operations. You decide when and how often Client runs are initiated by using the service scheduling settings in the Administrative Console. You can also use command files (shell scripts in UNIX) to interact with the service and automate processes. For more information, see Chapter 13, Automating Client Operations with the Service.

Enhanced Security

Because the service always starts the Client in the background, the Administrative Console is the only interface to service-initiated Client runs. Neither the service nor the background runs interact with the desktop or require that a user be signed on to the server. This makes service-initiated Client operations more secure than command-line operations and prevents the Client runs from being terminated, whether accidentally or maliciously. The service starts at system startup (by default), which ensures that replication can continue in the event of system failure without outside intervention.

In addition to launching Client runs, the service routes all log and informational messages to the Administrative Console and facilitates operator tasks using the DBClient and DBClntCfgServer programs. The first program, DBClient, performs DMSII processing and cloning tasks. The second program, DBClntCfgServer, handles Administrative Console requests that require access to the relational database (such as define, generateand reorganizecommands). The activity and output of these programs is displayed in the Administrative Console's console window.

The following diagram shows the Client architecture for the two types of clients: the command-line Client (dbutility) and the service-controlled client (DBClient and other components).

The Administrative Console can perform data source customizations by clicking on Settings > Customize after selecting the data source. The Customize command lets you easily customize your data sources without any knowledge of SQL or how your Client control tables work. Instead of hand-coding SQL user scripts, you can select options to configure your data sources and map DMSII data to your relational database tables.

To use the Customize command with existing Client configurations that employ user scripts, you must first upgrade your Client control tables using the dbscriptfixup utility. (See the Databridge Installation Guide.)

How Replication Works

The Databridge Client controls the replication process of DMSII databases and flat files. It initiates connections to Databridge Server on the host and maintains the state information necessary for resuming replication in case operations are interrupted. At the heart of the host-based Databridge system is the Databridge Engine---a system library that retrieves structural information and data from the DMSII database and passes it to Databridge Server. When you opt to use Enterprise Server with the Databridge Client instead, Enterprise Server takes over much of the functionality of the Databridge Engine and Databridge Server.

The Databridge Support Library, also installed on the host, provides filtering, formatting, and reformatting services for the Databridge Server. See Client vs. Host Filtering.

After the Databridge Server receives data from the Databridge Engine, it calls the Support Library to determine if the data should be replicated, and if so, it passes the data to the Support Library for formatting.

Replication involves three discrete phases, as described below. These three phases are tracked for each data set in the ds_mode column of the DATASETS control table as values 0, 1, and 2.

Client vs. Host Filtering

Use the following guidelines to determine when to use the host instead of the Databridge Client to perform filtering.

Filtering Columns

On the host side, you can filter columns by creating a filtering routine with the DBGenFormat utility. On the Databridge Client side, you can filter columns the same way you can filter data sets, which is to set the active column to 0 for the corresponding entry in the DMS_ITEMS Client control table.

The advantage of performing the filtering on the Databridge Client side is that you save on host resources. However, there are a few cases where you should consider filtering on the host side, as follows:

• If you plan to filter many columns, consider filtering on the host side to reduce TCP/IP traffic. The best way to determine this is to try the filtering both ways and see which gives you the best throughput.

• If you plan to filter columns with confidential or sensitive information, it is best to perform the filtering on the host.

Filtering Data Sets

You can filter data sets on the host side by using a logical database or by creating a filtering routine with the DBGenFormat program. On the Databridge Client side, you can filter data sets by setting the active column to 0 for the corresponding entry in the DATASETS Client control table.

If you want to filter data sets that contain confidential or sensitive information, consider using a logical database or a filtering routine in the DBGenFormat utility. In this case, the Databridge Client will have no record that these data sets exist.

Filtering Rows

Row filtering limits data to certain ranges; you can accomplish this via the WHERE clause of filtering routines created with the DBGenFormat program on the host. For more information, see Chapter 4 in the Databridge Host Administrator's Guide.

Filtering OCCURS Tables

OCCURS tables are secondary tables generated by the Databridge Client when OCCURS clauses for items (or GROUPs) are not flattened. Frequently, not all rows in such tables contain meaningful data, for this reason it is desirable to filter such rows to reduce the storage requirements and improve performance. Starting with version 6.5, the Databridge Client implements row filtering for OCCURS tables. For more information, refer to OCCURS Table Row Filtering.

Databridge Components

The following table lists all of the Databridge products and components that can have a role when replicating data with the Databridge Client.

Databridge Host (installed on the mainframe)

Databridge Enterprise Server

A Windows-based product that provides the same functionality as the Databridge Engine (DBEngine) and Databridge Server (DBServer) on the host. Enterprise Server offloads much of the replication workload from the Unisys mainframe to a Windows computer, reducing mainframe resource utilization and initial load time.

Databridge Clients can connect directly to Enterprise Server, which in turn connects to DBServer on the mainframe. If MCP disks are directly accessible from the Windows server, Enterprise Server extracts the DMSII data directly. Enterprise Server reads the audit trail on the host to retrieve updates that occurred during the extraction and sends the changed information from the audit file to the Client. If MCP disks are not directly accessible, Enterprise Server uses DBServer to retrieve blocks of data from DMSII data sets or the audit files. Enterprise Server provides high-speed file transfer between the host and the Windows environment and audit file mirroring.

Databridge Client

The Client initiates a connection with the Databridge Server and then specifies the DMSII data sets to be replicated from a DMSII database.

Databridge FileXtract

An application that allows you to clone and update Flat Files that reside on Unisys ClearPath NX, LX, or A Series mainframes. You can also use FileXtract with the Databridge Client to replicate this data. From the Client perspective, FileXtract data sources look like DMSII data sources.

FileXtract is bundled with Databridge Host software and includes several Reader libraries and other associated files.

Databridge Flat File Client

The Flat File Client (also known as PCSPAN) is a Windows implementation of the DBSPAN accessory on the MCP. As is the case with DBSPAN, rather than update the secondary database, the Flat File Client creates data files that contain the data records for the updates. This approach is useful when a Databridge Client does not exist for a particular database or platform or when the data has to be transformed before being loaded into a secondary database. The Flat Client has a very similar architecture to the relational database clients, such as the SQL Server and the Oracle Clients.

Databridge Kafka Client

The Databridge Client for Kafka, which is implemented on Linux and Windows platforms, enables the ability to utilize the Kafka messaging system within the Databridge architecture. The Kafka messaging system is a scalable fault-tolerate data management system that provides efficient real-time data processing.

Comparing the Databridge Client Manager Service to Command-Line Operations

The Databridge Client Manager service performs the same operations as the command-line Client, dbutility. We refer to these operations using the term "service" whether the Client Manager service is running on Windows platforms (service) or UNIX/Linux platforms (daemon). Each machine has its own service (Windows) or daemon (UNIX/Linux). The primary advantage to using the service is its ease of use and the ability to automate most of your Client processes. Additionally, Client runs initiated by the service can't be interrupted or tampered with as they occur as background runs.

The following table can give you a better idea of how the two modes of operations compare when performing Client-specific tasks.

For more information about dbutility commands, see dbutility Commands. For more information about the Administrative Console, see Using the Console and Service and the Help included in the Administrative Console.

Switching from Command-Line to Service Operations

Use this procedure if you currently run the Databridge Client from a command line and want the Client Manager to run it, or if you need to add existing data sources to the Client Manager.

To operate the Client Manager on the Client machine, you'll need a specific directory structure, referred to as the service's working directory.

To switch to the service based client (Windows)

1. Set up the service's working directory. For more information, see The Working Directory in the Databridge Installation Guide.

2. Install the Administrative Console that matches the version of Databridge Client software you use. For instructions, see the Databridge Installation Guide.

3. Do one of the following:

   • If you use a text Client configuration file, proceed to step 4.

   • If you use a binary Client configuration file (that is, you have not exported your configuration file to a text file to edit it), skip to step 6.

4. From the data source directory, locate the config folder, and copy the text configuration file "dbridge.cfg" to a file named "dbridge.ini". The following image shows an example of a working directory for a data source named "zdatabase".

   

5. From a command prompt, go to the working directory for your data source and run the dbutility import command. This creates a binary configuration file required by the service.

6. To make sure that the Client Manager service is running, from the Start menu, click Control Panel > System and Security > Administrative Tools > Services, double-click Micro Focus Databridge Client Manager 7.1, then click [Start].

7. Using a browser such as Microsoft Edge or Google chrome enter the following URL https://hostname:7445/ where hostname is the name or IP address of the Administrative Console server.

8. From the Databridge Server page click +Add and provide the hostname or IP address of the Client Manager and the port it listen on (typically 8001). This will add the node to the list of servers and detect the fact that it is a Client Manager.

9. Repeat step 8 for the Databridge server you wish the data source be associated with. This will add the node to the list of servers and detect the fact that it is a Databridge server on the MCP or a Databridge Enterprise Server. It will detect if the connection to the Databridge Server on the MCP is encrypted or not.

10. Go to the page for the Client Manager you just added by Clicking on Client Managers link and click on the Client Manager in question. Click on the +Add menu and click on New to open the "Add new data source" dialog.

11. Select the Databridge server from the dropdown list provided. Note that if you skipped Step 9, you can add the server by pushing the Add new server radio button and providing the server name or IP address and the port it listens on. When done push the continue button which move you to the next page of the dialog.

12. The Administrative console retrieves the data source names configured for the server in question and provides you with a list box from which you can select the desired data source.

13. Fill in the rest of the boxes on the page with the relational database information that is needed. This includes the ODBC data source name for the SQL Server Client, the PostgreSQL Client, the database name for the Oracle Client, and the relational database signon information user/password. And in the case of the SQL Server, you can use Integrated Windows Authentication. When done click the Add button on the lower right corner of the display.

To switch to the daemon based client (UNIX/Linux)

1. Log on as the userid specified in the USERID parameter of the file globalprofile.ini. This is the same userid you currently use to run the command-line Client.

2. Set the current directory to the home or other directory to which you have write access and copy the script dbdaemon from the install directory to it. This allows you to make changes to the script.

3. In an editor, open the script dbdaemon.

4. Make sure that the environment variables (such as INSTALLDIR, WORKING_DIR, ORACLE_HOME, LD_LIBRARY_PATH) are correct for your system and edit them as needed.

5. Save and close the script.

6. Start the daemon by typing the following:

   dbdaemon start
   

7. To verify that the daemon is running use the ps command, which should produce output similar to the following:

   databridge@VMOPENSUS114-64:~> ps -ef | grep DBC
   databridge   1110     1  0 12:00 ?        00:00:00
   /opt/dbridge71/DBClntControl
   

8. If the daemon doesn't start, in the script dbdaemon, make sure that the WORKING_DIR and INSTALLDIR environment variables are correct. Also, check the Client Manager's working directory to determine if a file "dbdaemon.log" was created and if it contains any clues.

9. Using a browser such as Microsoft Edge or Google chrome enter the following URL https://hostname:7445/ where hostname is the name or IP address of the Administrative Console server.

10. From the Databridge Server page click +Add and provide the hostname or IP address of the Client Manager and the port it listen on (typically 8001). This will add the node to the list of servers and detect the fact that it is a Client Manager.

11. Repeat step 10 for the Databridge server you wish the data source be associated with. This will add the node to the list of servers and detect the fact that a Databridge server on the MCP or a Databridge Enterprise Server. It will detect if the connection to the Databridge Server on the MCP is encrypted or not.

12. Go to the page for the Client Manager you just added by Clicking on Client Managers link and click on the Client Manager in question. Click on the +Add menu and click on New to open the "Add new data source" dialog.

13. Select the Databridge server from the dropdown list provided. Note that if you skipped Step 11, you can add the server by pushing the Add new server radio button and providing the server name or IP address and the port it listens on. When done push the continue button which move you to the next page of the dialog.

14. The Administrative console retrieves the data source names configured for the server it question and prides you with a list box from which you can select the desired data source.

15. Fill in the rest of the boxes on the page with the various relational database information that is needed. This includes the database name for the Oracle Client and the relational database signon information user/password. When done click the Add button on the lower right corner of the display.

16. Type su to switch to the root user and then copy the script dbdaemon from the location you specified in step 5 to the following location:

    • (Linux/Solaris) /etc/init.d

    • (AIX) /etc/rc.d/init.d

    • (HP-UX only) /sbin/init.d

17. To make the operating system automatically start the daemon whenever the system starts, consult the documentation for your operating system.

Command-Line Client Operations

This section describes the tasks required to clone a DMSII database and track changes using the Databridge Client from a command line. This sequence assumes that the relational database is properly configured and the required Databridge software is installed and running.

Before You Run the Command-Line Client

Before you use dbutility, do the following:

• Set up the relational database.

• Install the Databridge software on the host and set up and start Databridge Server (see the Databridge Host Administrator's Guide).
  If Databridge Server requires signon parameters, include these in the dbutility configuration file. If necessary, you can also provide these parameters in the environment variables or in dbutility command-line options.

• Install or upgrade the Databridge Client (see the Databridge Installation Guide) and create a separate working directory for each DMSII database to be cloned. In most cases, you'll add the relational database logon parameters to the data source configuration file. You can do this by supplying the signon parameters to the import command using command line switches (such as -U -P -O -W) when you create a new data source. Passwords are automatically encrypted.

Controlling and Monitoring dbutility

The Databridge Client includes a command-line console to help control and monitor dbutility. Commands are entered in the same command prompt session where dbutility is running. When dbutility is running, make sure that the focus is on the command prompt window and then use the keyboard to enter the command. You need to type only enough of the command to make it unique.

The configuration file parameter inhibit_console allows you to disable the command-line console, and the command-line option -C toggles this parameter. The available commands are as follows:

Databridge Client Exit Status Values

After each command, the Databridge Client command-line program, dbutility, returns an exit status value.

For detailed information about Client exit codes, see Appendix E: Client Exit Codes.

Testing for Exit Status

The following examples show how you can test for the exit status (exit_status).

Windows Example

This example tests for the exit_status in a Windows .cmd file:

dbutility process datasource
if errorlevel 2 if not errorlevel 3 goto reorg
exit
:reorg
echo DMSII reorganization has occurred
sendmail "message"

where <sendmail> is a hypothetical user application that sends a notification to you.

UNIX Example

The following example, written for the UNIX Korn shell (ksh), determines whether or not the value of the exit status (that appears after you run the dbutility program) indicates that a DMSII reorganization was detected. If a reorganization was detected, it echoes a message and runs the sendmail program:

dbutility process datasource
if [ $? -eq 2 ]
then
echo "DMSII reorganization has occurred"
sendmail "message"
fi

where sendmail is a hypothetical user application that sends a notification to you.