Introducing Databridge Host

This chapter introduces the Databridge Host software, its replication methods, and their benefits. It also provides factors to consider when choosing a replication method for your business.

About Databridge

Databridge is a combination of host software and optional client software that provides automated replication of DMSII databases and flat files. All replications occur while the DMSII database is active. After the initial clone, Databridge updates the secondary database, copying only the DMSII data changes from the audit trail.

By copying the DMSII data to another system, you can offload all queries, reporting, and decision support to a secondary database—such as a relational database on a remote UNIX or Windows server—and reduce the expense associated with mainframe resources. Alternatively, you can copy DMSII data to a secondary database on the host. During replication, the mainframe database can be available at all times instead of being shut down for extractions and downloads and then being reopened for updates.

The following diagram shows the Databridge architecture.

Advantages of Databridge

Databridge provides the following benefits:

It only requires cloning once.
It synchronizes data sets/tables in the primary and secondary database.
You can select only the items you want to replicate from the primary database.
You can replicate to many platforms.
Databridge has low resource requirements.
When you use the Databridge host and client software, no file transfer utility is required to move the replicated data.
The secondary database provides a secure way to make data available to selected individuals, departments, or sites while protecting the primary database on the host.
The data is available on the client system even when the host is down or experiencing long wait times, or when the data communication connection is broken.
The client system provides the processing resources for database queries with no impact on the host. And since the data is typically filtered (that is, the client database contains only the necessary data items) there is less overhead.
You can avoid the prohibitive time and expense of developing (or purchasing) custom data conversion and porting software, and you can avoid costly mainframe hardware purchases that might be necessary if all users accessed the host directly.

Security Considerations

When you run a Databridge Accessory, normal host security restrictions apply. For example, the Databridge Accessory must run under a usercode that has access to the appropriate DMSII DESCRIPTION files, database CONTROL files, and the audit files. All of the typical file access rules of the host apply to the Databridge Accessories. In other words, security is dependent on the usercode under which Databridge is run.

Usercodes

We recommend that you install and run Databridge under a privileged usercode. To run Databridge under a different usercode from the one under which you installed it requires that you copy Databridge Engine (DBEngine) to that usercode or establish DBEngine as a system library. For details, see “DBEngine Visibility to Accessories and Usercodes” on page26.

Guard Files

Databridge supports MCP guard file validation on the host. Guard files provide controls that restrict access to specific files. For information about setting up guard files, see the Databridge Installation Guide.

Custom Filtering

Databridge provides a method for restricting access to certain data sets (or remaps) and certain records within the data sets. You can put these visibility restrictions in a tailored support library via the GenFormat program. Before Databridge returns a record to the secondary database system, it applies the visibility constraints in the tailored support library. See “Creating a Filter” on page76.

Logical Databases

You can restrict access to data sets and records by creating a logical database and then running the Accessory against the logical database. DBEngine restricts access to data sets and remaps in the logical database. You can specify the logical database when you start the Accessories via the WFLs.

Databridge Support for DMSII Structure Types

Databridge supports data sets, remaps, and logical databases. Databridge Accessories can use a remap just like a data set.

For information on DMSII structure types supported by Databridge Clients, see the Databridge Client Administrator’s Guide.

Databridge supports all DMSII data set structure types and data items, except the following:

Data Sets Not Supported

Embedded data sets if INDEPENDENTTRANS is reset or if embedded within an ORDERED or COMPACT data set. If the EMBEDDED EXTRACTS option is set in the Databridge Span or Snapshot parameter file or the Databridge Client configuration, such embedded data sets can be extracted (cloned, but without any fixups) but not tracked.
Partitioned structures

Data Items Not Supported

POPULATION
COUNT
FILLER
AGGREGATE

Even though the Databridge Engine (DBEngine) does not return layout information for the types listed above, by using RAWFORMAT, a host Accessory (Databridge Span, for example) can gain access to the data for the item types listed above. If the Accessory uses a format generated by DBGenFormat, though, these special data items are not available. The Databridge clients do not have access to these special data items.

Choosing a Replication Method

Databridge offers several ways to clone(page210) a DMSII database. You can use any of these methods while the DMSII database is in operation and being updated.

Use this

For

Databridge Snapshot

Single DMSII data extraction or infrequent clones of a database.

Snapshot clones all of the selected data sets each time you run it (not just the changes). Because cleanup and consolidation takes place on the host, Snapshot uses more host resources than Span. See Span and Snapshot Compared for a comparison of these Accessories. For more information about Snapshot, see How Databridge Snapshot Works.

Databridge Span

A primary database that has frequent changes.

Span gathers only changes to the primary database when it writes updates to its output files. For more information, see How Databridge Span Works. For a comparison of these Accessories, see Span and Snapshot Compared.

NOTE: This Accessory does not apply the fixup records. In addition, the files must be manually transferred if used on a secondary system.

The following methods require Databridge Host and additional Databridge components.

Use this	For
Databridge Server and the Databridge Client	Cloning and transferring DMSII data and subsequent updates to a remote client. For more information, see the Databridge Client Administrator's Guide and the Databridge Client Console Help.
Databridge Enterprise Server and Databridge Client	Cloning and transferring DMSII data and subsequent updates to a remote client. Use this method to reduce mainframe usage and related expense by moving all cloning, tracking, and filtering to a Windows system. This method is similar to the preceding method except that Enterprise Server performs the functionality of Databridge Server and the Databridge Engine. For more information, see the Help included with the Databridge Enterprise Server.
Databridge FileXtract	Replicating flat files. For more information, see the Databridge FileXtract Administrator's Guide.
DMSII Client or Databridge Twin	Replication to a Unisys MCP Server. For more information, see the Databridge DMSII Client Administrator's Guide or the Databridge Twin Administrator's Guide.

Updating

Updating is an ongoing process for propagating changes from records in the DMSII database to the secondary database. Databridge uses the Audit Files created by DMSII to extract and apply the updates to the secondary database.

Note

Databridge cannot read audit files copied to tape with the Unisys QuickCopy program.

Databridge can clone an unaudited database but cannot track any changes, i.e. it cannot do any updating.

Updating can be accomplished using exclusively Databridge Host software. Alternatively, you can incorporate Databridge Client software to perform this function.

Host-based updating

In this method of updating, you run Databridge Span to retrieve updates from DBEngine and then write those records to one or more data files on the host. You then transfer those files to the new location and load the data into a secondary database.

An advantage of using host-based updating is that you can replicate DMSII data to any system, even if Micro Focus has not developed a Databridge Client for that particular system.

A disadvantage of this method is that it puts more responsibility on you to deliver and process the updates in the data files.

Client-based updating

In this method of updating, a Databridge Client program (typically running on another system) communicates with DBServer (or DBEnterprise), which retrieves information from DBEngine.

The Databridge Client requests updates from DBServer, which in turn calls the appropriate entry points (page 211) in DBEngine to retrieve the modified records. The Databridge Client program then updates the client database with those changes.

Loose vs. Tight Replication.

You can determine how current you want to make your secondary database by choosing between a loose replication and a tight replication. Regardless of which you choose, only Databridge updates should be allowed on the secondary database. This maintains data integrity between the primary database and the secondary database.

Loose replication

In the loose replication model, Databridge accesses only closed audit files for the DMSII changes. This results in a time delay between a change to the primary database and the corresponding change to the secondary database.

You can control the amount of time between secondary database updates in the following ways:

Using the AuditTimer(page179) utility to schedule audit file closes. For example, you could close the audit file every hour, every day, or even every week, depending on how crucial current information is.
Using the Copy Audit(page186) utility to schedule Databridge Span to start each time an audit file closes.
Using the Copy Audit utility or the Notify(page178) utility in conjunction with Databridge Server to notify the Client that additional audit file information is available.
Running the Databridge Accessory each time you want updates.

Tight replication

In the tight replication model, Databridge accesses the current audit file for the DMSII changes. Updates to the primary database are performed almost immediately on the secondary database.

Databridge can read the current audit file and provide near real-time updates to the secondary database.

Audit Files

An audit file is created by DMSII and contains the raw format of changes made to the DMSII database by update programs. Audit file records contain the deletes, adds, and modifies that were made to the various structures. It can contain, for example, hours', days', or weeks' worth of information.

Databridge uses the audit file for the raw data of each database change to exactly replicate the primary database. Databridge records the audit location (AFN, ABSN, SEG, IDX) between runs, so it can restart without losing any records.

If you set the DBEngine Read Active Audit option, Databridge can access the current audit file. If you do not set Read Active Audit = true in the DBEngine parameter file, Databridge can access audit information from audit files up to, but not including, the current DMSII audit file. The audit file contains the update level at the time the audit file was created. The update level in the audit file and the update level in the DMSII DESCRIPTION file used by Databridge must match before Databridge will update a replicated database.

When an audit file is closed, DMSII creates the next one in the series. Audit files are closed for several reasons, including the following:

An operator closes the audit file with the mixnumber SM AUDIT CLOSE command.
The audit file reaches the file size set in its DASDL.
There is an I/O error on the audit file.
There is not enough disk space for this audit file.
The database update level changes due to database definition changes
The current audit file could not be found.
A file reorganization was executed to modify the DMSII structure.

Forcing an Audit Switch..

If you generate less than one audit file a day and you don’t want to set Read Active Audit = TRUE, you may want to force an audit switch daily and run the Span Accessory or DBServer so that the secondary (replicated) database is at most one day behind. In this case, you can use the following procedure instead of running AuditTimer.

This works only for databases that are currently open for updating (that is, one or more programs have done an OPEN UPDATE databasename).

To force an audit switch

From the ODT or action line in MARC, transmit the following:

DBS

A list of active databases and their associated mix numbers appears.
Transmit the following for the database audit file you want to close:

mixnumber SM AUDIT CLOSE FORCE

where mixnumber is the mix number of an active database.

The current audit file is now closed, and a new one is created.

You can also use Audit Close utility to close the current audit file. See Audit Close Utility. For additional information on forcing an audit switch, see the Unisys DMSII Utilities documentation.

Replication Overview

The following steps outline the procedure for using host-based Accessories to replicate a DMSII database.

Run the Lister Accessory (page212) to determine which DMSII data sets you want to clone or replicate with Databridge.
From the Lister report, make a list of the data sets you want to replicate.
Edit the DBGenFormat(page212) file and create the record formats and filters that you want to use. When done, compile the Databridge Support library.
Choose the replication method (for example, using the Databridge Span or Snapshot Accessory).
Run the Accessory against the DMSII database. (Both Accessories generate a default parameter file the first time you run them.)
Edit the parameter file by selecting the data sets you want to replicate, the formatting and filtering routines to use, etc.
Replicate the DMSII data by running Databridge Span or Snapshot to write data files.
Transfer the data files for the replicated data sets to the secondary database system. This is site dependent. You might use a file transfer program, for example, or you might physically carry tapes to another location.
Update the replicated DMSII data.

If you use Databridge Span to replicate your DMSII database, update the secondary database by running the Span Accessory to write data files that contain changes gathered from the audit trail. Then, use one of your user-written programs to read these data files and update the secondary database.

If you use Databridge Snapshot to clone your DMSII database, update the secondary database in one of two ways:
- Run Databridge Snapshot again. The entire collection of data sets is recloned and the old secondary database is reloaded by running one of your user-written programs with the new data files.
- Insert the Snapshot audit location into a Databridge Span parameter file and then run Databridge Span.

Note

Databridge Accessories do not permanently reside in the mix. Instead, initiate the Accessory for the task you want to complete. Databridge then initiates and subsequently terminates any other programs, libraries, or utilities needed to complete the task. As long as you installed Databridge so it has visibility to your DMSUPPORT Library and DMSII DESCRIPTION and CONTROL files, you can run the Databridge Accessories whenever necessary.

Description of Databridge Components

Each Databridge product (for example, Databridge Host, Databridge Client) and the components they include are described in the following table.

Databridge Host (installed on the mainframe)

Component	Description
Engine (DBEngine)	The main component of the Databridge software, the Databridge Engine is a host library program that retrieves structural information, layout information, and data from the DMSII database and audit file and passes the information to Databridge Server.
DMSII Support Library (DBDMSIISupport)	DMSII Support is a Databridge library that retrieves data records from the DMSII database for cloning. The Databridge Engine links to this library to perform database functions such as reading records, switching the audit file, and getting database statistics.
Server (DBServer)	An Accessory that provides communications between DBEngine and the Databridge Client, and also between DBEngine and Databridge Enterprise Server. DBServer responds to Databridge Client requests for DMSII data or DMSII layout information. It retrieves updates by asking DBEngine to read the audit files on the host and send the changes to the Client.
Support Library (DBSupport)	A library that provides translation, formatting, and filtering to the DBServer and other Accessories. After DBServer receives data from the Databridge Engine, it calls the Support Library to determine if the data should be replicated, and if so, passes the data to the Support Library for formatting.
GenFormat (DBGenFormat)	A host utility that creates translation, filter, and format routines. The GenFormat utility interprets the GenFormat parameter file to generate ALGOL source code patches, which are included in the tailored Support Library.
Span (DBSpan)	Produces a replication of one or more data sets into flat sequential disk files that can be extended when more audit becomes available. The Databridge Span Accessory updates the extracted flat files by appending the changes to the end of the flat files (unlike the Databridge Snapshot Accessory, which replaces the changed records).
Snapshot (DBSnapshot)	Produces a clone of one or more data sets into tape or flat sequential disk files that consist of records suitable for bulk loading into a client application (for example, a spreadsheet or a relational database). The Snapshot Accessory clones the selected data sets each time you run it.
Lister (DBLister)	Produces a report of the layout of the structures in your DMSII database, including structure numbers and key sets.
Info Utility (DBInfo)	Produces a report of your DMSII database timestamps, update levels, DMSII release levels, etc.
WFL (Work Flow Language) Jobs	Provide customizable ways to run Databridge Accessories. For example, the Notify WFL makes DBServer notify the Client whenever audit files are available for processing.
AuditTimer Utility	Schedules times for closing an audit file.
Copy Audit Utility	Enables you to specify the number of closed audit files that should be saved on disk, automatically have DBServer notify the Client each time an audit file becomes available, and run the Span Accessory each time an audit file becomes available.
Audit Close Utility	Closes (switches) the current audit file.
Audit Remove Utility	Removes processed audit files.
Sample Source Code and Accessories	Illustrate how to use the Databridge API to write your own Accessories. These sample Accessories are not supported. For more information, see the Databridge Programmer’s Reference.
APIs (Application Program Interfaces)	Provide access to DBEngine from any Databridge or user-written Accessory (for example, Span) for the purpose of retrieving information for a DMSII database. For more information about the Databridge API, see the Databridge Host Programmer's Reference.

Databridge Enterprise Server

A Windows-based product that provides the same functionality as the Databridge Engine (DBEngine) and Databridge Server (DBServer) on the host. Enterprise Server offloads much of the replication workload from the Unisys mainframe to a Windows computer, reducing mainframe resource utilization and initial load time.

Databridge Clients can connect directly to Enterprise Server, which in turn connects to DBServer on the mainframe. If MCP disks are directly accessible from the Windows server, Enterprise Server extracts the DMSII data directly. Enterprise Server reads the audit trail on the host to retrieve updates that occurred during the extraction and sends the changed information from the audit file to the Client. If MCP disks are not directly accessible, Enterprise Server uses DBServer to retrieve blocks of data from DMSII data sets or the audit files. Enterprise Server provides high-speed file transfer between the host and the Windows environment and audit file mirroring.

Component	Description
Enterprise Server (DBEnterprise)	A Windows program that provides replication services to Clients.
Director (DBDirector)	A Windows Service that listens for Client connection requests and starts Enterprise Server whenever a connect request is received.
EnumerateDisk (EnumDisk)	A command-line program that lists MCP disks that are visible to Enterprise Server.

Databridge Client

The Databridge Client is a product that replicates the DMSII database to a relational database. The Client initiates a connection with DBServer on the MCP server (or DBEnterprise on a Windows server) and then specifies the DMSII data sets to be replicated from a DMSII database.

Component	Description
Client Service	The service (Windows) or daemon (UNIX) that automates most Client operations. It handles operator requests from the Client Console and routes all log and informational messages to the consoles.
DBClient	A Client program that is launched by the service. DBClient handles the processing of DMSII data and updates the same as the command line client (dbutility), except that it runs as a background run and uses the Client Console to display its output and interact with the operator.
DBClntCfgServer	A program that handles all requests from the Client Console specific to a data source. These requests include updating the client configuration file, providing access to the client control tables, and handling the Client Configurator. Like DBClient, this program is run by the service as a background run.
dbutility	A program that runs the Databridge Client from a command line.
Batch Console (bconsole)	A program that allows Windows command files (UNIX shell scripts) to issue process-related requests to the Client Service. The Batch Console interprets (and runs) scripts that are written in a language that vaguely resembles Visual Basic.
Administrative Console	A html-based console from which you can connect to the Client service. From the Administrative Console you can start the Client Configurator, which lets you customize the layout of the relational database.

Databridge FileXtract

Databridge FileXtract is an application that allows you to clone and update flat files (page212) that reside on Unisys MCP Servers. You can also use FileXtract with the Databridge Client to replicate this data. From the Client perspective, FileXtract data sources look like DMSII data sources.

FileXtract is bundled with Databridge Host software and includes several Reader libraries and other associated files.

Databridge DMSII Client

The Databridge DMSII Client is a product that installs on the MCP Server. DMSII Client clones and updates the DMSII database as another DMSII database. With DMSII Client, you can selectively replicate by filtering on both rows and columns.

Databridge Flat File Client

The Flat File Client (also known as PCSPAN) is a Windows implementation of the DBSPAN Accessory on the MCP. As is the case with DBSPAN, rather than update the secondary database, the Flat File Client creates data files that contain the data records for the updates. This approach is useful when a Databridge Client does not exist for a particular database or platform or when the data has to be transformed before being loaded into a secondary database.

The Flat Client has a very similar architecture to the relational database clients, such as the SQL Server and the Oracle Clients.

Databridge Twin

A mainframe program that replicates a DMSII database as another DMSII database. Twin typically runs on a development or departmental mainframe. As DMSII audit becomes available, updates to the primary database are applied to the secondary database.