Automating Client Operations with the Service

This chapter explains how to automate Client operations by using the service and scripts (that is, command files on Windows; shell scripts on UNIX). It also covers the Batch Console and its role in automation.

Configuring the Service

To configure update scheduling, error recovery, and other features of the service, use the Administrative Console. Your changes are automatically saved to the service's configuration file dbcontrol.cfg, located in the config directory of the working directory. See Managing Operations in the Administrative Console Help, available from within the program. For more information about the service's working directory, see the topic The Working Directory in the Databridge Installation Guide.

Automation Scripts

The Databridge Client 7.1 service uses scripts (command files in Windows; shell scripts in UNIX) to allow the user to gain control at key points of Client operations. Scripts allow the user to perform pre-processing and post-processing for Client runs (typically process commands) and supplement the service's built-in error recovery with additional recovery or error-reporting mechanisms.

Additionally, users can start a script from the mainframe by using the BCNOTIFY program. The BCNOTIFY program is designed to make the service start the script whose name it supplies. By inserting BCNOTIFY at key points in a WFL (Work Flow Language), you can trigger a task on the Client machine, such as restarting audit file processing. Scripts can interact with the service via the Batch Console (bconsole), a program that interprets source files that contain programs written in a language vaguely resembling to Visual Basic. Scripts can also perform arbitrary tasks that may not be directly related to the service (for example, start a run that generates a Crystal Report).

Automation scripts fall into two basic categories:

• Scripts that are associated with a process command. These scripts are associated with a data source and run before or after a Client run. (See Process-Related Scripts.)

• Scripts that are initiated by the BCNOTIFY program on the mainframe. (See BCNOTIFY Initiated Scripts.)

All scripts start with the current directory set to the service's working directory. Additionally, the service sets up the environment variable INSTALLDIR that points to the install directory where the Batch Console program resides. You must use this environment variable as the path when invoking the Batch Console (on Windows, %INSTALLDIR%\bconsole; on UNIX, $INSTALLDIR/bconsole). You cannot rely on the install directory being in the PATH. For more information about Batch Console, see Introducing the Batch Console.

Process-Related Scripts

This first category of automation scripts includes scripts that are associated with a process command. These scripts are referred to as start-of-run and end-of-run scripts. Start-of-run scripts are only applicable to runs started by the service without any outside interference, specifically runs that are started by the service's scheduler. This includes runs that are launched when the service is started in response to the run_at_startup parameter for a data source. End-of-run scripts on the other hand are applicable to all runs.

Client runs that are started from the Administrative Console, the Batch Console, and BCNOTIFY do not look for start-of-run scripts.

Both types of scripts follow strict filename conventions, as follows.

where source is the data source name in lowercase letters and ext is the file extension (.cmd for Windows or .sh for UNIX).

The service searches for these script files in the scripts subdirectory in the service's working directory. (For information about the working directory, see The Working Directory in the Databridge Installation Guide.) Before a scheduled run is started, the service looks for the existence of a start-of-run script. When the service finds a script, it runs it and then starts the process command after the script is complete. If the script does not exist, the process command is started without issuing any errors or warnings. When the service determines that the process command is complete, it checks for the existence of an end-of-run script and runs it.

When a process command terminates with an exit code that initiates auto-recovery, the service checks for the existence of an event-notice script and runs it if found. These type of scripts are designed to give the user the ability to generate event notices, such as e-mails when the service enter auto-recovery. In the absence of these scripts the DBA would be totally unaware that the service restarted the Client after a recoverable error.

If a Client run ends because of an error, the end-of-run script will run only after the service's built-in error recovery has been executed. The service has a built-in recovery mechanism for responding to errors and non-zero exit codes. In situations that are temporary or self-resolving, the service will attempt the failed operation after a brief delay. For example, if a connection to the server or database fails, the service will pause for a specified amount of time before attempting to reconnect, and will do so repeatedly until the server or database becomes available. (The parameter sched_retry_secs determines the initial retry interval. For connect failures, this interval doubles on each subsequent retry until it reaches a maximum value of 5 minutes. You can set this parameter value in the Client Configuration dialog box. When using the Administrative Console, from the data source's Settings menu click on Processing > Error Recovery, you will find these parameters in the "Options" group at the top of the page.

Both start-of-run and end-of-run scripts are passed the following set of parameters:

For start-of-run scripts, run_type is always 1, indicating a process command. For end-of-run scripts, run_type can be 1, 2 (clone command), 7 (redefine command), 8 (generatecommand) or 4 (Administrative Console **Customize command run).

BCNOTIFY Initiated Scripts

This second category of automation scripts are initiated by BCNOTIFY, a utility included with Databridge Host software.

BCNOTIFY is a host-based Databridge utility you can use to issue remote procedure calls (RPC) to the service and tell it to launch scripts. BCNOTIFY passes the script name and parameters in the RPC. BCNOTIFY can optionally pass a data source name as an additional parameter. If you do not supply a data source name in the RPC, the data source name must be provided within the script that the service launches. The advantage of including the data source name in the RPC is that the service will only launch the script if the data source is idle (versus always launching it).

Scripts initiated by BCNOTIFY are named as follows:

start_name.ext

where name is an arbitrary name, and <ext> is .cmd on Windows and .sh on UNIX.

When BCNOTIFY launches a script that initiates a process command, the service behaves differently when looking for an end-of-run script to execute. It first looks for a script named end_name.ext in the scripts subdirectory (where name is the name used in the original script and ext is the OS dependent file extension). If the service finds this script, it uses the script in place of the standard end-of-run script described earlier. Otherwise, the standard end-of-run script is used if it exists. This allows one to associate multiple end-of-run scripts with a data source, depending on which script started the process command.

These script files are passed the following set of parameters. The parameters for these scripts can change, depending on whether the data source is an empty string. For example, if no data source name is provided, parameter one is the AFN and parameter 2 is the token.

Parameters: - Data source name (optional) - Parameters supplied by BCNOTIFY. For example, the current database audit file number (AFN) - A token used as the password when connecting back to the service via the Batch Console

Introducing the Batch Console

The Batch Console automates routine Client tasks by allowing command files/shell scripts launched by the Databridge Client Manager to interact with the service. It interprets a source file that contains a set of statements written in a language similar to Visual Basic. These statements can initiate a connection, perform rudimentary tests, and issue console requests, to the service. For example, by using the Batch Console in an end-of-run script that runs daily reports, you can restart the Client after the reports are generated.

To use the Batch Console, you must first create a source file for the Batch Console and place it in the scriptsdirectory of the service's working directory (also referred to as the Client's global working directory). We recommend that you use a file extension that allows you to easily identify this file as a Batch Console source file (for example, .bcs). You can debug this source file by running the Batch Console from the command line, using the source filename (including directory, such as scripts/source_filename) as the first argument of the Batch Console command.

The Batch Console always runs as a background run. Its activity is written to a log file in the current directory. The log filename uses the source filename with the extension .log added to it. For example, if your source filename is sourcefile.bcs the log file is named sourcefile.bcs.log.

Running the Batch Console (bconsole)

The Batch Console program (bconsole) has a command line of the form:

bconsole [options] filename [argument list]

where filename is the name of a text file that contains commands for the Batch Console to interpret.

You can include additional arguments to pass parameters to the program. This allows you to use generic source files that work with externally-supplied values for the parameters. The command-line parameters in [argument list] are referenced in the script file using the Windows command file conventions (%1 is parameter 1, %2 is parameter 2, and so on). For example, if you invoke bconsole using the statement

bconsole /P secret resume.bcs 1234

the program substitutes the text "1234" for any occurrence of "%1" in the script file "resume.bcs" the same as Windows command files.

The following command file performs the aforementioned task for a fictional data source named MISDB. MISDB uses the service that runs on a machine named "galactica" using port 8001. (Included with the following command are words, such as "data source", which are ignored by the parser but make the script read more like plain English. These words appear in black text.)

connect to galactica port 8001
enable data source MISDB
process MISDB

If a command fails, the program returns a non-zero exit status indicating a failure. On successful execution of the script, the program returns an exit status of 0. Each executed script is logged; if something fails, you can look at the log file to determine what went wrong.

The user ID of the user that launches the bconsole run is used to sign on to the service. When the Batch Console program starts from a script that the service launches, the script is passed a handle for use as a temporary password. This eliminates any security problems that having user IDs or passwords in the script files might cause. If the service cannot authenticate the user ID password, it verifies that the user is the same one that is running the service. (This is typically the built-in SYSTEM account). If it is, the service verifies that the handle matches the one assigned to the thread that launched the script. (The handle contains the thread number.)

Signing On to the Service

The userid that starts the Batch Console is also used to sign on to the service. When you run the batch console (bconsole) from a command file that is not launched by the service, you need to specify a password using the -P option. Because the password is not encoded, some sites may find this objectionable.

To solve this problem, we implemented the -T option, which requires that the userid being used be registered as the trusted user. The batch console will then read the Windows Registry and determine if the userid is registered as the trusted user (there can be only one in the current implementation). To facilitate the registration process, the program setbcuserid was implemented. This program registers the userid you enter as the login userid. You must be an administrator to run this program and the userid that you specify must be a valid Windows user.

When the service launches a script, it passes a handle for the script to use as the Batch Console password. This password is set using the command-line option /P.

After the service identifies the userid as being the same as the service's userid, it validates the signon after it determines that the password matches the handle passed to the script. Handles are valid only while the script is running and cannot be reused.

Using Batch Console in Scripts Initiated by BCNOTIFY

You can use the Batch Console to make the service start a process command or notify a currently running process command to stop after a specified AFN. This mode of operation replaces dbauditwait working in conjunction with the deprecated NOTIFY program on the mainframe and uses the service-initiated DBClient runs instead of the command-line Client. When invoking the Batch Console in a script launched by the service, you must pass the handle to Batch Console using the /P option.

bconsole /P %2 sample_script2.bcs  mikera018684 MISDB 1234

The following Batch Console source file sample_script2.bcs uses command-line parameters similar to a Windows command file, except that the parameters are numbered starting with the one that follows the source filename. The program does a textual substitution by replacing %n with the exact text of the corresponding parameter. (This script is located in the scriptsdirectory of the service's working directory.)

connect to %1 port 8001
if run active %2 then
    stop run %2 after afn %3
else
    process %2 with option "/F %3"
end if

In the above example, the text %1 is replaced by mikera018640, %2is replaced by MISDB and %3 is replaced by "1234".

Using Batch Console to Get Status Information

You can use the Batch Console in query mode to get status information. This capability is not related to automation, but is provided to let you query the service about the status of your data sources. Query mode connects to the service, gets the status of the specified data sources, and then writes that information to a file.

To use query mode, you must provide the connect parameters and the command using command-line switches. The command line for query mode is as follows:

bconsole /s service_name /p port /P password /w filename /q command

where the service_name, output filename, and command can optionally be enclosed in double quotation marks. If the /w filename option is omitted, the program will write the output to the file bconsole.log. The syntax for the command is:

status [data_source]

If data_source is omitted (or is specified as _all) the status of all data sources is written to the output file in CSV format. A sample output file is as follows:

MISDB,0,0x00000000,0,9999
DEMODB,0,0x00000000,0,9999,,,,disabled
NTSL,0,0x00000000,1,0,2011-03-31@15:26:46,2011-03-31@17:46:52

Each line includes (in this order): a) the data source name; b) its state; c) the process-id of the current run (or 0 if there's no active run); d) type (state) of the last run; e) exit code of last run or 9999 if the run is active; f) start time of the active run or the last run (if there's no active run); g) stop time of the last run (0 if there is an active run); h) the next scheduled run (if idle); and i) the flag for the data source. If a Client run crashes, it will have an exit code of 9999 and the data source will be marked as disabled.

Batch Console Commands

The syntax for Batch Console (bconsole) scripts is loosely modeled after Visual Basic. The end-of-line character acts as a statement terminator. This limits you to one statement per line. The following table lists commands in alphabetical order, followed by a list of buzz words that are allowed to improve readability.

* Synonymous with define

** Synonymous with reorganize

Statements in Detail

Let's look at the individual statements and syntax of a Batch Console script. All statements are confined to a single line, except for the if statement.

If Statements

Use the "if" statement to test for the following three conditions for a data source:

1. Whether it is disabled

2. Whether a run is active

3. Whether a run is scheduled

The keywords "disabled", active, and "scheduled" are used to indicate the condition being tested. You must follow these keywords with a data source name and the keyword "then". Optionally, you can precede keywords with the buzzwords "run", "data source", or "source".

To reverse the test, you can place the keyword "not" in front of expressions that follow the keyword "if". The syntax of these expressions is summarized as follows:

[not] {[run]  | [[data] source]} active name
[not] {[run]  | [[data] source]} disabled name
[not] {[run]  | [[data] source]} scheduled name

Command-Line Options

* When you run the batch console (bconsole) from a command file that is not launched by the service, you need to specify a password using the -P option. Since the password is not encoded, some sites may find this objectionable. In order to solve this problem we implemented the -T option, which requires that the userid being used be registered as the trusted user. The batch console will then read the Windows Registry and determine if the userid is registered as the trusted user (there can only be one in the current implementation). To facilitate the registration process, the program setbcuserid.exe was implemented. This program registers the userid you enter as the login userid. You must be an administrator to run this program and the userid you specify must be a valid Windows user.