Automating Client Operations with the Service
This chapter explains how to automate Client operations by using the service and scripts (that is, command files on Windows; shell scripts on UNIX). It also covers the Batch Console and its role in automation.
Configuring the Service
To configure update scheduling, error recovery, and other features of
the service, use the Administrative Console. Your changes are automatically
saved to the service's configuration file dbcontrol.cfg, located in
the config directory of the working directory. See
Managing Operations in the Administrative Console Help, available from
within the program. For more information about the service's working
directory, see the topic The Working Directory in the Databridge
Installation Guide.
Automation Scripts
The Databridge Client 7.0 service uses scripts (command files in
Windows; shell scripts in UNIX) to allow the user to gain control at key
points of Client operations. Scripts allow the user to perform
pre-processing and post-processing for Client runs (typically
process commands) and supplement the service's built-in error
recovery with additional recovery or error-reporting mechanisms.
Additionally, users can start a script from the mainframe by using the BCNOTIFY program. The BCNOTIFY program is designed to make the service start the script whose name it supplies. By inserting BCNOTIFY at key points in a WFL (Work Flow Language), you can trigger a task on the Client machine, such as restarting audit file processing. Scripts can interact with the service via the Batch Console (bconsole), a program that interprets source files that contain programs written in a language vaguely resembling to Visual Basic. Scripts can also perform arbitrary tasks that may not be directly related to the service (for example, start a run that generates a Crystal Report).
Automation scripts fall into two basic categories:
-
Scripts that are associated with a
processcommand. These scripts are associated with a data source and run before or after a Client run. (See Process-Related Scripts.) -
Scripts that are initiated by the BCNOTIFY program on the mainframe. (See BCNOTIFY Initiated Scripts.)
All scripts start with the current directory set to the service's
working directory. Additionally, the service sets up the environment
variable INSTALLDIR that points to the install directory where the Batch
Console program resides. You must use this environment variable as the
path when invoking the Batch Console (on Windows,
%INSTALLDIR%\bconsole; on UNIX, $INSTALLDIR/bconsole). You cannot
rely on the install directory being in the PATH. For more information
about Batch Console, see Introducing the Batch Console.
Process-Related Scripts
This first category of automation scripts includes scripts that are
associated with a process command. These scripts are referred to as
start-of-run and end-of-run scripts. Start-of-run scripts are only
applicable to runs started by the service without any outside
interference, specifically runs that are started by the service's
scheduler. This includes runs that are launched when the service is
started in response to the run_at_startup parameter for a data source.
End-of-run scripts on the other hand are applicable to all runs.
Client runs that are started from the Administrative Console, the Batch Console, and BCNOTIFY do not look for start-of-run scripts.
Both types of scripts follow strict filename conventions, as follows.
| Type of Script | Naming Convention |
|---|---|
| start-of-run | source_startofrun.ext |
| end-of-run | source_endofrun.ext |
| event-notice | source_eventnotice.ext |
where source is the data source name in lowercase letters and ext is the file extension (.cmd for Windows or .sh for UNIX).
The service searches for these script files in the scripts
subdirectory in the service's working directory. (For information about
the working directory, see The Working Directory in the Databridge
Installation Guide.) Before a scheduled run is started, the service
looks for the existence of a start-of-run script. When the service finds
a script, it runs it and then starts the process command after the
script is complete. If the script does not exist, the process command
is started without issuing any errors or warnings. When the service
determines that the process command is complete, it checks for the
existence of an end-of-run script and runs it.
When a process command terminates with an exit code that initiates
auto-recovery, the service checks for the existence of an event-notice
script and runs it if found. These type of scripts are designed to give
the user the ability to generate event notices, such as e-mails when the
service enter auto-recovery. In the absence of these scripts the DBA
would be totally unaware that the service restarted the Client after a
recoverable error.
If a Client run ends because of an error, the end-of-run script will run
only after the service's built-in error recovery has been executed. The
service has a built-in recovery mechanism for responding to errors and
non-zero exit codes. In situations that are temporary or self-resolving,
the service will attempt the failed operation after a brief delay.
For example, if a connection to the server or database fails, the
service will pause for a specified amount of time before attempting to
reconnect, and will do so repeatedly until the server or database
becomes available. (The parameter sched_retry_secs determines the
initial retry interval. For connect failures, this interval doubles on
each subsequent retry until it reaches a maximum value of 5 minutes. You
can set this parameter value in the Client Configuration dialog box.
When using the Administrative Console, from the data source's Settings menu
click on Processing > Error Recovery, you will find these parameters
in the "Options" group at the top of the page.
Both start-of-run and end-of-run scripts are passed the following set of parameters:
| Script Parameter | In the script, referred to as |
|---|---|
| Data source name | %1 |
| Exit status | %2 |
| Run type (a number that indicates the type of command) | %3 |
| Token used as the password when connecting back to the service using the Batch Console | %4 |
For start-of-run scripts, run_type is always 1, indicating a
process command. For end-of-run scripts, run_type can be 1, 2 (clone command), 7 (redefine command), 8 (generatecommand) or 4
(Administrative Console **Customize command run).
BCNOTIFY Initiated Scripts
This second category of automation scripts are initiated by BCNOTIFY, a utility included with Databridge Host software.
BCNOTIFY is a host-based Databridge utility you can use to issue remote procedure calls (RPC) to the service and tell it to launch scripts. BCNOTIFY passes the script name and parameters in the RPC. BCNOTIFY can optionally pass a data source name as an additional parameter. If you do not supply a data source name in the RPC, the data source name must be provided within the script that the service launches. The advantage of including the data source name in the RPC is that the service will only launch the script if the data source is idle (versus always launching it).
Scripts initiated by BCNOTIFY are named as follows:
start_name.ext
where name is an arbitrary name, and <ext> is .cmd on Windows and
.sh on UNIX.
When BCNOTIFY launches a script that initiates a process command, the
service behaves differently when looking for an end-of-run script to
execute. It first looks for a script named end_name.ext in the scripts subdirectory (where name is the name used in the
original script and ext is the OS dependent file extension). If the
service finds this script, it uses the script in place of the standard
end-of-run script described earlier. Otherwise, the standard end-of-run
script is used if it exists. This allows one to associate multiple
end-of-run scripts with a data source, depending on which script started
the process command.
These script files are passed the following set of parameters. The parameters for these scripts can change, depending on whether the data source is an empty string. For example, if no data source name is provided, parameter one is the AFN and parameter 2 is the token.
Parameters: - Data source name (optional) - Parameters supplied by BCNOTIFY. For example, the current database audit file number (AFN) - A token used as the password when connecting back to the service via the Batch Console
Introducing the Batch Console
The Batch Console automates routine Client tasks by allowing command files/shell scripts launched by the Databridge Client Manager to interact with the service. It interprets a source file that contains a set of statements written in a language similar to Visual Basic. These statements can initiate a connection, perform rudimentary tests, and issue console requests, to the service. For example, by using the Batch Console in an end-of-run script that runs daily reports, you can restart the Client after the reports are generated.
To use the Batch Console, you must first create a source file for the
Batch Console and place it in the scriptsdirectory of the service's
working directory (also referred to as the Client's global working
directory). We recommend that you use a file extension that allows you
to easily identify this file as a Batch Console source file (for
example, .bcs). You can debug this source file by running the Batch
Console from the command line, using the source filename (including
directory, such as scripts/source_filename) as the first argument of
the Batch Console command.
The Batch Console always runs as a background run. Its activity is
written to a log file in the current directory. The log filename uses
the source filename with the extension .log added to it. For example,
if your source filename is sourcefile.bcs the log file is named
sourcefile.bcs.log.
Running the Batch Console (bconsole)
The Batch Console program (bconsole) has a command line of the form:
bconsole [options] filename [argument list]
where filename is the name of a text file that contains commands for the Batch Console to interpret.
Note
All scripts start with the current directory set to the service's working directory. Additionally, the service sets up the environment variable INSTALLDIR that points to the install directory where the Batch Console program resides. You must use this environment variable as the path when invoking the Batch Console (on Windows, %INSTALLDIR%\bconsole; on UNIX, $INSTALLDIR/bconsole). You cannot rely on the install directory being in the PATH. For more information about Batch Console, see Introducing the Batch Console.
You can include additional arguments to pass parameters to the program. This allows you to use generic source files that work with externally-supplied values for the parameters. The command-line parameters in [argument list] are referenced in the script file using the Windows command file conventions (%1 is parameter 1, %2 is parameter 2, and so on). For example, if you invoke bconsole using the statement
bconsole /P secret resume.bcs 1234
the program substitutes the text "1234" for any occurrence of "%1"
in the script file "resume.bcs" the same as Windows command files.
The following command file performs the aforementioned task for a fictional data source named MISDB. MISDB uses the service that runs on a machine named "galactica" using port 8001. (Included with the following command are words, such as "data source", which are ignored by the parser but make the script read more like plain English. These words appear in black text.)
connect to galactica port 8001
enable data source MISDB
process MISDB
If a command fails, the program returns a non-zero exit status indicating a failure. On successful execution of the script, the program returns an exit status of 0. Each executed script is logged; if something fails, you can look at the log file to determine what went wrong.
The user ID of the user that launches the bconsole run is used to sign on to the service. When the Batch Console program starts from a script that the service launches, the script is passed a handle for use as a temporary password. This eliminates any security problems that having user IDs or passwords in the script files might cause. If the service cannot authenticate the user ID password, it verifies that the user is the same one that is running the service. (This is typically the built-in SYSTEM account). If it is, the service verifies that the handle matches the one assigned to the thread that launched the script. (The handle contains the thread number.)
Signing On to the Service
The userid that starts the Batch Console is also used to sign on to the
service. This eliminates the security problems that can result from
including userids and passwords in script files. When the service
launches a script, it passes a handle for the script to use as the Batch
Console password. This password is set using the command-line option
/P.
After the service identifies the userid as being the same as the service's userid, it validates the signon once it determines that the password matches the handle passed to the script. Handles are only valid while the script is running and cannot be reused.
Using Batch Console in Scripts Initiated by BCNOTIFY
You can use the Batch Console to make the service start a process
command or notify a currently running process command to stop after a
specified AFN. This mode of operation replaces dbauditwait working in
conjunction with the deprecated NOTIFY program on the mainframe and uses
the service-initiated DBClient runs instead of the command-line Client.
When invoking the Batch Console in a script launched by the service, you
must pass the handle to Batch Console using the /P option.
bconsole /P %2 sample_script2.bcs mikera018684 MISDB 1234
The following Batch Console source file sample_script2.bcs uses
command-line parameters similar to a Windows command file, except that
the parameters are numbered starting with the one that follows the
source filename. The program does a textual substitution by replacing
%n with the exact text of the corresponding parameter. (This script is
located in the scriptsdirectory of the service's working directory.)
connect to %1 port 8001
if run active %2 then
stop run %2 after afn %3
else
process %2 with option "/F %3"
end if
In the above example, the text %1 is replaced by mikera018640, %2is
replaced by MISDB and %3 is replaced by "1234".
Using Batch Console to Get Status Information
You can use the Batch Console in query mode to get status information. This capability is not related to automation, but is provided to let you query the service about the status of your data sources. Query mode connects to the service, gets the status of the specified data sources, and then writes that information to a file.
To use query mode, you must provide the connect parameters and the command using command-line switches. The command line for query mode is as follows:
bconsole /s service_name /p port /P password /w filename /q command
where the service_name, output filename, and command can
optionally be enclosed in double quotation marks. If the /w filename option is omitted, the program will write the output to the file
bconsole.log. The syntax for the command is:
status [data_source]
If data_source is omitted (or is specified as _all) the status of
all data sources is written to the output file in CSV format. A sample
output file is as follows:
MISDB,0,0x00000000,0,9999
DEMODB,0,0x00000000,0,9999,,,,disabled
NTSL,0,0x00000000,1,0,2011-03-31@15:26:46,2011-03-31@17:46:52
Each line includes (in this order): a) the data source name; b) its state; c) the process-id of the current run (or 0 if there's no active run); d) type (state) of the last run; e) exit code of last run or 9999 if the run is active; f) start time of the active run or the last run (if there's no active run); g) stop time of the last run (0 if there is an active run); h) the next scheduled run (if idle); and i) the flag for the data source. If a Client run crashes, it will have an exit code of 9999 and the data source will be marked as disabled.
Batch Console Commands
The syntax for Batch Console (bconsole) scripts is loosely modeled after Visual Basic. The end-of-line character acts as a statement terminator. This limits you to one statement per line. The following table lists commands in alphabetical order, followed by a list of buzz words that are allowed to improve readability.
| Command | Reference |
|---|---|
abort |
abort [run [for [[data] source]]]... |
clone |
clone [[data] source]... |
connect |
connect [to]... |
define |
define [[data] source]... |
disable |
disable [[data] source]... |
disconnect |
|
display |
display "text" |
drop |
drop [[data] source]... |
dropall |
dropall [[data] source]... |
enable |
enable [[data] source]... |
exit |
exit (value) |
generate |
generate [[data] source]... |
| if expression then ... [else ...] end [if] |
|
launch |
[[[data] source]... |
process |
process [[data] source] |
redefine* |
redefine [[data] source]... |
reorg** |
reorg [[data] source]... |
reorganize |
reorganize [[data] source]... |
runscript |
runscript "filename" |
status |
status [[data] source]... |
stop |
stop [run [for [[data] source]]]... |
wait |
wait (value) |
* Synonymous with define
** Synonymous with reorganize
Statements in Detail
Let's look at the individual statements and syntax of a Batch Console script. All statements are confined to a single line, except for the if statement.
| This command | Does this |
|---|---|
| connect [to] service_name [port] number | Connects to the given service as a console. If the service name is an IP address or contains non alphanumeric characters, it must be enclosed in double quotation marks. |
| enable [[data] source] name | Enables the specified data source. If the data source doesn't exist, an error occurs. If the data source is not disabled, no command is executed and a warning results. This will not cause the script to terminate prematurely. To eliminate the warning, use an "if" statement; this test whether the data source is disabled before trying to enable it. |
| process [[data] source] name [[with] option[s] "option_list"] [reclone ds_list] | Initiates a process command. The options are specified the same as on the dbutility command line, using either slashes or a hyphen (depending on the operating system) followed by a letter and an argument, if applicable.When specifying options, you must include the keyword option to indicate that a list of options follows the command. Make sure that you separate each option with a space and enclose the entire set of options with double quotation marks. The program will validate the options before passing them to the service. You can force all of the specified data sets to be re-cloned by adding the keyword reclone followed by a list of data sets to the process command. This sets the ds_mode to 0 for all the specified data sets in a single command. If you use the name "all", all data sets will be re-cloned; you don't need to name them individually. |
| clone [[data] source] name [[with] option[s] "option_list" ] ds_list | Initiates a clone command. The options are specified like you would in dbutility, using slashes or dashes (depending on the operating system) followed by a letter and an argument when applicable. |
| [re]define [[data] source] name [[with] option[s] "option_list"]] | Initiates a redefine or define command, depending on whether the data source exists. This statement causes the service to launch DBClientCfgServer for the data source (unless it is already running) and then execute a define/redefine command. |
| reorg[anize] [[data] source] name [[with] option[s] "option_list"]] | Initiates a reorganize command. This statement causes the service to launch DBClientCfgServer for the data source (unless it is already running) and then execute a reorg command. |
| generate [[data] source] name [[with] option[s] "option_list"]] | Initiates a generate command. The statement causes the service to launch DBClientCfgServer for the data source (unless it is already running) and then execute a generate command. |
| stop [run [for [[data] source]]] name [{after [afn] number | at [time] hh:mm}] |
| abort [run [for [[data] source]]] name | This is equivalent to the DBConsole Abort command. The Client run is terminated immediately by closing the TCP/IP connection to the server. This will cause the last transaction group to be rolled back by the Client. |
| status [[[data] source] name] | This command writes the status of a data source to the Batch Console (bconsole) log file. If the data source name is omitted, or the name “_all” is used, the status of all data sources is written to the log file. |
| display "string" | This command writes the given string to the log file. It is mainly intended to help debugging. |
| if expression then ... [else ... ] end [if] |
The block of commands following the "if" line, which must end with the keyword "then", are executed if the expression evaluates to true. This block ends with either an "else" or “end [if]” keyword (in the absence of an else clause). The else clause starts a new block of commands that will be executed if the expression in the “if” statement evaluates to false. In all cases “end [if]” ends the block that is started by a "then" or "else". |
| launch [[[data] source]] name cmd_file params | This command makes the service launch an arbitrary command file. It is only useful in debugging BCNOTIFY scripts, as this is the easiest way to launch them. For name, list the data source name. If the run is not associated with a specific data source, use "_none" (quotation marks not required). The cmd_fileis the filename of the script in the scripts subdirectory of the service's working directory that you want the service to launch .cmd_file and params must be enclosed in double quotation marks if they contain non alphanumeric characters, such as a period (.). |
| disconnect | This command tells the program to disconnect from the given service it is connected to. This command will not normally be needed, as the program will automatically issue it when it reaches the end of the script file. |
| wait (integer value) | This command injects a delay, (measured in seconds) in execution of the script file. It is mainly intended for debugging purposes. |
| exit (integer value) | This command stops the program and returns the specified exit code. The command is only needed to return a non-zero exit code or to stop the flow of execution within an “if” statement. |
If Statements
Use the "if" statement to test for the following three conditions for a data source:
-
Whether it is disabled
-
Whether a run is active
-
Whether a run is scheduled
The keywords "disabled", active, and "scheduled" are used to indicate the condition being tested. You must follow these keywords with a data source name and the keyword "then". Optionally, you can precede keywords with the buzzwords "run", "data source", or "source".
To reverse the test, you can place the keyword "not" in front of expressions that follow the keyword "if". The syntax of these expressions is summarized as follows:
[not] {[run] | [[data] source]} active name
[not] {[run] | [[data] source]} disabled name
[not] {[run] | [[data] source]} scheduled name
Command-Line Options
Note
Options are case sensitive. -p and -P are separate options.
| This Switch | Argument | Does this |
|---|---|---|
| -d | Enables debug output | |
| -o | Overwrites the log file (versus appending to it) | |
| -p | port | Specifies the port on the command line. |
| -q | Switches into single query mode (status command only) | |
| -s | name | Specifies the domain name or IP address of the service machine on the command line. |
| -t | Enables RPC traffic tracing. | |
| -w | filename | Sets the name of the log file. |
| -p | password | Specifies the password to be used when connecting to the service. |
| -T | Specifies that the user is a trusted user* |
* When you run the batch console (bconsole) from a command file that is
not launched by the service, you need to specify a password using the -P
option. Since the password is not encoded, some sites may find this
objectionable. In order to solve this problem we implemented the -T
option, which requires that the userid being used be registered as the
trusted user. The batch console will then read the Windows Registry and
determine if the userid is registered as the trusted user (there can
only be one in the current implementation). To facilitate the
registration process, the program setbcuserid.exe was implemented.
This program registers the userid you enter as the login userid. You
must be an administrator to run this program and the userid you specify
must be a valid Windows user.