Working with Session Server Logging

The Host Integrator logging component coordinates all logging activity for the Host Integrator session server.

Host Integrator can log server errors and can issue warnings and informational messages as it runs. Using the Administrative Console, in the Logging perspective, Log view, you can build queries to provide valuable logging data.

Logging properties, such as storage location and disk utilization, are set on the Properties page. In the Session Server Explorer, Logging perspective, click to open the Properties page.

Log messages are shown in reverse order, that is from most recent at the top of the log to the oldest messages at the bottom.

Clearing the Log

To clear the log, click the Clear button. All the log messages currently in the log will be deleted. Note: The Clear button is not visible if security is enabled and the current user is not in the Administrator profile.

More information

• Building Queries
• Understanding SQL Dialect for the Logging System
• Setting Logging Properties

Setting Logging Properties

Host Integrator can log server errors and can issue warnings and informational messages as it runs. In the Logging perspective, Log view, you can build queries to provide valuable logging data.

To set logging properties

The options in this panel specify the type of logging that is performed, the location of log files, and time and space limits for log files.

Default Logging Level

The Default Logging Level item specifies which events are inserted into the log files by default. These messages are also sent to the operating system log if system logging is enabled (see below). This is a server wide setting. Client applications also have the ability to increase the logging level for their own private sessions. To record all server activity, select Log All Messages. This will include in the log all informational messages, warnings, and errors. Because this setting requires the server to track all activity, expect additional resource consumption on the server

Select a reduced logging level for improved performance on heavily loaded servers.

• Log Storage Directory

  This option specifies the directory in which log messages stored, which by default is <installation directory>\VHI\etc\logs\server (Windows) or <installation directory>/vhi/etc/logs/server (UNIX). Using this setting an administrator can specify a different local directory where new messages will be stored. The directory you specify must already exist. Any existing messages are transferred to the newly specified location.

• System Logging

  When System Logging is enabled, Host Integrator sends logging messages to both the Host Integrator log files and the operating system log, which is the Event Log on Windows systems and the syslog daemon on UNIX systems. Host Integrator will also send a message to the System Log when it detects a logging system failure, regardless of this setting.

Note

Host Integrator does not manage the size of the operating system log. When System Logging is enabled, you must take steps to limit the size of your system log.

• Failed Request Information Logging

  The Failed Request Information Logging option helps you troubleshoot client application problems.

  When Failed Request Information Logging is enabled and the server encounters an error, the server will write all activity for the failed request to the log file. This information will appear immediately before the error message in the log file. Because Failed Request Information Logging requires the server to track additional activity, enabling it will consume additional resources on the server.

  Failed Request Information Logging does not have to be enabled for request failures to be recorded; enabling this setting simply appends all activity associated with the failure in addition to the failure itself. When Failed Request Information Logging is disabled, the error itself is still logged.

  If Logging Level is set to Log All Messages, all activity is written to the log file even when no errors are encountered and regardless of the Failed Request Information Logging setting.

• Log Space Threshold Settings

  The Log Space Threshold is the total amount of disk space in MB that can be used to store log records. The default is 100 MB and the range of values is 1 - 100000 MB. When the Delete logs when disk space exceeds threshold check box is selected, the oldest log entries will be deleted when the amount of disk space used by the log records exceeds the threshold. The oldest messages are deleted until space utilization is decreased to 80% of the configured maximum limit.

  In addition to the space required to store messages, the logging system requires administrative storage space. For proper function of the logging system, ensure that the actual free disk space in the configured logging directory is at least twice the value of the Log Space Threshold setting.

  The space and time thresholds can be enabled at the same time. In this case, the most restrictive setting is used.

• Delete logs older than threshold

  Enable this option to delete logs older than the threshold set in the Log Space Threshold.

• Log Time Threshold (in days)

  The Log Time Threshold is the expiration date for all log records. The default is 30 days, and the range of values is 1 - 100000 days. When the Delete logs older than threshold check box is selected, messages older than the specified threshold will be purged.

  The space and time thresholds can be enabled at the same time. In this case, the most restrictive setting is used.

More information

• Exporting Log Files
• Understanding the SQL Dialect of the Logging System

Building Queries

You can securely perform detailed queries against session servers using the Host Integrator logging component.

• You can run these queries against any installed session server.
• Queries are specified through a flexible SQL SELECT expression.
• The messages returned from a query are displayed in the Logs view where you can rearrange and sort columns.

The Log view toolbar

This button	Does this...
	Runs the query
	Cancels a running query. If you want to interrupt a query while it is running, click this toolbar button.
	Brings up the Build Query dialog box where you can create a query.
	Clears the messages from the viewer and from the server.

To create a query

1. From the Host Integrator drop down list, open the Logging perspective.

2. From the list of session servers, select the ones you want to run the query against. You can run a query against multiple session servers.

3. The drop down list displays the default query string and the most recently used queries. You can run a query using this filter or build your own query. The default query, select * from messages, displays all messages.

   You can type a query directly into the text box, however, you must be careful to use the correct syntax.

4. If you decide to build your own query, click Build Query on the toolbar.

   The Build Query dialog box displays.

5. Set the parameters for your query.

   As you build the query it displays in the Query box.

6. Check Execute Query to activate the query automatically when you close the dialog box.

To set query parameters

You set query parameters using the Build Query dialog box. To use multiple values for Message ID, Session ID, Request ID, and so forth, separate them using commas.

• Event types

  You can filter the log to display any combination of error messages, warnings, and informational messages. Select the check box for each event type you want to be displayed.

• Specifying View From and View To Options

  The From (msgtime) options set the starting selection for the records that are displayed. You can decide to view records from the first record, or by selecting a date and time. If you choose First Record, the Start Date and Start Time options are ignored and the displayed list begins with the oldest available record.

  If you select Start Date/Time, then you can set the date and time to see the records that occurred after the date and time specified.

  The To (msgtime) options specify the ending date of the records that are displayed. You can decide to set the ending date when the last record displays, or you can set a specific date and time. When you select the Last Record option, the Stop Date and Stop Time options are ignored and the displayed list ends with the most recent available record.

  When the First Record and Last Record options are selected, all log records are displayed, up to a maximum of the first 1000 records. To view records prior the 1000 records that are displayed, specify a stop date and time that is before the 1000th record.

• Message ID

  The Message ID is the value associated with this particular message.

• Session ID

  The Session ID is the number assigned at run time to the session. The first byte in the number represents the processor; servers that contain multiple processors may have large values for the Session ID.

• Request ID

  The Request ID is assigned at run time for each unique request.

• Client IP Address

  The Client IP Address is the IP address of the user who connects to the Host Integrator Server.

• User Name

  The User Name is name of the person who connects to the Host Integrator Server.

• Model or Session Pool Name

  When connecting to the Host Integrator Server, a session will use either a session pool or a specific model, but not both.

Viewing Query Results

Logging results display in the Log view of the Administrative Console and are selectable using the right-click context menu. You can select multiple rows from the query table and save them to a clipboard in a comma-separated format.

Log messages are shown in reverse order, that is from most recent at the top of the log to the oldest messages at the bottom.

The results of the query are scrollable. The statement describing the query will determine the order and number of columns displayed.

More information

• Setting Log Properties
• Exporting Log Files
• Understanding the SQL Dialect of the Logging System

Exporting Log Files

The logexport command line utility provides you with the ability to securely perform detailed queries against the Host Integrator logging component. It is capable of performing bulks extracts of messages from the logging component suitable for parsing or importing into third-party applications.

• The utility can be run from any host on the network. (Windows or UNIX)
• Access control is integrated with the Host Integrator security architecture.
• Queries are specified through a flexible SQL SELECT expression.
• The output format is configurable and parseable by other applications.
• The utility can perform bulk extracts of messages form the Host Integrator logging system.

Operation

The logexport utility can be found in the bin directory of the Host Integrator installation. You can run it from this location, or you can add this directory to your PATH for easy access. The utility has a variety of command line switch options.

Help on available options can be obtained with the use of the command line switch -help.

logexport -help

LogExport -query sql [ -delim delim_char ] [ -escape esc_char ]
                  OR
          -purge msgserial

          -server servername
          -ds mgmtservername
          -user username
          -password userpass
          -profile ( Administrator | Developer )
          [ -config filename ]

The utility performs two primary functions:

• Query a message store for messages. (-query switch)
• Purge old messages from a message store. (-purge switch)>

One of these options must be specified, but they are mutually exclusive. The -server switch is required. This option specifies which log service to contact.

Regardless whether security is enabled on the Host Integrator installation in question, logs are protected from access by unauthorized users. Access to logs always requires a user to authenticate with the management server. The options -ds, -user, -password, and -profile are required. Both administrators and developers can perform queries against the logs, but only administrators can execute message purges.

Querying Messages

The -query option takes a SQL select statement which specifies which messages you want to see. For example:

logexport -server s1 -query "select msgtime, msgtext from messages"

This invocation will contact the logging service (Log Manager Service) running on host "s1" and output the message time and message text of all messages currently in the Host Integrator server logs. The results will be directed to standard out. Any errors that occur will be directed to standard error. Columns of output will be separated with the "|" character.

Along with the -query option, two optional parameters can be specified, -delim and -escape. Each of these options takes a single character. Use the -delim switch to specify the character used to separate columns in the output. It is the "|" character by default. Use the -escape option to specify the character used to escape occurrences of the column delimiter that occur within any column of message data. It is the "\" character by default.

For details on the SQL supported by the Host Integrator logging system, see Understanding the SQL Dialect of the Host Integrator Logging System.

Within the SELECT statement, specify the columns of the message entry desired in the output, and the order they will appear. To specify all columns in the default order, use "*" as the column identifier.

Each message entry has the following columns available:

Name	Type	Description
msgserial	INTEGER	Message primary key
msgtime	TIMESTAMP	Date/Time message
msgid	INTEGER	Numeric message identifier
sessionid	INTEGER	Session identifier
requestid	INTEGER	User request identifier
cltaddr	VARCHAR	User's network name
user	VARCHAR	User's name
model	VARCHAR	Model or pool name
msgtext	VARCHAR	Text of the message
severity	VARCHAR	Message severity

The SQL SELECT statement optionally can contain a WHERE clause that allows the user to specify complex matching criteria on the message entries they desire to see.

Purging Messages

If this utility is used to perform a bulk extract of messages from the Host Integrator logging system, you will need the ability to purge messages from the system after they have been safely loaded into another database or application. All messages within a message store are tagged with a unique serial number that can be obtained by specifying the msgserial column in a query. This column is the message's primary key. The -purge option takes a message serial number, and deletes all messages in the specified message store, up to and including the serial number specified. For example:

logexport -server s1 -purge 1234

This example deletes all messages with a msgserial number less than or equal to 1234 on machine "s1".

Advanced Features

Due to the relatively large number of available switches on the logexport utility, you may want to specify run-time arguments from a configuration file. Specify the file containing run-time options on the command line. To employ this feature, create a file containing the desired options, and specify the path to the file as the argument to the -config switch.

logexport -config sample.config

Any switch that can be specified at the command line, with the exception of the -config switch, can be placed in the config file. The format of the file is shown below:

#
# Comments look like this.
#
# Line tags are the same as switch names.  Any number required can 
# be specified in the file.
#
# server
# query
# purge
# delim
# escape
# ds
# user
# password
# profile
# 
# Here are some real entries.
#
server=s1
query=select msgtext from messages

A config file can be specified in combination with other command line switches. If a setting is specified more than once, the value of its final specification is used during run-time. This allows you to create a default configuration file, but override particular values for a given run. For example:

logexport -config myconfig -server somepc

In the above example, logexport will contact server "somepc" regardless of whether "server" is specified in the file "myconfig".

More information

• Building Queries
• Understanding the SQL Dialect of the Host Integrator Logging System
• Setting Logging Properties

Understanding the SQL Dialect of the Logging System

The use of SQL provides a flexible and expressive way to locate messages of interest within the Host Integrator logging system. This topic defines the dialect of SQL supported by the Host Integrator logging system.

Functional Specification

Host Integrator supports the SQL SELECT statement and its associated WHERE clause. Nested conditional expressions in the WHERE clause are supported, using the following relational and logical operators:

AND, OR, NOT, =, <>, >=, <=, LIKE

As an example take the following SQL statement:

select msgtime, severity, msgtext from messages where sessionid = 45 and msgtime < timestamp ‘2001-09-10 10:00:00’

This statement returns a result set of three columns, the message time, its severity, and the text of the message itself, in that order, for all messages whose session identifier was 45 that occurred prior to 10 am, September 10, 2001. Many more queries are possible.

The logging system contains a single table named “messages”. This table contains the following columns:

Column Name	Type	Description
msgserial	INTEGER	Message primary key
msgtime	TIMESTAMP	Date/Time of message
msgid	INTEGER	Numeric message identifier
sessionid	INTEGER	Session identifier
requestid	INTEGER	User request identifier
cltaddr	VARCHAR	User's network name
user	VARCHAR	User's name
model	VARCHAR	Model or pool name
msgtext	VARCHAR	Text of the message
severity	VARCHAR	Message severity

Ordinarily, you will be comparing column names to literal values in our WHERE clauses, so conditional expression phases will look something like the following:

Expression Description

Expression	Description
colname <&234	Compares a column to a numeric literal
colname = ‘Some Text’	Compares a column to a character literal
colname > TIMESTAMP ‘2001-08-12 22:12:45’	Compares a column to a TIMESTAMP literal

It is important to compare similar types to avoid a type mismatch error. Also note that if a character literal contains a single quote, as in ’Let’s’, it is escaped by a preceding single quote.

The LIKE operator performs simple pattern matching on character columns in accordance with the SQL standard. The pattern is specified as a character literal. The characters “” and “%” have special meaning. A “” matches any single character while a “%” matches zero to any number of characters. If you want to match a “_” or a “%” literally, you must specify an escape character using escape and use it in the expression. For example:

Expression	Result
msgtext LIKE ‘entity’	Matches “entity” exactly
msgtext LIKE ‘entity%’	Matches entries that start with “entity”
msgtext LIKE ‘%entity%’	Matches entries that contain “entity”
msgtext LIKE ‘___ entity%	Matches entries that start with “(any 3 letters) entity”
msgtext LIKE ‘%entity%error%’	Matches entries that contain “entity” before “error”
msgtext LIKE ‘%&%%’ ESCAPE ‘&’	Matches entries that contain “%”

Additional Example Statements

Below are several examples of supported statements. For a formal definition of what is supported, see the section: BNF Specification.

Statement	Result
select * from messages where msgtime > timestamp ‘2001-23-08 10:00:00’	Returns all columns, in the default order, of any messages that occurred after 10:00 am, August 23, 2001.
select msgtime, severity, msgtext from messages where sessionid = 23 and user = ‘ralf’	Returns columns msgtime, severity, and msgtext, in that order, of any messages that originated from session 23 under the control of “ralf”.
select msgtime, severity, msgtext from messages where sessionid = 12 and msgtext like ‘%session%’	Returns columns msgtime, severity, and msgtext, in that order, of any messages that originated from session 12 and whose msgtext column contains the string “session” anywhere within it.
select cltaddr, msgtext from messages where sessionid = 45 and (cltaddr like ‘150.123%’ or cltaddr = ‘grumpy’)	Returns columns cltaddr and msgtext in that order, of any messages that originated from session 45 and whose cltaddr column starts with “150.123” or whose cltaddr is “grumpy”.

BNF Specification

This section provides a detailed description of the supported grammar in BNF. (it all starts with 'select-exp').

select-exp
    ::=
   SELECT select-item-commalist FROM table-ref
          [ WHERE cond-exp ]

select-item-commalist
    ::=   select-item [ , select-item-commalist ]

select-item
    ::=   column-ref 
        | *

table-ref
    ::=   messages

cond-exp
    ::=   cond-term
        | cond-exp OR cond-term

cond-term
    ::=   cond-factor
        | cond-term AND cond-factor

cond-factor
    ::=  [ NOT ] cond-primary

cond-primary
    ::=   simple-cond
        | ( cond-exp )

simple-cond
    ::=  comparison-cond
        | like-cond

comparison-cond
    ::=   scalar-exp comparison-oper scalar-exp

like-cond
    ::=   char-string-exp LIKE char-string-exp [ ESCAPE char-string-exp ]

scalar-exp
    ::=   numeric-exp
        | char-string-exp
        | datetime-exp

comparison-operator
    ::=   =
        | <
        | >=
        | >
        | <=
        | <>

numeric-exp
    ::=   numeric-primary

numeric-primary
    ::=   column-ref
        | numeric-literal

char-string-exp
    ::=   character-string-primary

character-string-primary
    ::=   column-ref
        | character-literal

datetime-exp
    ::= datetime-primary

datetime-primary
    ::=   column-ref
        | timestamp-literal

timestamp-literal
    ::=  TIMESTAMP character-literal

More information

• Building Queries
• Setting Logging Properties