Using the Sample Reader Libraries

FileXtract includes several sample Reader libraries, which are configured to replicate flat files as standard data sets for the given file type. You can typically use the sample Reader libraries without modifications, except in the following circumstances:

Your flat files contain data in a proprietary format
New fields have been added to a standard layout
You want to replicate additional types of records that the Reader library does not support (for example, a new SUMLOG record type)

The Reader libraries work in conjunction with a physical database that DBEngine uses to determine the layout of the flat files. DBEngine gets the information it needs from the DESCRIPTION file for the layout database. The DMSII CONTROL file and the DMSUPPORT library will exist, but the data sets and audit files will not.

A DASDL file is provided for each sample Reader library so that you can create the corresponding layout database. For example, if you want to replicate SUMLOGs, you would use the SYSFILESDB DASDL to create a physical database called SYSFILESDB. Within this database is a logical database called SUMLOG that contains typical layout information for SUMLOG files. If needed, you can modify this information. If you customize the layout information in the logical database, you must make the corresponding changes in the Reader library.

Creating a Custom Reader Library

You must create a custom Reader library in either of the following situations:

The flat files that you want to replicate contain information in a “proprietary” format (for example, a flat file that requires reading the nth word to find the address of the next record).
The sample Reader library does not meet your needs. For example, you want to replicate additional types of SUMLOG records.

The easiest way to create a custom Reader library is simply to customize one of the sample Reader libraries to meet your needs. If this is not possible, complete the following steps to create a custom Reader library.

To create a custom library

Describe the file layouts in either DASDL or COBOL FD format.

Note

If you use COBOL FD format, use the COBOL-to-DASDL utility to generate the DASDL. For instructions, see Generating Database Layout from COBOL 01-Level File Record Descriptions.

Append a VERIFY clause to each data set that indicates when a flat file record is considered part of that data set. The DASDL must group the data sets into a logical database, as follows:

Logical_dbname ["reader [; file_spec]"] DATABASE (dataset_list)

where reader is the title of the Reader library object code and file_spec is a file or directory containing the flat files. Note that the semicolon is required before the file_spec, if present.
Write the Reader library source code in ALGOL or COBOL. If you are writing the Reader library in ALGOL, use the TEXT, SUMLOG, or TTRAIL Reader library as an example. If you are writing the Reader library in COBOL, use the BANKFILE Reader library as an example.

SThe procedure called in the Reader library must be named FileRead. Use DBFileReaderHead to define the procedure in ALGOL. Use $FEDLEVEL =5, $ SET LIBRARYPROG, and set PROGRAM-ID to FILEREAD for COBOL. FileRead is passed two arrays—the FileInfo structure and FileRecord for the resulting record image.
Perform required initialization the first time FileRead is called. This might include linking to DBEngine, getting the structure index for the data sets, identifying the location of the flat files, etc.

Note

Linking to DBEngine, whether explicit or implicit, must not occur before the first call of FileRead.
Configure the Reader library to read the file, filter records, copy the data to the FileRecord array, and set items in FileInfo such as the structure index, file record location, timestamp, etc. Set the appropriate structure index, as defined in the Databridge Programmer’s Reference, in the FileInfo array to determine the data set to which a record belongs. Note that by setting the structure index to 0 in the FileInfo array, you can rely on the DASDL VERIFY clauses to determine the data set to which a record belongs; however, this is not the preferred method because it is slower.

Note

In addition to filtering and formatting done by the Reader library, DBSupport can also provide filtering and formatting. See the Databridge Host Administrator’s Guide for more information about DBSupport.

Sample Reader Libraries

SUMLOG Reader Library

This Reader library reads system summary log files and returns the data records to DBEngine. This Reader library uses the SUMLOG logical database of the SYSFILESDB layout database.

Closed SUMLOG files are typically titled:

*SUMLOG/ssss/mmddyy/nnnnnn ON familyname

where ssss is the system serial number, mmddyy is the date in month-day-year format, and nnnnnn is a sequential number.

The Reader option in the Accessory’s parameter file should specify the directory and family name where the closed log files are located. The following is an example of the DBServer SOURCE declaration for this Reader library: Source SUMLOG:

% ------
Database = SUMLOG of DESCRIPTION/SYSFILESDB ON DISK,
Support = OBJECT/DATABRIDGE/SUPPORT/SYSFILESDB ON DISK,
Filter = SUMLOGFILTER,
Reader using "SUMLOG/= ON DISK"

If the Reader option is missing or empty, the SUMLOG Reader library will use the following:

"*SUMLOG/ssss/= ON sumlogpack"

where ssss is the serial number of the system it is running on and sumlogpack is the pack containing *SYSTEM/SUMLOG. When all of the closed SUMLOGs have been processed, the Reader library will automatically switch to reading the active SUMLOG.

TTRAIL Reader Library

This Reader library reads COMS Transaction Trail files and returns the data records to DBEngine. This Reader library uses the TTRAIL logical database of the SYSFILESDB layout database. TTRAIL files are typically titled:

*COMS/TTRAIL/dbname/nnnn ON familyname

where dbname is the database that participates in synchronized recovery and nnnn is a sequential number. The TTRAIL for all non-synchronized recovery programs has a database name of TPLIBRARY. The Reader option in the accessory parameter file must specify the directory and family name where the TTRAIL files for a particular database are located. The following is an example of the DBServer SOURCE declaration for this Reader library:

Source TTRAIL:
% ------
Database = TTRAIL of DESCRIPTION/SYSFILESDB ON
DISK,
Support = OBJECT/DATABRIDGE/SUPPORT/SYSFILESDB
ON DISK,
Filter = TTRAILFILTER,
Reader using "COMS/TTRAIL/= ON DISK";

PRINTFILE Reader Library

This Reader library reads printer backup files and returns the printer file attributes and print lines to DBEngine. This Reader library uses the PRINTFILE logical database of the SYSFILESDB layout database.

The Reader option in the Accessory’s parameter file should specify the directory and family name where the print files are located. The following is an example of the DBServer SOURCE declaration for this Reader library:

Source LISTER:
% ------
Database = REPORTS of DESCRIPTION/PRINTFILESDB ON
DISK,
Reader using "DBBD/RUN/LISTER/= ON DISK";

If the Reader option is missing or empty, the PRINTFILE Reader library will use:

"*BD ON printerbackuppack"

where printerbackuppack is the value identified by issuing a DL BACKUP command.

BICSS Reader Library

This Reader library reads BICSS log files and returns the data records to DBEngine. This Reader library uses the UVMSBICSS or IVRBICSS logical database of the NAPFILESDB layout database. BICSS log files are typically titled:

(usercode)BICSSLOG/ACTIVITY/yymmdd/hhmnss ON familyname

where yymmdd is the date in year-month-day format and hhmnss is the time in hours-minutes-seconds format.

The Reader option in the accessory parameter file must specify the directory and family name where the log files are located. Optionally, it can also specify the particular application system name that must appear in all records by appending a colon and the LINC application name, such as IVRSYS (see the example below) or UVMS. If you want records from only one application and the log contains records from more than one application, specifying the application name after the colon speeds up the processing.

The following is an example of the DBServer SOURCE declaration for this Reader library: Source IVRBICSS:

% --------
Database = IVRBICSS of DESCRIPTION/NAPFILESDB ON
DISK,
Support = OBJECT/DATABRIDGE/SUPPORT/NAPFILESDB ON
DISK,
Filter = BICSSFILTER,
Reader using "(BICSS)BICSSLOG/ACTIVITY/= ON DISK:
IVRSYS";

DISKFILE Reader Library

This Reader library reads flat files and returns the data records to DBEngine. When you use the COBOL-to-DASDL utility to create the various structures needed to replicate flat files that use COBOL 01-level file record descriptions, as explained in Generating Database Layout from COBOL 01-Level File Record Descriptions, the DISKFILE Reader library is the default Reader library.

The DISKFILE Reader library processes disk files in creation timestamp order. It assumes that the file it is reading can be extended as long as there is no file with a later creation timestamp. During processing, the creation timestamp of the disk file is used as the timestamp associated with each record. If you need to process a flat file having a creation timestamp prior to what FileXtract has already processed, see Out-of-Order Processing.

The Reader option in the accessory parameter file must specify the directory and family name where the disk files are located. The following is an example of the DBServer SOURCE declaration for this Reader library:

Reader using "(PROD)DATA/TRANLOG/= ON USERPACK"

If you want to change this behavior, you may modify the DISKFILE Reader library or substitute your own Reader library. If you use a name other than BJECT/FILEBRIDGE/READER/DISKFILE, make sure to specify the title of the Reader library code file in the READER "codefiletitle" USING "..." option of the accessory parameter file.

Some programs, such as CANDE and FileXpress®-XST, rewrite the entire file when records are added, deleted, or modified. The (modified) file will have a new creation timestamp. When the DISKFILE Reader searches the disk directory for files to process, it will find the file with a new creation timestamp and will attempt to process it from the beginning. This is likely to cause many “duplicates” errors because it will send each record as an “add” and all of the records from the old file are already present in the client database. To avoid these errors, you must configure the client to reclone every time it is run.

Similarly, if the mainframe application updates records in place, the DISKFILE Reader will not detect the update if it has already processed that file. In this situation, you must also configure the client to reclone every time it is run.

LINCLOG Reader Library

This Reader library reads LINC Activity logs and returns the data records to DBEngine.

LINC log files are typically titled:

(usercode)lincsystem/LINCLOG/nnn ON familyname

where nnn is the Activity log number.

The Reader option in the accessory parameter file should specify the directory and family name where the closed log files are located.

To generate the LINCLog database so that Databridge can replicate the log records to the client database, perform the following steps.

Edit the DATA/COBOLTODASDL/SAMPLE/LINCLOGDB/FD file, which describes the layout of the LINC Activity Log records, to reflect your particular LINC system. (Different LINC implementations sometimes have new fields added to the log record layout.)
Save the file as DATA/COBOLTODASDL/LINCLOGDB/FD, i.e., without the SAMPLE node.
Edit the DATA/COBOLTODASDL/SAMPLE/LINCLOGDB/CONTROL file with the proper location for the LINC log files.
Save the file as DATA/COBOLTODASDL/LINCLOGDB/CONTROL, again without the SAMPLE node.
Use the COBOL-to-DASDL utility to generate the database: START WFL/FILEBRIDGE/COBOLTODASDL ("LINCLOGDB")

This also generates a file called DATA/SERVER/LINCLOGDB/CONTROL that you can insert into DATA/SERVER/CONTROL to define the LINCLog SOURCE.
If you want to filter the log records, declare a filter and generate a tailored DBSupport library using DBGenFormat. For more information see the Databridge Client Administrator’s Guide for instructions. A sample file is provided - DATA/GENFORMAT/SAMPLE/LINCLOGDB/CONTROL.
Use the Databridge Client or an accessory such as DBSpan to replicate the LINC Log data just like any other database.

The following is an example of the DBServer SOURCE declaration for this Reader library:

Source "LINCLOGFILES": % ------------ Database = "LINCLOGFILES" OF "DESCRIPTION/LINCLOGDB", Reader using "(PROD)G2/LINCLOG/= ON AUDIT"; Reader processes the disk files in the LINC log file number order (with wraparound from 999 to 1).

BANKFILE Reader Library

This Reader library is a sample flat file Reader library that demonstrates how to write a FileXtract Reader in COBOL. For an example of how to use this Reader library to replicate a sample flat file called DATA/LOAD/SAMPLE/DATABASE, see BANKFILE COBOL-to-DASDL Example.

USERDATA Reader Library

This Reader library reads the system USERDATA file layout and returns the usercode attributes to DBEngine. It interprets the structure of the system USERDATA file, which contains attributes associated with usercodes. Because the layout of the USERDATA file can vary from site to site, this Reader library generates a DMSII database layout corresponding to your site-specific USERDATA file.

The USERDATA Reader library generates the following:

The corresponding DASDL layout for a DMSII database for FileXtract
The corresponding source code for the USERDATA Reader library

To generate and compile the DASDL layout and the USERDATA Reader library, do the following:

START WFL/FILEBRIDGE/COMP ("USERDATA")

The Reader option in the accessory parameter file should specify the directory and family name where the USERDATA file is located. The following is an example of the DBServer SOURCE declaration for this Reader library:

Source USERDATA:
% --------
Database = USERFILE of DESCRIPTION/USERDATADB
Support = OBJECT/DATABRIDGE/SUPPORT/USERDATADB
Filter = USERDATAFILTER
%- Reader using "SYSTEM/USERDATAFILE/othersystem ON
DISK";

If the Reader option is missing or empty, the USERDATA Reader library will use:

*SYSTEM/USERDATAFILE ON userdatapack

where userdatapack is disk location specified by the DL USERDATA command.

TEXT Reader Library

This is a skeleton of a Reader library written in ALGOL. It is intended to be used when writing a custom ALGOL Reader library.