IMS Fileshare Databases

There are two ways to share IMS Option databases between multiple users. Which technique you use depends on whether the database being shared is to be updated or not. This chapter describes the sharing of IMS Option databases which can be updated.

2.1 Overview

Sharing databases for read-only is much simpler than shared update databases. It only requires copying the database data files to a server folder and setting Shared read-only databases on the IMS page of the Project Settings dialog box to indicate the shared folder. There is a smaller overhead for read-only databases because there is no requirement for locking and unlocking of records and updates do not have to be logged for a possible rollback. See the section IMS Option Settings in the chapter Advanced Customization for further information.

The Mainframe Express Fileshare component provides support for shared update databases - hence the name Fileshare databases. Fileshare databases support database record level locking and dynamic rollback. Database record level locking ensures no one can update or retrieve segments which are in use by another user. In theory, shared update databases could be supported by locking entire databases but this would cause prohibitive performance in long-running transactions, such as when debugging a program. It could also cause high levels of unnecessary data contention even for short-running transactions, such as in a system test phase.

You can use Fileshare to configure databases for use in single-user mode. This can be helpful for testing checkpoint and restart logic in batch programs.The processing IMS Option performs is the same between single-user and multi-user access. You need to set up both the client and the server components, however, setup is easier because they are on the same machine and there is no network communications protocol required.

The rest of this chapter describes configuring and using Fileshare databases. You should read the chapter IMS Syncpoint Coordination for related issues.

2.2 Network Support

You need a network which is supported by Fileshare. The Fileshare server can run on a network server or on any PC on the network. If your network server is not supported, you may be able to run the Fileshare server on an unused PC. See your Fileshare User Guide for details.

2.3 Client Setup

To indicate a database is a Fileshare database, use the DB Catalog, which is accessed on the IMS page of the Project view. You can define DBD-names in the Database Catalog for one, a group or all of your databases. The DB Catalog defaults, accessed on the DB Catalog defaults page of the IMS System Properties dialog box, provide default values when a DBD is not listed in the Database Catalog. See the chapter Developing IMS Applications in your User's Guide for information on Database Catalog settings.

Fileshare databases (set on the IMS tab on the Project Settings dialog box) must also be set to indicate the name of the Fileshare server which processes the databases and, optionally, a drive and folder where the database data files are stored. The Fileshare databases entry value has the following format:

where fsname is the name of the Fileshare server which processes the database (Fileshare's /S parameter). The drive and folder are optional. If specified, they indicate the drive and folder on the machine on which the Fileshare server is running.

There are also two DB configuration options for database sharing which are accessed on the DB page of the IMS System Properties dialog box. One is named Update warning limit and the other is Initial wait for locked resource.

The Fileshare server has a companion program that runs on the client machine and is mostly transparent to your application. One exception to this is if you need to specify communication parameters for the Fileshare client to communicate with the Fileshare server. Some networking environments may require special settings. Refer to your Fileshare User Guide for more information.

This example indicates that IMS Option Fileshare databases are processed by the Fileshare server named FSSRVR. The databases are in the folder in which the Fileshare server was started.

This example indicates the same Fileshare server name but indicates that the database data files are in the c: drive and dbdata folder on the Fileshare server machine.

2.4 Server Setup

The Fileshare server needs to be installed and setup on your database server machine. This machine can be a network server or a machine on your network dedicated for this purpose. Your Fileshare User Guide provides complete instructions for the Fileshare server. This section and the sections which follow describe considerations for using Fileshare for IMS Option databases.

Fileshare automatically provides for commit and dynamic rollback support for IMS Option databases. There are no configuration requirements to enable this support. Specifically, you do not have to create a Fileshare server database reference file. If you do create a database reference file for IMS Option databases, you must ensure that you do not disable the transaction processing setting for a database. Disabling transaction processing prevents dynamic backout of updates.

You may optionally specify a forward recovery log. This would enable you to recover from damaged databases due to a power failure or other failures since starting the Fileshare server. Generally, the forward recovery log is relatively slow and should not be selected unless you need this support. During the early stages of application development, it might be easier to restore your databases from a backup. If you want to implement forward recovery, you must specifically add the IMS Option database filenames to the database reference file and specify the logging option. You can list individual databases or all of your databases. See your Fileshare User Guide for information on forward recovery.

For forward logging of secondary index databases, it may be better not to add them to the database reference file. During recovery, after you apply the log to the primary indexed database, you can run an IMSDBU Reorg for the database. The Reorg rebuilds the secondary index databases. This is a slower and more complicated recovery process than specifically logging the secondary index databases but it provides faster application execution. To run the IMSDBU utility choose IMS > Database Utility on the Tools menu.

2.5 Single-user Setup

Using Fileshare databases in single-user mode is fairly easy. You just need to indicate which databases are Fileshare databases and then configure Fileshare for single-user mode operation. Setup for the client is the same for both single or multi-user access. That is, you indicate your DB Catalog Type on the DB Catalog defaults page of the IMS Systems Options dialog box and select the path in Fileshare databases on the IMS page of the Project Settings dialog box.

You indicate single-user mode by setting the FSCOMMS value for the communications protocol for Fileshare. The value to set for FSCOMMS is "$LOCAL". The Fileshare server is started automatically, so you must include any Fileshare server start-up parameters when using $LOCAL.

As with multi-user access, the name following the /S: parameter in the FSCOMMS variable must match the name in the Fileshare databases field.

2.6 Fileshare Timeout

The Fileshare server keeps track of users by means of a unique internal session ID value. It keeps the locks for the session until a commit or a rollback when the locks are released. Fileshare and IMS Option cannot detect a hard failure while running a transaction. For example, switching off power to your PC.

Your first reaction might be to set the timeout value to a very large value so that you are not rolled back in the middle of a transaction. However, if you set it to be too long, you might lock yourself (or somebody else) out for the timeout period. This is easier to describe with an example. Supposing you made some IMS Option Fileshare database updates while debugging your program and accidentally switched off power to your PC. Fileshare would not detect this and so retains your locks. Then, you power up and start debugging the same program which would cause a lock request to the same records you had locked before. The lock request would not be granted as you appear to be a new user to Fileshare. The lock request would be re-tried until the timeout expired. When it expires, your previous locks are released and updates are rolled back. You will then acquire a new lock. Database integrity has been maintained but you have been locked out for the duration of the timeout period.

You must be careful when setting the timeout value: if you were debugging a program, it may be a fairly long time between database requests so you must set this timeout to reflect a long enough period so you are not rolled back while working. But, setting it to a very long time period can result in long lock out periods after a hard failure.

2.7 Record Locking Details

Before talking about record locking we must first define what a record is. For IMS Option, a database record is always the root segment and all of its physical children. Specific segments are never locked, only database records, although the root segment itself is locked to lock the database record. When accessing a database through a secondary index, the database record is the root segment of the target segment and its physical children. IMS/ESA basically uses the same technique but also has different locking schemes for different databases and different lock managers. It sometimes also does DL/I segment locking. For this reason, the database contention which occurs with IMS/ESA can be different than with IMS Option. The IMS Option locking strategies were chosen to provide data integrity, good performance in a PC network environment and on the understanding that they will frequently be used in an application development environment with very long running transactions.

Database records are locked when they are needed to maintain database position during and after processing a database call. The lock is released when the database record is no longer needed for database position as long as no segment in the database record was updated. If an update did occur to any segment in the database record, the lock is held until a syncpoint.

A lock request may not be granted depending on who else has a lock on the database record and what processing they have carried out. If the lock request is not granted, IMS Option continues to re-try the lock request. If the lock request is not granted for a configured time interval, a pop-up window is displayed indicating this condition. You have two choices from this window: you can choose to continue waiting or you can choose to terminate your application and rollback any update requests you've made to that point. This time interval, specified in seconds, can be changed on the window and its original value is indicated in the Initial wait for locked resource setting, which is accessed on the DB page of the IMS Systems Properties dialog box.

2.7.1 Performance and Locking Strategies

There are a number of issues affecting performance, for example: lock duration, lock contention and the expense of obtaining a lock. You need to determine which combination of these works best in your environment and you may find that it changes as you change your transaction types. For example, you may go from long running transactions while debugging a program to a system test cycle or deployment where transactions may be short and the expense of acquiring a lock is a more significant part of the transaction response time.

IMS Option provides three locking strategies for testing whether a lock request is granted, each having advantages and disadvantages where performance is concerned.

The three strategies are:

Lock strategy "1" is not used except for internal testing

Lock strategy "2" in which all Get calls obtain the same level of lock while any update call causes an update level lock The advantage of this strategy over lock strategy "3" is that it needs less computer resource, so possibly less time, to acquire and release the locks - it is also slightly easier to setup and administer. This strategy may provide the best performance if the transactions are very short or different users do not often access the same database records.
Calls are handled as follows:
- Get and Insert call locks are only granted if no other lock is outstanding against the database record
- Replace and Delete calls only change the lock level and are always granted
- Get call locks are released if the database record is no longer needed for database position
Although this provides for database sharing, it does not provide very well for database record sharing. That is, with this strategy only one person at a time can access a particular database record.
Lock strategy "3" is more sophisticated and is more like the locking performed by IMS/ESA. This requires using the IMS Option Supervisor and is explained in the next section.

2.8 Using the IMS Option Supervisor

Lock strategy "3" uses an IMS Option Supervisor and sets different levels of locks for get, and sometimes update, calls depending on the processing option (PROCOPT) value in the PCB for the database. If all of your PCBs specify update processing options, that is A, I, R or D, you are likely to get better performance using lock strategy 2, as described in the previous section.

The IMS Option Supervisor runs as a companion to Fileshare. This is described later in this section. Lock strategy 3 is automatically enabled if an IMS Option Application Region, which is making database requests to the Fileshare server, detects the presence of the Supervisor. Each IMS Option Application Region can only make database requests of one Fileshare server. This means that all databases being processed by one copy of a Fileshare server use the same lock strategy and all database requests from a single IMS Option "requester" use the same lock strategy.

IMS/ESA also uses the PCB processing options for setting different lock levels. This section describes how IMS Option sets lock levels based on processing options. Refer to your IMS/ESA manuals for details on how it manages locks.

Different levels of locks are set depending on the processing option in the PCB. This provides for multiple users to access the same database record at the same time. The different processing levels are:

Get Only - This is indicated by a processing option of GO (GOT and GON are processed in the same way)
Get - This is indicated by a G PROCOPT
Get with Update Intent or Update - This is indicated by an A, I, R or D PROCOPT

If the PCB has different processing options for different SENSEGs, the lock level used is the most restrictive one set for any SENSEG. For example, even if the PCB default PROCOPT specified G and one SENSEG specified GR, all locks made for this PCB would be "Get with Update Intent or Update".

A "Get" call lock is granted as long as there are no "Get with Update Intent or Update" locks, by another user, for the database record. Thus, many users can have "Get" call locks on the same database record at the same time. A "Get with Update Intent or Update" lock is only granted if there is no other user with a lock on the database record.

There are also locks required for the database before any database records are locked. This is generally of no concern but provides support for processing option L and E for a PCB. These indicate exclusive use of the database and the exclusive lock is granted only if no other user is using the database. No other user will be able to use this database until the exclusive database lock is released. The database lock is acquired when the first call is made to the database in this unit of work. Note that Zeroloading a database acquires an exclusive lock during the Zeroload process. All database locks other than exclusive are always granted as long as no exclusive locks are present.

If a lock request is not granted at the database or database record level, IMS Option continues trying the lock request, as described in the section Record Locking Details. When the Supervisor is enabled, the "resource in use" window contains the user ID of the person holding the lock. The user ID is obtained from System Configuration. For Fileshare, the user ID does not have to be unique (IMS Option and Fileshare have their own internal user IDs) so it is up to you to provide a meaningful user ID if one is desired. The IMS System Configuration user ID can be used for Fileshare security. See the chapter IMS Security for details.

2.9 Supervisor GO PROCOPTs

The GO processing option, "read only", is handled differently by IMS Option than on IMS/ESA. There are two options for the Supervisor for GO PROCOPT handling: a simple one and a more sophisticated one.

The simple one treats GO PROCOPTs as if they were G PROCOPTs. The advantage of this is that there is less locking activity and it can be faster than the more sophisticated one. The disadvantage is that a GO lock request is not granted if a Get with Update Intent or Update lock has been set by another user. However, if you do not often use processing option GO, you may want to change the IMS Option Supervisor's default from the more sophisticated GO processing to this simpler type to improve performance. See the next section for details on setting these values.

The more sophisticated GO handling, which is the Supervisor's default, grants a GO lock request as long as no insert or delete calls have been made for this database record. If only get calls and replace calls have been made against the database record (and thus only get or replace locks are present), the GO lock request is granted. With this method, the GO lock is a very low level lock. Before an insert or delete call is processed, it checks for any GO level locks and, if present, waits for the GO lock to be released. A GO level lock does not prevent any get or replace locks from being granted. This requires more locks to be sent to the Supervisor but was chosen as the default behavior as GO processing is fairly common when using alternate processing sequences and it provides better throughput in this case.

With this technique, invalid pointers are never encountered when using processing option GO. However, it may return data which was updated by another user but is later backed out. If you do not want this to occur then do not choose GO for the processing option.

2.10 Supervisor Installation

The IMS Option Supervisor is a program named Imssuper.gnt. You need to copy this program to the machine where Fileshare resides and place it in the same folder as the fs.exe and fs.dll files. Fileshare servers for Novell NLM, AIX and UNIX provide an IMSSUPER program for their environments.

IMSSUPER runs as a Fileshare virtual file handler and you need to create, or add to, the Fileshare server's database reference file. See your Fileshare User Guide for details. Below is an example command to add the Imssuper.gnt program as a virtual file handler:

The first call to an IMS Option Fileshare Database initializes the Supervisor. The Supervisor displays various messages on the Fileshare screen indicating that it has been initialized and the options with which it was started.

The Supervisor uses two indexed files to maintain information. One of the files maintains a list of user IDs and the other maintains the data set and record level locks. These files should be deleted before starting the Fileshare server with the Supervisor. The filenames are:

Option	Decscription
/D	Indicates the name of the database reference file. Any valid filename can be specified
/F	Must be Imssuper.dat and indicates the filename processed by the virtual file handler
/AP	Must be IMSSUPER as the name of the virtual file handler
/O	Value "LT" indicates that I/O requests to this file are not to be logged and are not part of a transaction. Logging and transaction processing must not be specified for the Supervisor program.

2.10.1 Supervisor Configuration

The Supervisor configuration settings are set as environment variables or are placed in a Supervisor configuration file. Configuration takes place on the machine where the Fileshare server is running, before the Fileshare server is started. When the first database call is made to a Fileshare database, the Supervisor displays the start-up configuration values. For Fileshare servers running on Windows, you can use either the environment variable settings or the configuration file. For Novell, AIX and UNIX Fileshare servers, you must use the configuration file.

The configuration file is named imssuper.cfg and must be in the current folder. That is, the folder used when starting the Fileshare server. It is a line sequential text file which contains one setting on each line. The setting on each line is the same value as the SET statement but without the word SET. For example, to set the IMSGO value to 1 using the configuration file, put IMSGO=1 on a line by itself. The file values take precedence over any environment variable settings.

To change the default GO PROCOPT processing, you set the IMSGO option to 1. A value of 2 is the default.

An IMSLOCK option is also available. It controls whether the Supervisor's locks are maintained on disk or in memory. This option is described in the section Supervisor Performance.

The IMSMSGS option controls the output of IMS Supervisor messages. The default is for messages to be displayed on the screen. This option redirects all IMS Supervisor messages to a file. This includes initialization, informational and error messages. This setting must be placed in the imssuper.cfg file; it is not recognized when set in an environment variable. It should also be the first setting in imssuper.cfg. The syntax of this setting is IMSMSGS=filename. For example:

If IMS Supervisor is unable to create the named file, it creates a file named imssuper.lst in the current folder.

2.10.2 Supervisor Performance

The Supervisor can maintain the data set and record locks in an indexed file or in memory. By default, all locks are placed in a indexed data set residing on disk. Having the locks in a disk data set enables support for a very large number of locks with minimal memory consumption. But, it can take a long time to perform the I/O to the disk, depending on your system load, disk performance and other factors.

You can have the locks maintained in memory, specifically in a Micro Focus Run-time Heap. Maintaining the locks in memory can provide better performance than the disk data set, if there is enough memory to contain all the locks. If the memory consumed by the lock table causes your system to begin swapping, total system performance is reduced. Many servers have very fast disk drives, therefore there may be very little performance advantage with locks in memory with very fast systems. You will need to experiment to determine your best settings.

The amount of memory consumed is equal to the amount of memory per lock, (see KEYSIZE below) multiplied by the maximum number of active locks held simultaneously. The memory used by released locks is reused for subsequent lock requests but the size of the Heap is never decreased. That is, once the memory is requested by the operating system, it is not released until Fileshare is terminated. However, new memory may not be allocated for a new lock if it can be placed in memory which was held by a released lock. It is difficult to know how many locks are actually held: you should use your operating system to determine the memory usage of tasks and swapping activity to determine the impact.

An IMSLOCK configuration option is used to control which kind of lock table is used.

Configuration Option	Description
dataset-name	Any valid operating system data set name. It indicates the name of the data set when the lock table is on disk. The default is a disk data set lock table named imslock.dat in the current folder. You can use this setting to change the name of the lock data set or to place it in a different location.
/HEAP	Indicates the lock table is kept in memory, in a Heap
KEYSIZE:xxx	Is an optional parameter which can be set to the length of the largest database root key field length. The default is 256. The KEYSIZE plus 17 bytes is the amount of memory consumed by each outstanding database record lock. Considerable memory can be saved by setting this value lower if possible. For example, if none of your databases has a root key larger than 64, you would save 192 bytes per lock - for 2,000 locks that would be 384,000 bytes.

If you place these in the imssuper.cfg file, leave the SET off the statement, for example, IMSLOCK=/HEAP,KEYSIZE:50

Note: If you specify KEYSIZE, it must be followed by a colon. Any other character results in the IMSLOCK value being ignored and this is indicated during the Supervisor initialization.

2.11 Special Considerations and Restrictions

HDAM databases with non-unique roots are not supported as Fileshare databases. You must change the root segment to have a unique sequence field and re-load the database if you want this support. Non-unique HDAM root databases can be shared for read-only by using Shared read-only databases on the IMS page of theProject Settings dialog box.

You must use the IMS Option Supervisor for accessing secondary index databases as stand-alone databases. (You can use either lock strategy for alternate processing sequences). If the PCB for the stand-alone index indicates anything other than a GO processing option, it is automatically upgraded to an exclusive database lock.

You must use the IMS Option Supervisor for accessing logical databases which contain concatenated segment definitions or for physical databases which define a logical child segment.

You must use the IMS Option Supervisor to enable the IMSDBU utility to operate on Fileshare databases.

If you use DBUTIL to load Fileshare databases, DBUTIL issues a checkpoint call before inserting every 100th root segment.

Note: Generally we do not recommend loading or unloading a database as a Fileshare database. There can be a significant performance reduction caused by the network communications overhead, especially compared to using IMSDBU for exclusive use databases. Also, if you are using DBUTIL and do not specify processing option L, there can be a significant amount of locking activity. If disk space allows, we recommend loading the database as a local drive exclusive use database and then copying or moving the database data files to the Fileshare server machine. If disk space does not allow, you could still set Exclusive use databases to the network server's drive and folder and load it. This incurs some network overhead but is not as severe as loading it as a Fileshare database.

A deadly embrace occurs when two users access the same two databases in reverse order with mutually prohibitive processing options.A deadly embrace is not detected for database records. When using the Supervisor, deadly embraces are detected at the database level (exclusive versus non-exclusive use). A deadly embrace "victim" is not automatically chosen: instead you choose it yourself. The "in use window" which appears for a locked resource contains an extra line indicating the deadly embrace condition and notifies you that your request may never complete.We recommend that you set unique user ID values so that you can identify the person with whom you are locked. Three-way (or more) deadly embraces are not detected by the Supervisor.

The "Q" SSA command code is ignored for determining the type and duration of lock. The lock set by a call with the Q command code is set based on the processing options and the locking strategy used. It is released if the database record is not updated and is not needed for database position. The DEQ call is accepted but has no effect other than appearing in the DL/I call trace, if tracing is enabled.