How to Create and Test Jobs

The procedures in this section are geared to users who are members of the default File Transfer Administrators group, or other groups with the Manage jobs role enabled. These procedures assume that initial configuration has already been completed by a Reflection Gateway administrator.

Connect to Gateway Administrator

Gateway Administrator is a web-based application that you can use to manage users and Jobs.

Before you begin

Know the address of the server running Reflection Gateway Administrator
Have login credentials for an account that is a member of the File Transfer Administrators group, the Administrators group, or any group with the Manage jobs role enabled.

To connect to Gateway Administrator

From a web browser, enter the following URL, replacing with the name or IP address of your server:

https://<gateway_administrator_host>:9490

Note

You might see a certificate warning message before you see the login page. This warning shows up if the Gateway Administrator is still using the default self-signed security certificate that installs with Reflection Gateway. This message is not displayed if the Gateway Administrator server has been configured to use a certificate signed by the Certificate Authority (CA) that is in your browser's list of trusted CAs.

Create a Job

To create a Job, you designate a server and directory to scan, set a scan interval, and optionally set filters to determine which files in the directory should trigger the actions you define in the Actions list. When Reflection Gateway detects new or updated files in the scanned directory that meet the filter conditions, it executes your defined Job actions.

Note that you can configure one or more actions for each Job. Reflection Gateway executes actions in the order listed. If an action fails:

If a Failure email is configured, the email is sent to specified recipients.
No subsequent actions are processed.

Before you begin

To create a Job, you must be a member of the File Transfer Administrators group, the Administrators group, or any group with the Manage jobs role enabled.
Initial configuration of SFTP servers, Hubs, and email should be complete. This must be done by a user with System setup privileges.

To create a new Job

Log in to Gateway Administrator.
Go to Jobs > New.
For Name, enter a descriptive name for the Job.
Leave Disabled selected for initial testing. You can use the Run Now feature to test disabled Jobs. After you are satisfied that the Job runs as expected, select Enabled to run the Job at the specified scan interval.

Under Source Files, configure the following:


Server	Select the server you want to scan.
Directory	Browse to select the server on the directory to scan for new or updated files.
Recursive	(Optional) Select Recursive if you want the scan to include subdirectories of the specified directory.
Scan Interval	(Optional) By default the directory is scanned every 10 minutes. You can change the frequency or days for scanning.
Filters	(Optional) By default all files are scanned. You can filter by name, size, modified time. You can also configure a minimum number of files that must arrive before Job actions begin. Use the check box to invoke the Failure email notification if no files meet the filter conditions at the end of the scan interval.

Under Actions, from the Add action... drop-down list, select either Transfer file or Execute command. For details see:
- Configure a Job Action that Transfers Files
- Configure a Job Action that Executes a Command
(Optional) Under Success and Failure, click Change to configure email notifications.
Click Save.
Click Run Now to test. You'll see a Running Job message. You can close this message at any time; it has no effect on execution of Job actions.

Note

If you make modifications to your Job, ensure that there are new or updated files in the scanned directory before each test.
(Optional) Configure additional actions for this Job.
After your Run Now tests work as expected, click Enabled and save the Job to have the Job run automatically on the specified Scan Interval.

To copy an existing Job

You can copy existing Jobs to create new Jobs based on existing Jobs.

On the Jobs tab, select the Job you want to copy and click Copy. A renamed copy of the selected Job is added to your Jobs list.

Note

All copied Jobs are marked as Disabled initially. This ensures that you don't have identical Jobs running simultaneously.
Edit the copied Job and test it using Run Now.
After your Run Now tests work as expected, click Enabled and save the Job to have the Job run automatically on the specified Scan Interval.

More Information

Notes for Testing Job Actions
Job Troubleshooting

Configure a Job Action that Transfers Files

Use this procedure to configure automated file transfer between servers. The source server can be the scanned server or a server that received files from a previous action. The destination can be a different server or a different directory on the same server.

Create a Job and specify the scan server and directory.
On the New Job (or Edit Job) page, under Actions, from the Add action... list, select Transfer file.

This opens the Transfer File dialog box.
(Optional) Select Move if you want to delete files from the source location after the transfer.
(Optional) Select Preserve file attributes if you want the modification date and time on the destination server to match the source. When this option is not selected, the destination file's timestamp shows the date and time of the transfer.
Configure the source server:
- For From, select the source server from the list of available servers. If this is the initial transfer, select the scanned server.
- Enter an expression to describe the files to transfer. For most transfers, use \$SCANNED_PATH\$/\$RELATIVE_FILE_NAME\$ (the default) for the initial transfer. This will transfer all updated files on the scanned server that meet your filter criteria. If the scan is recursive, this option will also include all subdirectories in that directory. For subsequent transfers, replace $SCANNED_PATH$ with the correct actual path for the files on the server you are transferring from.
Configure the destination server:
- For To, select the destination server.
  
  Note
  
  If File Server groups have been configured by a Reflection Gateway administrator, the list of servers available to you is determined by your membership in one or more File Server groups.
- Enter an expression to identify the location and naming pattern you want to use for files on the destination server. The expression you enter for destination files can use a combination of text and tokens. For details, see Entering Expressions for Destination Files below.
Click OK to close the Transfer File dialog box.
Click Save.
Click Run Now to test the transfer. (If this is a repeat test, first update the files in the source location.)

Entering Expressions for Destination Files

The expression you enter for destination files in the Transfer File dialog box can use a combination of text and tokens. Refer to the examples below to understand how Reflection Gateway interprets these expressions. Note the following:

UserID in these descriptions is the user whose UserID is configured for authentication to the destination SFTP server. (See New/Edit SFTP Server.) Any destination location you specify must be accessible to this user.
If the destination server is a Reflection for Secure IT Server for Windows, specify destination paths using the virtual directory names that have been made accessible to UserID, not the actual physical directories on the server. (Virtual directory access is configured using the console's SFTP Directories pane.)
If the destination server is a Reflection for Secure IT Server for UNIX (or other UNIX server), specify actual physical paths; UNIX servers do not use virtual directories.
Use forward slashes (/) for specifying paths on both Windows and UNIX servers.

Destination Expression	Output on Destination Server
$RELATIVE_FILE_NAME$ (the default)	Files are created in UserID’s SFTP login directory. The token is replaced by the relative path and filename of each file on the scanned server. If the scan is recursive, subdirectories that don’t yet exist are created in the login directory. If the scan is not recursive, files in subdirectories are not transferred, and this token is equivalent to $FILENAME$.
$FILENAME$	Files are created in UserID’s SFTP login directory. The token is replaced by the filename of each file on the scanned server. If the scan is recursive, files are not transferred to subdirectories. All files–including those in subdirectories on the scanned server– are transferred to the SFTP login directory.
`upload/$RELATIVE_FILE_NAME$`	Files are created in a subdirectory called upload in UserID’s SFTP login directory. If this directory does not exist, it is created. If the scan is recursive, subdirectories are created in the upload directory.
`/upload/$RELATIVE_FILE_NAME$`	Files are created in a directory called upload in the root directory. UserID must have write access to this directory. In most cases, the upload directory must already exist: If the server is a Reflection for Secure IT Server for Windows, it is not possible for any user to create a new directory in the root directory. UserID must have access to a virtual directory called upload that is mapped to an existing physical directory. If the server is a UNIX server, creating a new directory at the root level is only possible if UserID has root privileges (not recommended).
`$JOB_ID$-$FILENAME$`	Files are created in UserID’s SFTP login directory. Filenames use the Job ID followed by a hyphen and the filename of the file on the scanned server. NOTE: Don’t prepend text or tokens if you use $RELATIVE_FILE_NAME$ with a recursive transfer. Doing this will change both filenames and directory names.

More Information

Configure a Job Action that Executes a Command

Job actions are triggered when Reflection Gateway detects new or updated files in a scanned directory. Use this procedure to invoke a command action. You can configure commands that run once or specify a command to act on each file that met the scan conditions.

Note

The command runs as a remote SSH command executed using the user account specified for UserID on the New/Edit SFTP Server page. The executable file and output locations must be available to this user. The working directory is the SSH terminal login directory for that user.

To configure a Job action that executes a command

Create a Job and specify the scan server and directory.
On the New Job (or Edit Job) page, under Actions, from the Add action... list, select Execute command.

This opens the Execute Command dialog box.
For Options, select one of the following:
- Run once executes the command a single time.
- Run for each file executes the command for each file that has been added or updated since the last scan.
For Server, select the SFTP server on which this command will run.
For Command, specify the command to run and any arguments you want to pass to this command. See Sample Commands for Job Actions below for examples.
- The command runs as a remote SSH command executed using the user account specified for UserID on the SFTP server setup page. The executable file and output locations must be available to this user. The working directory is the user's home directory.
- On Windows servers, precede DOS commands with cmd /c, for example:
  
  cmd /c echo $FILENAME$ >> c:\output\filelist.txt
- If you include full path information for output, as shown in the example above, the directories in the specified path must exist, and the user who is specified for UserID in the file server definition must have access to these directories.
- If you do not include full path information for command output, it is created in the terminal session login directory for the user who is specified for UserID in the file server definition. On UNIX systems this is typically the home directory (home/userID). On the Reflection for Secure IT Server for Windows, this is the user's Windows Profile directory by default (c:\users\userID). This default can be modified from the Reflection for Secure IT Server console using the Permissions pane.
Click OK to close the Execute command dialog box.
Click Save.
Click Run Now to test the Job. (If this is a repeat test, first update the files in the source location.)

Sample Commands for Job Actions

Use these examples as models for testing and configuring Jobs that execute commands. Samples include appropriate syntax for both Windows and UNIX servers.

Example 1: Run Once

This example uses the echo system command and the $RELATIVE_FILE_LIST$ token to send a list files on the scanned server that meet the criteria for the Job.

Options is set to Run once
Server can be the scanned server or a destination server. In this example, the output file (filelist.txt) is created in a directory called output in the root directory of the selected server. The user account specified for authenticating to this server must have write access to this directory.

Windows SFTP server

Command: cmd /c echo $RELATIVE_FILE_LIST$ >> c:\output\filelist.txt

UNIX SFTP server

Command: echo $RELATIVE_FILE_LIST$ >> /output/filelist.txt

Example 2: Run for each file

This example use a script to create a text file that lists token values for each transferred file. The action runs on the destination server after a transfer Job action has copied files to this server.

Files have been transferred to a directory called upload in the root directory of the destination server. The script is in a directory called script on the same server, and output of the script is created in the script directory. The user account specified for this SFTP server must have write access to both upload and script.
Options is set to Run for each file
Server is set to the destination server name.
File tokens are passed to the script as command line arguments. Because the returned values of these tokens can include spaces and special characters, the arguments are enclosed in double quotes.

Windows SFTP server

c:\script\tokenoutput.bat "$FILENAME$" "$FULL_PATH$" "$RELATIVE_FILE_NAME$"

Contents of tokenoutput.bat:

echo FILENAME:      %1 >> c:\script\tokenvalues.txt
echo FULL_PATH:     %2 >> c:\script\tokenvalues.txt
echo RELATIVE PATH: %3 >> c:\script\tokenvalues.txt
echo ================= >> c:\script\tokenvalues.txt

UNIX SFTP server

/script/tokenoutput.sh "$FILENAME$" "$FULL_PATH$" "$RELATIVE_FILE_NAME$"

Contents of tokenoutput.sh:

echo FILENAME:      $1 >> /script/tokenvalues.txt
echo FULL_PATH:     $2 >> /script/tokenvalues.txt
echo RELATIVE PATH: $3 >> /script/tokenvalues.txt
echo ================= >> /script/tokenvalues.txt

::: {.note} The script file must have execute permissions set. For example:

chmod +x tokenoutput.sh

More Information

Notes for Testing Job Actions

Note the following as you develop Job actions:

Use the Reflection Gateway Administrator and Reflection Hub server.log for troubleshooting. See Job Troubleshooting for additional troubleshooting help.
You can use the Run Now button regardless of State (Enabled or Disabled). This enables you to ensure that the Job actions perform as expected before setting up the scan interval and enabling the Job.
A Run Now test won't trigger Job actions if the scan directory contents have not changed since your last test. For testing repeatedly from the same directory, you can use the following commands to update the timestamp on all files in a directory:

On UNIX, from a terminal window:

touch *

On Windows, from a command window

copy *.* +,,
Use cmd /c before DOS commands to run on a Windows server. The /c switch specifies that cmd should exit after the specified command is carried out. For example

cmd /c echo $FILENAME$ >> filelist.txt
Job actions configured using Execute Command run on the specified server as a remote SSH command. The command is executed using the user account specified for the UserID of that server. This user must have access to all directories required by the command. When $RELATIVE_FILE_NAME$ or $FILENAME$ are used with no preceding path information, output is directed to the terminal login directory for that user. For more information see Set Up Directory Access on your SFTP Servers.
Job actions configured using Transfer File use the SFTP directory access for the UserID for the specified server. When $RELATIVE_FILE_NAME$ or $FILENAME$ are used with no preceding path, output is directed to the SFTP login directory for that user. For more information see Set Up Directory Access on your SFTP Servers.
If you modify settings on an SFTP Server that affect SFTP directory access, you may not see the effects of your changes when you rerun the Job. This is because the Hub maintains and reuses existing connections to SFTP servers for up to 3 minutes of inactivity. (This improves performance because a typical scan requires multiple connections to the scanned server. It also helps improve the performance of Job actions that transfer multiple files.) To ensure that your Job action reflects changes made to the SFTP server configuration, either restart the Reflection Hub service or wait at least 3 minutes after all active scans and Job actions are complete.
The Transfer Site base directory specified in the SFTP server definition is used for Transfer Sites only. It does not affect the output locations for Job actions.
The Actions tab in Gateway Administrator is used for configuring Post Transfer Actions only. These actions can be associated with Transfer Sites and run after successful uploads from a transfer client. They are not associated with Jobs. Job actions are configured using the Transfer File and Execute Command dialog boxes, which are accessible from the Jobs tab.