Developing Custom Parsers for Syslog SmartConnectors

This section provides general instructions on developing a custom parser for a Syslog SmartConnector with the help of the Regex Tool. This section assumes that you are aware of Syslog SmartConnectors and their basic and advanced configurations.

Before you proceed, ensure that the following are present:

• A Raw Syslog SmartConnector to generate raw syslog events that serve as the input to the Regex tool.

• The Regex tool to create a custom parser for the Syslog SmartConnector. You can get the Regex tool by installing a Test Alert Connector on your desktop machine or on another machine where you have GUI access.

• A Syslog SmartConnector with which the custom parser will be tested.

To create a custom parser for a Syslog SmartConnector:

1. Run the Raw Syslog SmartConnector to receive raw syslog events from the syslog server or servers.

2. Extract the raw syslog from the Raw Syslog destination specified in the Raw Syslog SmartConnector configuration, and then save a sample of the log in a desired location on the machine where the Regex tool has been installed.

   Important: Consider the following:

   • Do not pull the raw syslog events from Logger, because it does not format the output correctly.

   • If you are extracting the raw syslog events from ESM, then you need to open the raw syslog events file and do the following:

   1. Remove double quotes (") at the beginning of the lines. The command in vi is :s/^"//
   2. Remove double quotes (") at the end of the lines. The command in vi is :s/"$//
   3. Replace adjacent instances of double quotes (") with single quotes ('). The command in vi is :s/""/"/g

3. Run the Regex tool present in the following location: $ARCSIGHT_HOME\current\bin\arcsight regex

4. Select File > Load Log File and browse to the location where you saved the sample log file in step 2, select the file, and then load it. The first line of the file is displayed in the Message field, and the number of lines in the file is displayed on the window title bar. The tool can hold up to 5000 lines from the file.

5. Select Options > Treat as Syslog SubAgent. The Regex tool will automatically detect the syslog header if the header is in the correct format (that is, timestamp hostname/hostIP).

   If the header is not in the correct format, do any of the following:

   1. Set the syslog.headers.ip value in the agent.properties file to parse the timestamp and hostname/IP address from the header. You can copy the default value from $ARCSIGHT_HOME\current\config\agent\agent.defaults.properties to $ARCSIGHT_HOME\current\user\agent\agent.properties, and modify the value to match the header.

   2. If the events do not have a syslog header, then just parse the whole line in your flex parser.

6. Select File > New FlexAgent Regex File to create a new parser.

7. Specify the name of the parser as myProduct_syslog.subagent.sdkrfilereader.properties, where myProduct is the device for which you are creating a new parser, and then complete the parser configuration. For a list of the syslog tokens that can be used in the parser for mapping to event fields, see ArcSight Built-in Tokens.

8. On the machine where the Syslog SmartConnector is present, copy the parser you created to the following folder: $ARCSIGHT_HOME\current\user\agent\flexagent\syslog

9. Stop the connector if it is already running, and then delete the syslog.properties file. This step ensures that any associations between the new events and other parsers, such as generic_syslog, are removed.

   Note: You need not save a copy of this file because the connector rebuilds it when the connector is run again.

10. (Conditional) To modify any configurations of the connector before you start it again to test the new parser, see the Managing SmartConnector Configurations section in the ArcSight SmartConnector Installation and User Guide.

11. Start the connector and send the events to the events' syslog listener.

12. View the results in ESM or Logger and verify that the events are being parsed properly.