Default Content Tab Configuration Fields

Batching

SmartConnectors can batch events to increase performance and optimize network bandwidth. When activated, SmartConnectors create blocks of events and send them when they either (1) reach a certain size or (2) the time window expires. You can also prioritize batches by severity, forcing the SmartConnector to send the highest-severity event batches first and the lowest-severity event batches later.

Enable Batching (per event)

Create batches of events of this specified size (50, 100, 200, 300, 400, 500, or 600 events). The default is 100.

Caution: You could potentially lose data with batch sizes 500 and 600. Contact Customer Support before using 500 or 600 batch size.

Enable Batching (in seconds)

The SmartConnector sends the events if this time window expires (1, 5, 10, 15, 30, 60).

Batch By

This is Time Based if the SmartConnector should send batches as they arrive (the default) or Severity Based if the SmartConnector should send batches based on severity (batches of Highest Severity events sent first).

Time Correction

The settings in this group provide several ways to fix problems with devices that do not report the time correctly.

Use Connector Time as Device Time

(No | Yes) Override the time the device reports and instead use the time at which the connector received the event. This option assumes that the connector is more likely to report the correct time.

The default is No.

Enable Device Time Correction (in seconds)

The SmartConnector can adjust the time reported by the device Detect Time, using this setting. This is useful when a remote device's clock isn't synchronized with the ArcSight Manager. This should be a temporary setting. The recommended way to synchronize clocks between Manager and devices is the NTP protocol.

Enable Connector Time Correction (in seconds)

The SmartConnector can also adjust the time reported by the Connector Time SmartConnector itself, using this setting. This is for informational purposes only and allows you to modify the local time on the SmartConnector. This should be a temporary setting. The recommended way to synchronize clocks between Manager and SmartConnectors is the NTP protocol.

Set Device Time Zone To

(Disabled | <TimeZone>) (Default is Disabled) Ordinarily, it is presumed that the original device is reporting its time zone along with its time. And if not, it is then presumed that the SmartConnector is doing so. If this is not true, or the device isn't reporting correctly, you can switch this option from Disabled to GMT or to a particular world time zone. That zone is then applied to the time reported.

Refer to the SmartConnector User's Guide for the updated list of zones.

Device Time Auto-correction

These are the time span and frequency factors for doing device-time auto-correction.

Future Threshold

The connector auto-corrects if the detect time is greater than the connector time by Future Threshold seconds.

Past Threshold

The connector auto-corrects if the detect time is earlier than the connector time by Past Threshold seconds.

Device List

A comma-separated list of the devices to which the thresholds apply. The default, (ALL) means all devices.

Time Checking

Future Threshold

The number of seconds by which to extend the connector's forward threshold for time checking. Default is 5 minutes (300 seconds).

Past Threshold

The number of seconds by which to extend the connector's rear threshold for time checking. Default is 1 hour (3600 seconds).

Frequency

The SmartConnector checks its future and past thresholds at intervals specified by this number of seconds. Default is 1 minute (60 seconds).

Cache

Changing these settings does not affect the events cached, it only affects new events sent to the cache.

Cache Size

SmartConnectors use a compressed disk cache to hold large volumes of events when the ArcSight Manager is down or when the SmartConnector receives bursts of events. This parameter specifies the disk space to use. The default is 1 GB which, depending on the connector, can hold about 15 million events, but it also can go down to 5 MB. When this disk space is full, the SmartConnector drops the oldest events to free up disk cache space. (5 MB, 50 MB, 100 MB, 200 MB, 250 MB, 500 MB, 1 GB, 2.5 GB, 5 GB, 10 GB, 50 GB.)

Notification Threshold

The size of the cache's contents at which to trigger a notification. Default is 10,000.

Notification Frequency

How often to send notifications when the Notification Threshold is reached. (1 min, 5 min, 10 min, 30 min, 60 min.)

Network

Heartbeat Frequency

This setting controls how often the connector sends a heartbeat message to the ArcSight Manager. The default is 5 seconds, but it can go from 5 seconds to 10 minutes. Note that the heartbeat is also used to communicate with the SmartConnector; therefore, if its frequency is set to 10 minutes, then it could take as much as 10 minutes to send any configuration information or commands back to the SmartConnector.

Enable Name Resolution

(No | Source/Dest only | Yes) The SmartConnector tries to resolve IP addresses to host names, and host names to IP addresses, if the event rate allows it and if required. This setting controls this functionality. The Source, Target and Device IP addresses and Hostnames may also be affected by this setting. The Source/Dest Only choice means that the device address and device host name fields are ignored for name resolution. (The default is Yes.)

• IPv4 Only for Legacy Events (default)
• IPv6 (Prefer IPv4 for reverse resolution) for Legacy Events
• IPv6 (Prefer IPv6 for reverse resolution) for Legacy Events

Name Resolution TTL (secs)

This is the amount of time (Time to Live) the name resolution is to be in effect. The name resolution entries are cached for this time (default is 3600).

Wait For Name Resolution

(Yes | No) If set to Yes, the SmartConnector waits for name resolution to be completed. When Yes is selected, event processing might be slowed down significantly and even cause lost events. (default is No).

Name Resolution Host Name Options

• Set host name only (default)
• Set host name only (lowercase)
• Set host and domain names
• Set host and domain names (lowercase)

For reverse resolution (IP Address to Host name), only the host name field is set. If host name only is not used, the host name is split up and put into both the DNS domain and the host name fields. This affects the source, destination, device and agent address.

Name Resolution Domain from Email

(Yes | No) If set to Yes, the host name and DNS domain fields are empty, and the corresponding user name field appears as an e-mail address, then the domain from the e-mail address is put in the DNS domain field. This only affects the source and destination fields (default is Yes).

Clear Host Names Same as IP Address

(Yes | No) If set to Yes and the host name field is set to an IP Address that matches the corresponding IP Address field, then the host name field is cleared. This affects the source, destination, and device fields (default is Yes).

Set Host Names to IP Addresses When Unknown

(Yes | No) If set to Yes, host names that remain unresolved are set to IP addresses (default is No).

Don’t Resolve Host Names Matching

By default, host names are resolved to their IP addresses. You have the option to specify a regular expression for all or part of a host name for which you do not want the system to attempt host name resolution to an IP address.

When this option is configured, the system cannot resolve host names matching this expression.

Don’t Reverse-Resolve IP Ranges

By default, IP addresses are resolved to their domain names. You have the option to specify IP address ranges for which you do not want the system to attempt reverse-resolution to domain names.

Click in the field to enter the IP address range. To enter a single IP address, enter the address under the From column and leave the To column blank, then click Apply. For an address range, enter the starting IP address under From and the ending address under To, then click Apply. This field allows you to enter a list of ranges.

When this option is configured, the system cannot reverse-resolve IP addresses that fall within any of the specified ranges.

Remove Unresolvable Names/IPs from Cache

(Yes | Yes (w/ negative cache) | No) If set to No, unresolvable host names or IP addresses continue to be in the cache. If set to Yes, unresolvable host names or IP addresses are removed from the cache. If set to Yes (w/negative cache), the connector remembers what names/IPs have been unresolvable so that time is not wasted trying to resolve them frequently.

Limit Bandwidth To

Default is Disabled. A list of bandwidth options you can use to constrain the connector's output over the network. (Disabled, 1 kbit/sec to 10 Mbits/sec.)

Transport Mode

You can configure the SmartConnector to cache to disk all the processed events it receives. This is equivalent to pausing the SmartConnector. However, you can use this setting to delay event-sending during particular time periods. For example, you could use this setting to cache events during the day and send them at night. You can also set the connector to cache all events, except for those marked with a very-high severity, during business hours, and send the rest at night. (Normal | Cache | Cache but send Very High severity events).

Cache Mode

(Normal | Drop if Dest Down) This option is meant to be used on a primary destination to control the caching behavior of the primary destination when it is down, and the connector starts sending events to the failover destination. In the Normal mode, events are cached and sent to the primary destination when it comes back up. In the Drop if Dest Down mode, the events are not cached and dropped and therefore not sent to the primary destination when it becomes available again (default is Normal).

If specified in setup or ArcMC, this is a comma-separated list that must contain a multiple of three items. The first of each three is the starting IP address of a zone, the second is the ending IP address of the zone, and the third is the URI of the zone to assign to addresses in that range. These zones are only used if a network model has not been sent by ESM or ArcMC, or if that network model does not cover some addresses. If Address-Based Zone Population Defaults Enabled is set to Yes, the zones specified here take precedence over those.

For example for two zones this could be: 15.0.0.0,15.255.255.255,/All Zones/ArcSight System/Public Address Space Zones/Hewlett-Packard Company,17.0.0.0,17.255.255.255,/All Zones/ArcSight System/Public Address Space Zones/Apple Computer Inc.

Zone Population Mode

(Normal | Rezone (override) | No Zoning (clear)) Setting to Normal means zones are computed and assigned, if not already set. Rezone (override) re-computes and re-assigns already populated zones. No Zoning (clear) clears the zones, if already populated. (default is Normal).

Customer URI

Applies the given customer URI to events emanating from the connector. Provided the customer resource exists, all customer fields are populated on the ArcSight Manager. If this particular connector is reporting data that might apply to more than one customer, you can use Velocity templates in this field to conditionally identify those customers.

Field Based Aggregation

This feature is an extension of basic connector aggregation. Basic aggregation aggregates two events if, and only if, the fields of the two events are the same per the following fields:

• Agent ID

• Name

• Device event category

• Agent severity

• Destination address

• Destination user ID

• Destination port

• Request URL

• Source address

• Source user ID

• Source port

• Destination process name

• Transport protocol

• Application protocol

• Device inbound interface

• Device outbound interface

• Additional data (if any)

• Base event IDs (if any)

Field-based aggregation implements a more flexible aggregation mechanism; two events are aggregated if only the selected fields are the same for both events.

Note: Field-based aggregation creates a new alert that contains only the fields that were specified, so the rest of the fields are ignored, unless “Preserve Common Fields” is set to “Yes”.

Field-based aggregation offers several advantages over basic aggregation, including:

• Control over what fields to aggregate on

• Start and end time set to the earliest start time and latest end time, respectively (instead of taking the values from the first event in the group, like basic aggregation)

• Option to preserve common fields

• Option to sum one or more numeric fields

SmartConnector aggregation significantly reduces the amount of data received, and should be applied only when you use less than the total amount of information the event offers. For example, you could enable field-based aggregation to aggregate "accepts" and "rejects" in a firewall, but you should use it only if you are interested in the count of these events, instead of all the information provided by the firewall.

Time Interval

Choose a time interval, if applicable, to use as a basis for aggregating the events the connector collects. Aggregation time interval and threshold settings need to both be set in order for the aggregation to be enabled.

(Disabled, 1 sec, 5 sec, and so on, up to 1 hour.)

Event Threshold

Choose a number of events, if applicable, to use as a basis for aggregating the events the connector collects. This is the maximum count of events that can be aggregated; for example, if 150 events were found to be the same within the time interval selected (i.e., contained the same selected fields) and you select an event threshold of 100, you then receive two events, one of count 100 and another of count 50. This option is exclusive of Time Interval. (Disabled, 10 events, 50 events, and so on, up to 10,000 events.)

Field Names

Choose one or more fields, if applicable, to use as the basis for aggregating the events the connector collects. Use Ctrl+click to select multiple fields. The result is a comma-separated list of fields to monitor. For example, "eventName,deviceHostName" would aggregate events if they have the same event- and device-host names. You can use any of the event fields displayed in the event inspector; the name can contain no spaces and the first letter should not be capitalized.

Fields to Sum

Choose one or more fields, if applicable, to use as the basis for aggregating the events the connector collects.

If specified, this set of numeric fields is summed rather than aggregated, preserved, or discarded. The most common fields to sum are bytesIn and bytesOut. Note that if any of the fields listed here are also in the list of field names to aggregate, they are aggregated and not summed.

Preserve Common Fields

(Yes | No) Choosing Yes adds fields to the aggregated event if they have the same values for each event. Choosing No, the default, ignores non-aggregated fields in aggregated events.

Filter Aggregation

Filter Aggregation is a way of capturing aggregated event data from events that would otherwise be discarded due to an agent filter. Only events that would be filtered out are considered for filter aggregation (unlike Field-based aggregation, which looks at all events).

Time Interval

Choose a time interval, if applicable, to use as a basis for aggregating the events the connector collects. It is exclusive of Event Threshold. (Disabled, 1 sec, 5 sec, and so on, up to 1 hour.)

Event Threshold

Choose a number of events, if applicable, to use as a basis for aggregating the events the connector collects. This is the maximum count of events that can be aggregated; for example, if 150 events were found to be the same within the time interval selected (that is, contained the same selected fields) and you select an event threshold of 100, you then receive two events, one of count 100 and another of count 50. This option is exclusive of Time Interval. (Disabled, 10 events, 50 events, and so on, up to 10,000 events.)

Fields to Sum

(Optional) Choose one or more fields, if applicable, to use as the basis for aggregating the events the connector collects.

Processing

Preserve Raw Event

(Yes | No) Some devices contain a raw event that can be captured as part of the generated alert. If that is not the case, most connectors can also produce a serialized version of the data stream that was parsed/processed to generate the ArcSight event. This feature allows the connector to preserve this serialized "raw event" as a field in the event inspector. This feature is disabled, by default, since using raw data increases the event size and therefore requires more database storage space.

You can enable this by changing the Preserve Raw Event setting. If you choose Yes, the serialized representation of the "Raw Event" is sent to the selected destination and preserved in the Raw Event field.

Turbo Mode

If your configuration, reporting, and analytic usage permits, you can greatly accelerate the transfer of a sensor's event information through SmartConnectors by choosing one of two "turbo" (narrower data bandwidth) modes.

Complete is the default transfer mode, which passes all the data arriving from the device, including any additional data (custom, or vendor-specific). This corresponds to turbo.enabled=false on the Manger. Since this value is not the default, be sure to add this property to the Manager’s server.properties file.

The first level of Turbo acceleration is called Faster and drops just additional data, while retaining all other information. The Fastest mode eliminates all but a core set of event attributes, in order to achieve the best throughput. Consider the possible effects such a restricted data set might have from a given device (for example, on reports, rules, threat resolution) before selecting it.

The specific event attributes that apply to these modes in your enterprise are defined in the self-documented $ARCSIGHT_HOME/config/connector/agent.properties file for the ArcSight Manager. Because these properties may have been adjusted for your needs, you should refer to this file for definitive lists.

Only scanner SmartConnectors must run in Complete mode, to capture the additional data.

Note: SmartConnector Turbo Modes are superseded by the Turbo Mode in use by the ArcSight Managers processing their events. For example, a Manager set to Faster cannot pass all the data possible for a SmartConnector that is set for the default of Complete.

Enable Aggregation (in secs)

Note: If you have already used this feature for setting up previous SmartConnectors, you can continue to do so. However, ArcSight recommends that you use the new Field Based Aggregation feature as a more flexible option.

Here is the description of the legacy “Enable Aggregation” feature, for those who are still using it:

When enabled, Enable Aggregation (in seconds) aggregates two or more events on the basis of the selected time value. (Disabled, 1, 2, 3, 4, 5, 10, 30, 60)

The aggregation is performed on one or more matches for a fixed subset of fields:

• Agent ID

• Name

• Device event category

• Agent severity

• Destination address

• Destination user ID

• Destination port

• Request URL

• Source address

• Source user ID

• Source port

• Destination process name

• Transport protocol

• Application protocol

• Device inbound interface

• Device outbound interface

• Additional data (if any)

• Base event IDs (if any)

The aggregated event shows the event count (how many events were aggregated into the displayed event) and event type. The rest of the fields in the aggregated event take the values of the first event in the set of aggregated events.

Limit Event Processing Rate

You can moderate the SmartConnector's burden on the CPU by reducing its processing rate. This can also be a means of dealing with the effects of event bursts.

The choices range from Disabled (no limitation on CPU demand) to 1 eps (pass just one event per second, making the smallest demand on the CPU).

Be sure to note that this option's effect varies with the category of SmartConnector in use, as described in the SmartConnector Processing Categories table below.

Fields to Obfuscate

Using MD5 hashing, this option allows you to specify a list of fields for obfuscation in a security event.

Store Original Time In

This parameter allows you to move the original device receipt time to a specified field if altered by the time correction.

Enable Port-Service Mapping

If Enabled and one of the two fields destination port and application protocol is set, and the other is not, the one that is set is used to set the other. For example, if the destination port is 22 and application protocol is not set, then the application protocol is set to ssh.

Default is Disabled.

Uppercase User Names

(Disabled | Enabled)

Default is Disabled. If set to any of the enabled settings, the two user name fields are automatically changed to uppercase.

The original values are saved as follows:

• Enabled (orig to ID) saves the original values to the sourceUserID and destinationUserID fields, respectively, overwriting any values that may have been there previously.

• Enabled (orig to ID or Flex) saves the original values in the same fields if they do not already contain values, or to the flexString1 (source) and flexString2 (destination) fields if the ID fields do contain values.

• Enabled (orig to Add. Data) saves the original values to additional data fields called OrigSrcUsrName and OrigDstUsrName, respectively.

Note: The uppercase operation is typically done using the default Locale for the chosen platform. You can set this to a particular Locale by setting the connector.uppercase.user.name.locale property in agent.properties to the desired Locale (using "en_US" for U.S. English, for example).

Enable User Name Splitting

(Yes | No) If this is set to yes and the destination user name contains commas in the event, this parameter duplicates that event. Each user name in the list is placed in one of the events.

For example, if the destination user name in an event is “User 123, User 456”, then that event is sent twice, with the destination user name set to “User 123” in the first and “User 456” in the second.

Default is No.

Split File Name into Path and Name

(Yes | No) If this is set to yes and an event’s file name field is set but its file path field is not, this parameter splits the file name into a path and a name, placing each part into appropriate fields.

For example, if the file name field is set to C:\dir\file.ext and the file path is not set, then the file path is set to C:\dir and the file name to file.ext. The separator character can be either \ or / as the system looks to the SmartConnector to determine its platform.

Default is No.

Event Integrity Algorithm

(Disabled | SHA-256 | SHA-1 | MD5 | SHA-512)

If this is set to one of the algorithms (such as SHA-256), and the Preserve Raw Event parameter is Enabled, then additional event integrity internal events are generated, normally at a rate of about 1 per 50 normal events.

The crypto signature field is also set in each event in the format: "#seq(alg):digest", where seq is a persistent event sequence number, alg is the message digest algorithm, and digest is the hexadecimal message digest.

These extra events and the crypto signature field values can be used to verify that no events were tampered with after generation.

Supported algorithms are: SHA-256, SHA-1, MD5, and SHA-512.

Default is Disabled (that is, no algorithm is applied).

Generate Unparsed Events

(Yes | No) If set to yes and some incoming event data cannot be parsed (perhaps because a device has been upgraded since the SmartConnector parser was written), then a special event named “Unparsed Event” is generated. The raw event appears in the event message field.

If set to No, the SmartConnector log files indicate the unparsed events.

Default is No.

Preserve System Health Events

(Yes | No) If set to yes, internal system health events are preserved.

SmartConnectors generate system health events that provide information about the systems on which they are installed (for example, disk usage, network memory, JVM memory, percentage of processing of CPU memory usage, and so forth). By default, these events are not retained or passed on to ArcSight destinations and, therefore, not available for viewing. Setting this option to yes makes them available in the Console.

Enable Device Status Monitoring (in millisec)

(<NumberOfMilliseconds> | -1 (disabled))

If set to a <NumberOfMilliseconds>, the selected SmartConnector generates internal events periodically 1 minute (60000 milliseconds) or greater with the status of the devices for which the connector is receiving normal events. These events have the name "Connector Device Status."

Enabling periodic device status monitoring events helps monitor both the SmartConnector and device uptime.

Device status monitoring events include this information, if available:

• Event name (Connector Device Status)

• Vendor and Product information

• Source Address and Host Name

• Zone

• Last event received

• Total number of events for the device since the connector started

• Event count since last call

Device status monitoring events can be set to generate every 1 minute (60000 milliseconds), or less frequently (that is, a greater number of milliseconds than the minimum).

If you specify less than 60000, you get a warning in the log that the minimum is 60000 milliseconds (1 minute) and the system uses the minimum.

If you enter a non-number in the field, it generates an error in the log that the value could not be parsed. In this case, the feature is disabled (and logged as such).

In such cases, there is no indication on the Console that anything went wrong because there is no way for the Connector to convey that error.

Payload Sampling (when available)

Some SmartConnectors use Payload sampling to send a portion of packet payload (as opposed to the complete payload) along with the original event. This portion is retrieved using the on-demand payload retrieval in the event inspector.

Maximum Length

You can configure the maximum length of the payload sample using the following values:

• Discard

• 128 bytes

• 256 bytes

• 512 bytes

• 1 Kbyte

When the Discard option is chosen, no payload sample is sent inside the original event.

Mask Non-printable Characters

This feature allows you to mask the non-printable characters in the payload sample.