Best Practices to Optimize Channel Performance

This topic compares active channels, reports, query viewers, and trends in terms of goals and optimal resources for various use cases.

Active Channels or Reports?

Active channels are the better choice if you would rather see results streaming in as the queries proceed, rather than wait for the results to appear in one view in a report.

However, if speed of results is your goal, you might want to run a report instead. The total completion time of an equivalent report would be faster than the total time it takes for the channel to load 100%. This is because the active channel runs multiple smaller queries instead of one large query to display initial results quicker.

See also Queries, Building Trends, and Understanding the Reporting Workflow; and “Query and Trend Performance Tuning” in the ESM Administrator’s Guide.

Active Channels or Query Viewers?

Query Viewers behave more like reports in that they issue a single query against the database and return all results in one batch instead of the streaming progression of results from an active channel. Query viewers are most suitable if you have to slice and dice these query results further, for example, by changing the sort columns, changing types of charts/grid, and so on. These operations are performed on the client side with the results of the already-executed query. If you were using active channels instead, these types of changes would result in a re-running the query as many times as you sort columns.

Active Channel Query Time Ranges

Take note of the query time range in your active channels. The more hours you are querying, the slower the results are to load. An active channel shows results in minutes if you are querying a few hours of data. But the channel might start taking several hours to query larger time ranges that span more than 24 hours of data.

If you are querying over more than a day's worth of data, we recommend running a report (using queries and trends) or a query viewer instead of using an active channel.

Active Channel Filters

The more filter conditions you define in an active channel, the more work the channel has to do in the database to evaluate the conditions. A channel that does not have any filter conditions loads data fastest. (This does not mean that the query will run on all events in the database. Only a subset of events are queried, based upon the page you are looking at in the channel.)

If you must use a filter, try to make the filter as restrictive as possible.

Filtering on Indexed Fields

In the ESM, all fields are indexed. There may be search features that can only search a subset of fields, such as query viewer or active channel events, but there are no restrictions based on indexing, since all fields are indexed.

Filtering on Join Fields

The ESM event schema consists of a main arc_event table and several other tables. These other tables hold fields related to Annotation, Device fields, Agent fields, Resource References, and so on. If your query has a filter condition on a join field, the resulting channel would have to do more work to evaluate the field.

Continuously Updating Time Parameters

A channel that is “live” (querying against a moving time window and continuously updating the query time ranges) has to do more work than a channel based on fixed time windows. Performance will be better and faster on a channel with a fixed time window than on a live channel. See also Use of the “Live” Channel from Standard Content for a similar example.

Sorting by End Time or Manager Receipt Time

By default, all fields can be sorted on the active channel. This sometimes causes performance issues. You can set a property on the Console so that sorting is restricted to the End Time or Manager Receipt Time. If you want sorting on those fields only, follow these steps.
1. If you have active channels that already sort on fields other than ET or MRT, remove sorting on those other fields manually from the channels. Remove sorting by editing the channel, going to the Sort Fields tab, and change the sorting to either End Time or Manager Receipt Time.
2. Stop the Console.
3. Add this property to the console.properties file in the system where the Console is installed (for example, C:\arcsight\console\current):
  
  console.ui.channel.disable.sorting=true
  
  For more infomation about ESM properties files, refer to the section, "Managing and Changing Properties File Settings" in the ESM Administrator's Guide.
4. Restart the Console.
Avoid creating channels that are based on one time field but sorted on a different time field. A common cause of poor channel performance is user-created channels with this configuration; that is, a channel based on End Time, but sorted on Manager Receipt Time (or the reverse).

Note: When an event arrives at the Manager so late, that is, beyond its retention period, the Manager adjusts the event's time range so that the event is persisted in the second-to-oldest retained partition. The event is stored in the second-to-oldest partition because the oldest partition may be purged or archived anytime (such as during the data transfer).

This behavior changes the event start and end times, which could cause correlation issues, but the chances of an event being delayed for longer than the retention period are low.

Sorting in Active Channels

By default, the channel has a sort order based upon the time field that was used for creating the channel (End Time or Manager Receipt Time). Note that the sorting operation is done in the database query, so every time you change sort by on any column in your open active channel, ESM has to re-create the complete channel.

The sorting operation can be resource-intensive, especially when millions of events match your filter conditions. Avoid sorting if your filter conditions are not restrictive. For example, the base channel with no filter conditions is normally fastest to load, but it would become the slowest to load if you change its default time based sort order.

Tip: You can use a query viewer instead, that does sorting on the client side with the data it has already queried.

Use of the “Live” Channel from Standard Content

If you are using /All Active Channels/ArcSight System/Core/Live or any similar channel, be aware that the performance of that channel is slower because it has several complex joins (Joins with Annotations, Resource Reference, Device), and performs additional bit-wise operations to evaluate its filter conditions. Depending upon your specific use-case, you can simplify and create your own “Live” channel that is more efficient.

Case Sensitive or Case-Insensitive Conditions?

Wherever possible, use case-sensitive conditions. That will save the extra computation needed for TOUPPER operation required for case-insensitive matches.

I/O Subsystem Performance

Channel query performance is typically limited by the performance of the I/O subsystem on the database. The more events you are inserting, the more load it would cause on the I/O. SAN performance, RAID levels, I/O caches, and so on play a role in how much performance we can obtain.

Diagnostics: Start with Basic Channel Characteristics

To diagnose channel performance issues, start with the most basic active channel to see whether it meets your performance needs, and then keep refining/expanding to come to a point where you can tell what change is affecting performance. Start with the most basic active channel that has the following characteristics:

Based upon End Time
No filter conditions (also, make sure to run as an administrator user so that there is no access control filter)
Query time is two hours ago to Now
No continuous updates of time parameters

With the above basic active channel, you should see less than a minute wait in starting the channel and doing random scrolls in the channel.