Top Value Counts Data Monitor
This data monitor type is a selection when you create a new data monitor. For information on how to create a data monitor, see Creating a Data Monitor.
The Top Value Counts data monitor displays top events by selected data fields, the total number of events, and the event Severity within the total number of events as defined by the filter (Restrict by Filter
parameter). Data is displayed in Table and BarChartTable viewer configurations.
Top Value Counts uses an aggregation mechanism that precisely and predictably controls the time dimension of the data being evaluated. "Bucketized" means that the monitor evaluates a specific number of time-based event data units of a certain size (buckets). As time increments forward, the evaluation refreshes, using the most recent set of qualifying buckets.
The data monitor's latest bucket process live data. You should expect some delay ranging from milliseconds to seconds between the Manager’s receipt of the event and when the event is processed by the data monitor. The latest bucket may therefore not have counted all the events up to the current millisecond. Eventually the count discrepancy is resolved and the bucket counts will be correct.
Parameter |
Description |
---|---|
Data Monitor Name |
Enter a data monitor name. |
Enable Data Monitor |
Select the check box to enable the data monitor and collect data from the Manager. If not selected, the associated viewer configuration will not display any data. Depending on the permissions associated with the user group to which you belong, you may or may not have an option to Enable (deploy) or disable (un-deploy) the data monitor. For more information, see Enabling or Disabling a Data Monitor. |
Restrict by Filter |
Specify a filter to focus on events that are of particular interest and to reduce the number of events the data monitor processes. Use a filter when the number of possible Aggregate Field values can exceed the maximum for # of Distinct Events. |
Availability Interval |
Sets the number of seconds to use as the interval between monitor updates. |
Select Field Set |
Specify a field set for use in data monitor drill-downs. When this data monitor is displayed, the user can double-click on a chart area or table row that represents an event to bring up a drill-down channel for that event. The field set specified here will determine the columns (fields) shown in the drill-down channel. (See Monitoring Dashboards for information on data monitor drill-downs.) |
Bucket Size in Seconds |
The time dimension for individual event data units. A number of these units make up the value used in Number of Buckets. For example, you might use a value of 300 to create five-minute buckets. Bucket size and frequency (increasing freshness and resolution) does have a performance cost so it is wise to set buckets to run only as small and fast as actually necessary. |
Number of Buckets |
The overall time dimension to evaluate, expressed as the appropriate number of Bucket Size units. For example, to evaluate the most recent hour using five-minute buckets, you would enter 12. Bucket size and frequency (increasing freshness and resolution) does have a performance cost so it is wise to set buckets to run only as small and fast as actually necessary. |
Time Field |
Choose the specific event timestamp to use to apply events to time buckets. |
# Top Entries |
The number of entries to show as "top" values. |
# of Distinct Events |
This value must equal or exceed the maximum number of values that the Aggregate Field can possibly have. The default is 1,000. The maximum is 10,000. This value controls the upper limit on the number of aggregate field values. If it is smaller than necessary, then when it encounters one more Aggregate Field value than allowed, the Data Monitor resets all the counters, clears the data, and starts over at zero. If you specify more than one Aggregate Field, the maximum number of possibilities is the product of the possible values of all fields. For example, if you are aggregating by users and zones in an environment with 200 users and 15 zones, the number of possibilities is 200 x 15 = 3,000. If the number of possibilities is larger than the maximum of 10,000, use a filter to reduce them. |
Aggregate Field |
Specify one or more data fields to monitor. For more information, see Data Fields. To monitor the top 10 source IP addresses, for example, select the Source Address data field from the drop-down menu. If you specify more than one field, the total number of possible combinations is the product of the number of possible values for each field you specify. Make sure that the # of Distinct Events field is large enough to accommodate this number. |
Value Field |
Specify what the data monitor will use when determining the top value counts: the number of matching events, or the sum of a particular data field value in all matching events.
In either case, counts from aggregated events will be properly adjusted. |
Send Audit Events |
Specify generation of audit events for this data monitor. By default, audit events are not generated. Refer to Audit Events and look for the audit events under “Top Value Counts Data Monitor.” |
Troubleshooting
You might see warnings about the Top Value Counts data monitor type in the server logs, stating that internal data structures are being discarded to prevent overflow. "Data structures" in this warning refer to the counts being tracked. The events are not actually lost. This warning indicates the data monitor is using system resources but not providing any useful statistical data because the data monitor's conditions are poorly selected. When this warning appears, the problem continues until you fix your data monitor configuration.
Try these:
-
Choose your event filter and aggregation settings carefully. For example, if you are counting top N source addresses from within your organization, you should not see these warnings. But if your top N source addresses are from a broader source like the internet, the top 1000 will be easily exceeded and the data will be flushed. This means counting will start all over repetitively.
-
The data monitor will monitor up to 1000 distinct matches (can be increased to up to 10000) based on the event filter and aggregation settings and from that, determine the top N (10 by default). Increasing the
# of Distinct Events
default from 1000 to a maximum 10000 might help, but will cause the data monitor to consume more resources.