17.1 Understanding Time in Sentinel

Sentinel is a distributed system that is made up of several processes distributed through out your network. In addition, there can be some delay introduced by the event source. To accommodate this, the Sentinel processes reorder events into a time-ordered stream before processing.

Every event has three time fields:

  • Event Time: This is the event time used by all analytical engines, searches, reports, and so on.

  • Sentinel Process Time: The time Sentinel collected the data from the device, which is taken from the Collector Manager system time.

  • Observer Event Time: The time stamp the device put in the data. The data might not always contain a reliable time stamp and can be quite different than the Sentinel Process Time. For example, when the device delivers data in batches.

The following illustration explains how Sentinel does this in a traditional storage setup:

Figure 17-1 Sentinel Time

  1. By default, the Event Time is set to the Sentinel Process Time. The ideal, however, is for the Event Time to match the Observer Event Time, if it is available and trustworthy. It is best to configure data collection to Trust Event Source Time if the device time is available, accurate, and properly parsed by the Collector. The Collector sets the Event Time to match the Observer Event Time.

  2. The events that have an Event Time within a 5 minute range from the server time (in the past or future) are processed normally by Event Views. Events that have an Event Time more than 5 minutes in the future do not show in the Event Views, but are inserted into the event store. Events that have an Event Time more than 5 minutes in the future and less than 24 hours in the past still are shown in the charts, but are not shown in the event data for that chart. A drill-down operation is necessary to retrieve those events from the event store.

  3. Events are sorted into 30-second intervals so that the Correlation Engine can process them in chronological order. If the Event Time is more than 30 seconds older than the server time, the Correlation Engine does not process the events.

  4. If the Event Time is older than 5 minutes relative to the Collector Manager system time, Sentinel directly routes events to the event store, bypassing real-time systems such as Correlation Engine and Security Intelligence.