Depending on the EPS rate, you can choose to use traditional storage to store and index your Sentinel data.
Table 6-1 Traditional Storage
|
Traditional Storage |
|---|
|
By default, data is stored in file-based traditional storage and indexing is done locally on the Sentinel server. In addition to file-based data storage, you can also choose to store and index events in the Visualization Data Store to leverage data visualization capabilities. For more information, see Configuring the Visualization Data Store. |
|
Seamlessly scales up to approximately 20000 EPS. Beyond that you must add additional Sentinel servers to scale up to much higher EPS. |
|
Data collection is load-balanced across several Sentinel servers. Therefore, data is spread across different Sentinel servers and should be managed individually. |
|
Data is labeled tenant-wise but not segregated tenant-wise on disk. |
|
Data replication and availability must be done either manually or by using expensive storage mechanisms such as SAN disk. |
File-based data storage has a three-tier structure:
|
Online Storage |
Primary storage, formerly known as local storage. |
Optimized for quick writes and fast retrieval. Stores the most recently collected event data and the most frequently searched event data. |
|
Secondary storage, formerly known as network storage. (optional) |
Optimized to reduce space usage on optionally less expensive storage while still supporting fast retrieval. Sentinel automatically migrates data partitions to the secondary storage. |
|
|
NOTE:Using the secondary storage is optional. Data retention policies, searches, and reports operate on event data partitions regardless of whether they are residing on primary or secondary storage, or both. |
||
|
Offline Storage |
Archival storage |
When the partitions are closed, you can back up the partition to any file storage service, such as Amazon Glacier. You can temporarily re-import the partitions for use in long-term forensic analysis whenever necessary. |
You can also configure Sentinel to extract the event data and event data summaries to an external database by using data synchronization policies. For more information, see Configuring Data Synchronization in the Sentinel Administration Guide.
When you install Sentinel, you must mount the disk partition for primary storage in the location where Sentinel will be installed, by default the <sentinel_installation_path>/var/opt/novell directory.
The entire directory structure under the <sentinel_installation_path>/var/opt/novell/sentinel directory must reside on a single disk partition to ensure correct disk usage calculations. Else, the automatic data management capabilities might delete event data prematurely. For more information about the Sentinel directory structure, see Sentinel Directory Structure.
As a best practice, ensure that this data directory is located on a separate disk partition than the executables, configuration, and operating system files. The benefits of storing variable data separately include easier backup of sets of files, simpler recovery in case of corruption, and provides additional robustness if a disk partition fills up. It also improves the overall performance of systems where smaller file systems are more efficient. For more information, see Disk Partitioning.
NOTE:There is a limitation in ext3 file systems for file storage, which prevents a directory from having more than 32000 files or subdirectories. You can use XFS file system if you are going to have a large number of retention policies or if you are going to retain the data for longer periods of time, such as an year.
On traditional installations, you can modify the disk partition layout of the operating system before installing Sentinel. The administrator should create and mount the desired partitions to the appropriate directories, based on the directory structure described in Sentinel Directory Structure. When you run the installer, Sentinel is installed into the pre-created directories resulting in an installation that spans multiple partitions.
NOTE:
You can use the --location option while running the installer to specify a different top-level location than the default directories to store the file. The value that you pass to the --location option is prepended to the directory paths. For example, if you specify --location=/foo, the data directory will be /foo/var/opt/novell/sentinel/data and the config directory will be /foo/etc/opt/novell/sentinel/config.
You must not use filesystem links (for example, soft links) for the --location option.
Many organizations have their own documented best-practice partition layout schemes for any installed system. The following partition proposal is intended to guide organizations without any defined policy, and considers Sentinel specific use of the filesystem. Generally, Sentinel adheres to the Filesystem Hierarchy Standard where practicable.
|
Partition |
Mount point |
Size |
Notes |
|---|---|---|---|
|
Root |
/ |
100GB |
Contains operating system files and Sentinel binaries/configuration. |
|
Boot |
/boot |
150MB |
Boot partition |
|
Primary storage |
<sentinel_installation_path>/var/opt/novell/sentinel |
Calculate using the System Sizing Information. |
This area will contain the primary Sentinel collected data, and other variable data such as log files. This partition can be shared with other systems. |
|
Secondary storage |
Location based on the type of storage, NFS, CIFS, or SAN. |
Calculate using the System Sizing Information. |
This is the secondary storage area, which can be mounted locally as shown or remotely. |
|
Archival storage |
Remote system |
Calculate using the System Sizing Information. |
This storage is for archived data. |
Sentinel provides event visualizations that present data in charts, tables, and maps. These visualizations make it easier to visualize and analyze large volumes of events. You can also create your own visualizations and dashboards.
Sentinel leverages Opensearch Dashboard, a browser-based analytics and search dashboard, that helps you to search and visualize events. Opensearch Dashboard accesses data from visualization data store (OpenSearch) to present events in dashboards. By default, Sentinel includes an OpenSearch node that stores and indexes only alerts. You must enable event visualization to store and index events in OpenSearch.
When you enable OpenSearch to store and index data, Sentinel indexes only some specific event fields required for visualizations and stores the indexed fields in OpenSearch. Sentinel creates a dedicated index for each day and uses the UTC timezone (midnight-midnight) to calculate the index date. The index name is in the security.events.normalized_yyyyMMdd format. For example, the index security.events.normalized_20160101 contains all events that with an event time of January 01, 2016.
Configuring the visualization data store involves the following:
Installing OpenSearch nodes in a cluster mode: By default, Sentinel includes an OpenSearch node. For optimal performance and stability of the Sentinel server, it is mandatory that you install additional OpenSearch nodes in a cluster mode. For more information, see Section 12.0, Installing OpenSearch.
Enable event visualization: Event visualization is disabled by default. To enable event visualization, see Section 18.0, Configuring OpenSearch for Event Visualization.
Performance tuning: Sentinel automatically configures certain OpenSearch settings for optimal performance. You can customize these settings as needed. For example, you can modify the event fields you want OpenSearch to index. For more information, see Performance Tuning for OpenSearch.
By default, the Sentinel directories are in the following locations:
The data files are in <sentinel_installation_path>/var/opt/novell/sentinel/data and <sentinel_installation_path>/var/opt/novell/sentinel/3rdparty directories.
Executables and libraries are stored in the /opt/novell/sentinel directory.
Log files are in the <sentinel_installation_path>/var/opt/novell/sentinel/log directory.
Temporary files are in the <sentinel_installation_path>/var/opt/novell/sentinel/tmp directory.
Configuration files are in the /etc/opt/novell/sentinel directory.
The process ID (PID) file is in the /home/novell/sentinel/server.pid directory.
Using the PID, administrators can identify the parent process of Sentinel server and monitor or terminate the process.