wazuh/wazuh-indexer-plugins
1
0
Fork 0
mirror of https://github.com/wazuh/wazuh-indexer-plugins.git synced 2025-12-10 14:32:28 -06:00
Code Issues Packages Projects Releases Wiki Activity

Replace time-series indices with data streams (#652)

Browse Source 
* Use v2 index templates

* Replace stateless indices with data streams

Replace legacy index template with v2 index templates

* Clean-up and document code

* Update 5_builderpackage_plugins_onpush.yml

Signed-off-by: Álex Ruiz Becerra <alejandro.ruiz.becerra@wazuh.com>

* Downgrade jackson to 2.18.2 to match OpenSearch's version

* Adapt tests

* Divide testClusters into two separates one to fix the test and allow the run

* Format files

* Fix index patterns and data streams creation

---------

Signed-off-by: Álex Ruiz Becerra <alejandro.ruiz.becerra@wazuh.com>
Co-authored-by: Jorge Sanchez <jorge.sanchez@wazuh.com>
This commit is contained in:
Álex Ruiz Becerra 2025-11-14 11:58:26 +01:00 committed by GitHub
parent daa70b6e0b
commit b8879394b8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
71 changed files with 122542 additions and 122314 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -55,6 +55,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Restructuring of the repository tooling [(#625)](https://github.com/wazuh/wazuh-indexer-plugins/pull/625)
- Fix mdbook to version 0.4.x [(#627)](https://github.com/wazuh/wazuh-indexer-plugins/pull/627)
- Adapt WCS generation script to save the ECS flat of stateless module [(#646)](https://github.com/wazuh/wazuh-indexer-plugins/pull/646)
- Replace time-series indices with data streams [(#562)](https://github.com/wazuh/wazuh-indexer-plugins/pull/562)
### Deprecated
-

6
docs/ref/modules/setup/index.md
View File

@ -3,9 +3,9 @@
The `wazuh-indexer-setup` plugin is a module composing the Wazuh Indexer responsible for the initialization of the indices required by Wazuh to store all the data gathered and generated by other Central Components, such as the agents and the server (engine).
The Wazuh Indexer Setup Plugin in responsible for:
- Create the index templates, to define the mappings and settings of the indices.
- Create the initial indices. We distinguish between **stateful** and **stateless** indices. Stateful indices are unique, and its data is update over time (agents' inventory), stateless indices are rotated and static (alerts).
- For stateless indices, it creates the indices aliases and lifecycle policies for rollover.
- Create the index templates, to define the mappings and settings for the indices.
- Create the initial indices. We distinguish between **stateful** and **stream** indices. While stream indices contain immutable time-series data and are rolled over periodically, stateful indices store dynamic data that can change over time and reside in a single index.
- Stream indices are created with a data stream configuration and an ISM rollover policy.
## Indices

4
ecs/generator/generate_schema.sh
View File

@ -53,12 +53,10 @@ function detect_modified_modules() {
modified_files=$(git diff --name-only origin/"$BASE_BRANCH")
for file in $modified_files; do
if [[ $file == ecs/state* && ( $file == *.yml || $file == *.json ) ]]; then
matched=false
# Try to match the file to one of the known module keys for exact detection
for key in "${!module_to_file[@]}"; do
if [[ $file == ecs/$key/* || $file == ecs/$key ]]; then
ecs_module="$key"
matched=true
break
fi
done
@ -157,7 +155,7 @@ function copy_files() {
echo "---> Index templates"
local destination_file
local resources_path="plugins/setup/src/main/resources"
local mappings_path="mappings/${ECS_VERSION}/generated/elasticsearch/legacy/template.json"
local mappings_path="mappings/${ECS_VERSION}/generated/elasticsearch/legacy/opensearch-template.json"
for ecs_module in "${modules_to_update[@]}"; do
# Copying index templates to the initialization plugin resources folder
destination_file=${module_to_file[$ecs_module]}

14
ecs/generator/images/generator.sh
View File

@ -74,6 +74,19 @@ generate_mappings() {
fi
# Transform legacy index template for OpenSearch compatibility
if [[ "$ecs_module" =~ "stateless/" ]]; then
# Transform time-series templates to use data streams
jq '{
"index_patterns": .index_patterns,
"priority": .order,
"data_stream": {},
"template": {
"settings": .settings,
"mappings": .mappings
}
}' "$in_file" >"$out_dir/generated/elasticsearch/legacy/opensearch-template.json"
else
# Stateful templates remain unchanged except for the formatting
jq '{
"index_patterns": .index_patterns,
"priority": .order,
@ -82,6 +95,7 @@ generate_mappings() {
"mappings": .mappings
}
}' "$in_file" >"$out_dir/generated/elasticsearch/legacy/opensearch-template.json"
fi
echo "Mappings saved to $out_dir"
}

2
ecs/stateless/access-management/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-access-management-*"
"wazuh-events-v5-access-management*"
],
"order": 1,
"settings": {

3
ecs/stateless/access-management/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-access-management-*"
"wazuh-events-v5-access-management*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-access-management",

2
ecs/stateless/applications/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-applications-*"
"wazuh-events-v5-applications*"
],
"order": 1,
"settings": {

3
ecs/stateless/applications/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-applications-*"
"wazuh-events-v5-applications*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-applications",

2
ecs/stateless/cloud-services/aws/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-aws-*"
"wazuh-events-v5-cloud-services-aws*"
],
"order": 10,
"settings": {

3
ecs/stateless/cloud-services/aws/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-aws-*"
"wazuh-events-v5-cloud-services-aws*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-aws",

2
ecs/stateless/cloud-services/azure/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-azure-*"
"wazuh-events-v5-cloud-services-azure*"
],
"order": 10,
"settings": {

3
ecs/stateless/cloud-services/azure/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-azure-*"
"wazuh-events-v5-cloud-services-azure*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-azure",

2
ecs/stateless/cloud-services/gcp/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-gcp-*"
"wazuh-events-v5-cloud-services-gcp*"
],
"order": 10,
"settings": {

3
ecs/stateless/cloud-services/gcp/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-gcp-*"
"wazuh-events-v5-cloud-services-gcp*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-gcp",

2
ecs/stateless/cloud-services/main/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-*"
"wazuh-events-v5-cloud-services*"
],
"order": 1,
"settings": {

3
ecs/stateless/cloud-services/main/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-*"
"wazuh-events-v5-cloud-services*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services",

2
ecs/stateless/main/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-alerts-v5-*"
"wazuh-alerts-v5*"
],
"order": 1,
"settings": {

3
ecs/stateless/main/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-alerts-v5-*"
"wazuh-alerts-v5*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-alerts-v5",

2
ecs/stateless/network-activity/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-network-activity-*"
"wazuh-events-v5-network-activity*"
],
"order": 1,
"settings": {

3
ecs/stateless/network-activity/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-network-activity-*"
"wazuh-events-v5-network-activity*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-network-activity",

2
ecs/stateless/other/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-other-*"
"wazuh-events-v5-other*"
],
"order": 1,
"settings": {

3
ecs/stateless/other/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-other-*"
"wazuh-events-v5-other*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-other",

2
ecs/stateless/security/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-security-*"
"wazuh-events-v5-security*"
],
"order": 1,
"settings": {

3
ecs/stateless/security/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-security-*"
"wazuh-events-v5-security*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-security",

2
ecs/stateless/system-activity/fields/template-settings-legacy.json
View File

@ -1,6 +1,6 @@
{
"index_patterns": [
"wazuh-events-v5-system-activity-*"
"wazuh-events-v5-system-activity*"
],
"order": 1,
"settings": {

3
ecs/stateless/system-activity/fields/template-settings.json
View File

@ -1,8 +1,9 @@
{
"index_patterns": [
"wazuh-events-v5-system-activity-*"
"wazuh-events-v5-system-activity*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-system-activity",

2
ecs/stateless/template/fields/template-settings-legacy.json
View File

@ -1,5 +1,5 @@
{
"index_patterns": ["wazuh-events-v5-<integration-name>-*"],
"index_patterns": ["wazuh-events-v5-<integration-name>*"],
"order": "<priority>",
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-<integration-name>",

3
ecs/stateless/template/fields/template-settings.json
View File

@ -1,6 +1,7 @@
{
"index_patterns": ["wazuh-events-v5-<integration-name>-*"],
"index_patterns": ["wazuh-events-v5-<integration-name>*"],
"priority": "<priority>",
"data_stream": {},
"template": {
"settings": {
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-<integration-name>",

28
plugins/setup/build.gradle
View File

@ -1,4 +1,5 @@
import org.opensearch.gradle.test.RestIntegTestTask
import java.util.concurrent.Callable
buildscript {
@ -83,6 +84,11 @@ dependencies {
implementation "org.apache.logging.log4j:log4j-slf4j-impl:2.25.2"
implementation "org.slf4j:slf4j-api:2.0.17"
// Use Jackson provided by OpenSearch (2.18.2) - only for compile time
compileOnly 'com.fasterxml.jackson.core:jackson-core:2.18.2'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.18.2'
implementation 'com.fasterxml.jackson.core:jackson-annotations:2.18.2'
// Job Scheduler & ISM stuff (required for the initialization of ISM policies)
zipArchive group: 'org.opensearch.plugin', name: 'opensearch-job-scheduler', version: opensearch_build
zipArchive group: 'org.opensearch.plugin', name: 'opensearch-index-management', version: opensearch_build
@ -142,12 +148,13 @@ integTest {
}
}
testClusters.integTest {
testClusters {
integTest {
plugin(getPlugin("opensearch-job-scheduler"))
plugin(getPlugin("opensearch-index-management"))
testDistribution = "INTEG_TEST"
// This installs our plugin into the testClusters
plugin(project.tasks.bundlePlugin.archiveFile)
@ -157,9 +164,23 @@ testClusters.integTest {
}
}
runCluster {
plugin(getPlugin("opensearch-job-scheduler"))
plugin(getPlugin("opensearch-index-management"))
testDistribution = "ARCHIVE"
plugin(project.tasks.bundlePlugin.archiveFile)
if (System.getProperty("run.cluster.debug") != null) {
jvmArgs '-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005'
}
}
}
run {
useCluster testClusters.integTest
useCluster testClusters.runCluster
}
// updateVersion: Task to auto update version to the next development iteration
@ -172,4 +193,3 @@ task updateVersion {
ant.replaceregexp(file: 'build.gradle', match: '"opensearch.version", "\\d.*"', replace: '"opensearch.version", "' + newVersion.tokenize('-')[0] + '-SNAPSHOT"', flags: 'g', byline: true)
}
}

15
plugins/setup/src/main/java/com/wazuh/setup/SetupPlugin.java
View File

@ -43,7 +43,7 @@ import com.wazuh.setup.index.IndexStateManagement;
import com.wazuh.setup.index.StateIndex;
import com.wazuh.setup.index.StreamIndex;
import com.wazuh.setup.settings.PluginSettings;
import com.wazuh.setup.utils.IndexUtils;
import com.wazuh.setup.utils.JsonUtils;
/**
* Main class of the Indexer Setup plugin. This plugin is responsible for the creation of the index
@ -87,14 +87,13 @@ public class SetupPlugin extends Plugin implements ClusterPlugin {
// ISM index
this.indices.add(new IndexStateManagement(IndexStateManagement.ISM_INDEX_NAME, "templates/ism-config"));
// Stream indices
this.indices.add(new StreamIndex("wazuh-alerts-v5-000001", "templates/streams/alerts", "wazuh-alerts-v5"));
this.indices.add(new StreamIndex("wazuh-archives-v5-000001", "templates/streams/archives", "wazuh-archives-v5"));
this.indices.add(new StreamIndex("wazuh-alerts-v5", "templates/streams/alerts"));
this.indices.add(new StreamIndex("wazuh-archives-v5", "templates/streams/archives"));
// Decoder indices
for (String category : this.categories) {
this.indices.add(new StreamIndex(
"wazuh-events-v5-" + category + "-000001",
"templates/streams/" + category,
"wazuh-events-v5-" + category
"wazuh-events-v5-" + category,
"templates/streams/" + category
));
}
@ -122,12 +121,12 @@ public class SetupPlugin extends Plugin implements ClusterPlugin {
// spotless:on
// Inject dependencies
IndexUtils utils = new IndexUtils();
JsonUtils utils = new JsonUtils();
this.indices.forEach(
index -> {
index.setClient(client);
index.setClusterService(clusterService);
index.setIndexUtils(utils);
index.setUtils(utils);
});
return Collections.emptyList();

84
plugins/setup/src/main/java/com/wazuh/setup/index/Index.java
View File

@ -16,22 +16,26 @@
*/
package com.wazuh.setup.index;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.ResourceAlreadyExistsException;
import org.opensearch.action.admin.indices.create.CreateIndexRequest;
import org.opensearch.action.admin.indices.create.CreateIndexResponse;
import org.opensearch.action.admin.indices.template.put.PutIndexTemplateRequest;
import org.opensearch.action.support.clustermanager.AcknowledgedResponse;
import org.opensearch.action.admin.indices.template.put.PutComposableIndexTemplateAction;
import org.opensearch.cluster.metadata.ComposableIndexTemplate;
import org.opensearch.cluster.service.ClusterService;
import org.opensearch.common.compress.CompressedXContent;
import org.opensearch.common.settings.Settings;
import org.opensearch.transport.client.Client;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.io.InputStream;
import com.wazuh.setup.model.IndexTemplate;
import com.wazuh.setup.settings.PluginSettings;
import com.wazuh.setup.utils.IndexUtils;
import com.wazuh.setup.utils.JsonUtils;
/**
* Abstract class with the required logic to create indices. In our context, an index always require
@ -45,7 +49,7 @@ public abstract class Index implements IndexInitializer {
// Dependencies.
Client client;
ClusterService clusterService;
IndexUtils indexUtils;
JsonUtils jsonUtils;
// Properties.
String index;
@ -87,12 +91,12 @@ public abstract class Index implements IndexInitializer {
}
/**
* Sets the IndexUtils instance.
* Sets the JsonUtils instance.
*
* @param indexUtils the IndexUtils instance to set.
* @param jsonUtils the JsonUtils instance to set.
*/
public void setIndexUtils(IndexUtils indexUtils) {
this.indexUtils = indexUtils;
public void setUtils(JsonUtils jsonUtils) {
this.jsonUtils = jsonUtils;
}
/**
@ -137,42 +141,45 @@ public abstract class Index implements IndexInitializer {
}
log.warn("Operation to create the index [{}] timed out. Retrying...", index);
this.retry_index_creation = false;
this.indexUtils.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.createIndex(index);
}
}
/**
* Creates an index template.
* Creates an index template (v2).
*
* @param template name of the index template to create.
*/
public void createTemplate(String template) {
try {
Map<String, Object> templateFile = this.indexUtils.fromFile(template + ".json");
// Read JSON index template
ObjectMapper mapper = new ObjectMapper();
InputStream is = this.getClass().getClassLoader().getResourceAsStream(template + ".json");
IndexTemplate indexTemplate = mapper.readValue(is, IndexTemplate.class);
PutIndexTemplateRequest putIndexTemplateRequest =
new PutIndexTemplateRequest()
.mapping(this.indexUtils.get(templateFile, "mappings"))
.settings(this.indexUtils.get(templateFile, "settings"))
.order((int) templateFile.getOrDefault("order", 0))
.name(template)
.patterns((List<String>) templateFile.get("index_patterns"));
// Create a V2 template (ComposableIndexTemplate)
String indexMappings = mapper.writeValueAsString(indexTemplate.getMappings());
CompressedXContent compressedMapping = new CompressedXContent(indexMappings);
Settings settings = Settings.builder().loadFromMap(indexTemplate.getSettings()).build();
ComposableIndexTemplate composableTemplate =
indexTemplate.getComposableIndexTemplate(settings, compressedMapping);
AcknowledgedResponse createIndexTemplateResponse =
// Use the V2 API to put the template
PutComposableIndexTemplateAction.Request request =
new PutComposableIndexTemplateAction.Request(template)
.indexTemplate(composableTemplate)
.create(false);
// Put index template
this.client
.admin()
.indices()
.putTemplate(putIndexTemplateRequest)
.execute(PutComposableIndexTemplateAction.INSTANCE, request)
.actionGet(PluginSettings.getTimeout(this.clusterService.getSettings()));
log.info(
"Index template created successfully: {} {}",
template,
createIndexTemplateResponse.isAcknowledged());
} catch (IOException e) {
log.error("Error reading index template from filesystem {}", template);
log.error(
"Error reading index template from filesystem [{}]. Caused by: {}",
template,
e.toString());
} catch (ResourceAlreadyExistsException e) {
log.info("Index template {} already exists. Skipping.", template);
} catch (
@ -188,7 +195,7 @@ public abstract class Index implements IndexInitializer {
}
log.warn("Operation to create the index template [{}] timed out. Retrying...", template);
this.retry_template_creation = false;
this.indexUtils.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.createTemplate(template);
}
}
@ -201,4 +208,17 @@ public abstract class Index implements IndexInitializer {
this.createTemplate(this.template);
this.createIndex(this.index);
}
/**
* Utility method to wrap up the call to {@link Thread#sleep(long)} on a try-catch block.
*
* @param millis sleep interval in milliseconds.
*/
void sleep(long millis) {
try {
Thread.sleep(millis);
} catch (InterruptedException ex) {
throw new RuntimeException(ex);
}
}
}

32
plugins/setup/src/main/java/com/wazuh/setup/index/IndexStateManagement.java
View File

@ -16,6 +16,8 @@
*/
package com.wazuh.setup.index;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.ResourceAlreadyExistsException;
@ -25,10 +27,12 @@ import org.opensearch.action.index.IndexRequest;
import org.opensearch.core.xcontent.MediaTypeRegistry;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.wazuh.setup.model.IndexTemplate;
import com.wazuh.setup.settings.PluginSettings;
/**
@ -38,7 +42,7 @@ import com.wazuh.setup.settings.PluginSettings;
public class IndexStateManagement extends Index {
private static final Logger log = LogManager.getLogger(IndexStateManagement.class);
// ISM index name
/** ISM index name */
public static final String ISM_INDEX_NAME = ".opendistro-ism-config";
// ISM policies names (filename without extension)
@ -74,10 +78,11 @@ public class IndexStateManagement extends Index {
*
* @param policy policy name to create.
*/
private void indexPolicy(String policy) {
void indexPolicy(String policy) {
try {
Map<String, Object> policyFile;
policyFile = this.indexUtils.fromFile(STREAM_ROLLOVER_POLICY_PATH);
policyFile = this.jsonUtils.fromFile(STREAM_ROLLOVER_POLICY_PATH);
IndexRequest indexRequest =
new IndexRequest(this.index)
@ -102,7 +107,7 @@ public class IndexStateManagement extends Index {
}
log.warn("Operation to create the policy [{}] timed out. Retrying...", policy);
this.retry_index_creation = false;
this.indexUtils.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.indexPolicy(policy);
}
}
@ -119,12 +124,16 @@ public class IndexStateManagement extends Index {
// For some reason the index template is not applied to the ISM internal index
// ".opendistro-ism-config", so we explicitly set the index mappings and settings
// as part of the CreateIndexRequest.
Map<String, Object> templateFile = this.indexUtils.fromFile(this.template + ".json");
// Read JSON index template
ObjectMapper mapper = new ObjectMapper();
InputStream is =
this.getClass().getClassLoader().getResourceAsStream(this.template + ".json");
IndexTemplate indexTemplate = mapper.readValue(is, IndexTemplate.class);
CreateIndexRequest request =
new CreateIndexRequest(index)
.mapping(this.indexUtils.get(templateFile, "mappings"))
.settings(this.indexUtils.get(templateFile, "settings"));
.mapping(indexTemplate.getMappings())
.settings(indexTemplate.getSettings());
CreateIndexResponse createIndexResponse =
this.client
.admin()
@ -136,10 +145,13 @@ public class IndexStateManagement extends Index {
createIndexResponse.index(),
createIndexResponse.isAcknowledged());
}
} catch (IOException e) {
log.error("Error reading index template from filesystem {}", this.template);
} catch (ResourceAlreadyExistsException e) {
log.info("Index {} already exists. Skipping.", index);
} catch (IOException e) {
log.error(
"Error reading index template from filesystem [{}]. Caused by: {}",
this.template,
e.toString());
} catch (
Exception
e) { // TimeoutException may be raised by actionGet(), but we cannot catch that one.
@ -150,7 +162,7 @@ public class IndexStateManagement extends Index {
}
log.warn("Operation to create the index [{}] timed out. Retrying...", index);
this.retry_index_creation = false;
this.indexUtils.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.createIndex(index);
}
}

67
plugins/setup/src/main/java/com/wazuh/setup/index/StreamIndex.java
View File

@ -19,9 +19,8 @@ package com.wazuh.setup.index;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.ResourceAlreadyExistsException;
import org.opensearch.action.admin.indices.alias.Alias;
import org.opensearch.action.admin.indices.create.CreateIndexRequest;
import org.opensearch.action.admin.indices.create.CreateIndexResponse;
import org.opensearch.action.admin.indices.datastream.CreateDataStreamAction;
import org.opensearch.action.support.clustermanager.AcknowledgedResponse;
import com.wazuh.setup.settings.PluginSettings;
@ -32,58 +31,58 @@ import com.wazuh.setup.settings.PluginSettings;
public class StreamIndex extends WazuhIndex {
private static final Logger log = LogManager.getLogger(StreamIndex.class);
private final String alias;
/**
* Constructor.
*
* @param index index name.
* @param template index template name.
* @param alias index alias name for advanced management such as automatic rollover.
*/
public StreamIndex(String index, String template, String alias) {
public StreamIndex(String index, String template) {
super(index, template);
this.alias = alias;
}
/**
* Overrides {@link Index#createIndex(String)} to include the {@link #alias} to the index creation
* request.
* Overrides {@link Index#createIndex(String)} to create a Data Stream instead.
*
* @param index Name of the index to create.
* @see Alias
* @param index Name of the data stream to create.
*/
@Override
public void createIndex(String index) {
try {
if (!this.indexExists(index)) {
CreateIndexRequest request =
new CreateIndexRequest(index).alias(new Alias(this.alias).writeIndex(true));
CreateIndexResponse createIndexResponse =
this.client
.admin()
.indices()
.create(request)
.actionGet(PluginSettings.getTimeout(this.clusterService.getSettings()));
log.info(
"Index created successfully: {} {}",
createIndexResponse.index(),
createIndexResponse.isAcknowledged());
}
this.createDataStream(index);
} catch (ResourceAlreadyExistsException e) {
log.info("Index {} already exists. Skipping.", index);
} catch (
Exception
e) { // TimeoutException may be raised by actionGet(), but we cannot catch that one.
// Exit condition. Re-attempt to create the index also failed. Original exception is rethrown.
log.info("Data stream {} already exists. Skipping.", index);
} catch (Exception e) {
// TimeoutException may be raised by actionGet(), but we cannot catch that one.
// Exit condition. Re-attempt to create the data stream also failed. Original exception is
// rethrown.
if (!this.retry_index_creation) {
log.error("Initialization of index [{}] finally failed. The node will shut down.", index);
log.error(
"Initialization of data stream [{}] finally failed. The node will shut down.", index);
throw e;
}
log.warn("Operation to create the index [{}] timed out. Retrying...", index);
log.warn("Operation to create the data stream [{}] timed out. Retrying...", index);
this.retry_index_creation = false;
this.indexUtils.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.sleep(PluginSettings.getBackoff(this.clusterService.getSettings()));
this.createIndex(index);
}
}
/**
* Creates a Data Stream.
*
* @param name name of the data stream to create.
*/
public void createDataStream(String name) {
CreateDataStreamAction.Request request = new CreateDataStreamAction.Request(name);
AcknowledgedResponse response =
this.client
.admin()
.indices()
.createDataStream(request)
.actionGet(PluginSettings.getTimeout(this.clusterService.getSettings()));
log.info("Data Stream created successfully: {} {}", name, response.isAcknowledged());
}
}

123
plugins/setup/src/main/java/com/wazuh/setup/model/IndexTemplate.java Normal file
View File

@ -0,0 +1,123 @@
/*
* Copyright (C) 2024, Wazuh Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.wazuh.setup.model;
import com.fasterxml.jackson.annotation.JsonProperty;
import org.opensearch.cluster.metadata.ComposableIndexTemplate;
import org.opensearch.cluster.metadata.Template;
import org.opensearch.common.compress.CompressedXContent;
import org.opensearch.common.settings.Settings;
import java.util.List;
import java.util.Map;
/**
* Index Template Data Transfer Object.
*
* <p>Fill-in values automatically using Jackson Databind.
*/
public class IndexTemplate {
/** Default constructor */
public IndexTemplate() {}
@JsonProperty("index_patterns")
private List<String> indexPatterns;
private long priority;
private Map<String, Object> settings;
private Map<String, Object> mappings;
@JsonProperty("data_stream")
private Map<String, Object> dataStream;
@SuppressWarnings("unchecked")
@JsonProperty("template")
private void unpackNested(Map<String, Object> template) {
this.settings = (Map<String, Object>) template.get("settings");
this.mappings = (Map<String, Object>) template.get("mappings");
}
/**
* Index pattern getter.
*
* @return returns the list of index patters this index template applies to, or null.
*/
public List<String> getIndexPatterns() {
return this.indexPatterns;
}
/**
* Index template priority getter.
*
* @return returns the priority of the index template, or null.
*/
public long getPriority() {
return this.priority;
}
/**
* Index settings getter.
*
* @return returns the index settings to apply to indices matching the index pattern, or null.
*/
public Map<String, Object> getSettings() {
return this.settings;
}
/**
* Index mappings getter.
*
* @return returns the index mappings for the indices matching the index pattern, or null.
*/
public Map<String, Object> getMappings() {
return this.mappings;
}
/**
* Data Stream getter.
*
* @return returns the "data_stream" property of the index template, or null.
*/
public Map<String, Object> getDataStream() {
return this.dataStream;
}
/**
* Builds up a ComposableIndexTemplate resulting from the properties of the index template.
*
* @param settings index template settings as a string.
* @param compressedMapping index template mappings a CompressedXContent instance.
* @return instance of ComposableIndexTemplate.
*/
public ComposableIndexTemplate getComposableIndexTemplate(
Settings settings, CompressedXContent compressedMapping) {
ComposableIndexTemplate.DataStreamTemplate dataStreamTemplate =
this.getDataStream() != null ? new ComposableIndexTemplate.DataStreamTemplate() : null;
// Create the composable template
return new ComposableIndexTemplate(
this.getIndexPatterns(),
new Template(settings, compressedMapping, null),
null,
this.getPriority(),
null,
null,
dataStreamTemplate);
}
}

50
plugins/setup/src/main/java/com/wazuh/setup/utils/IndexUtils.java → plugins/setup/src/main/java/com/wazuh/setup/utils/JsonUtils.java
View File

@ -27,35 +27,24 @@ import java.util.Map;
import reactor.util.annotation.NonNull;
/** Util functions to parse and manage index templates files. */
public class IndexUtils {
/** Util functions to parse and manage JSON files. */
public class JsonUtils {
/** Default constructor */
public IndexUtils() {}
public JsonUtils() {}
/**
* Read index template file from the resources folder and returns its JSON content as a map.
* Read JSON file from the resources folder and returns its JSON content as a map.
*
* @param filename name of the index template to read from the resources folder
* @return the JSON index template as a map
* @param filename name of the JSON file to read from the resources folder
* @return the JSON file as a map
* @throws IOException file not found or could not be read
* @deprecated Use jackson's ObjectMapper instead.
*/
@Deprecated
public Map<String, Object> fromFile(@NonNull String filename) throws IOException {
InputStream is = IndexUtils.class.getClassLoader().getResourceAsStream(filename);
return this.toMap(is);
}
InputStream is = JsonUtils.class.getClassLoader().getResourceAsStream(filename);
/**
* Convert from a JSON InputStream into a String, Object map.
*
* <p>Used to convert the JSON index templates to the required format.
*
* @param is: the JSON formatted InputStream
* @return a map with the json string contents.
* @throws IOException thrown by {@link JsonXContent#createParser(NamedXContentRegistry,
* DeprecationHandler, InputStream)}
*/
public Map<String, Object> toMap(InputStream is) throws IOException {
XContentParser parser =
JsonXContent.jsonXContent.createParser(
NamedXContentRegistry.EMPTY, DeprecationHandler.THROW_UNSUPPORTED_OPERATION, is);
@ -64,29 +53,14 @@ public class IndexUtils {
}
/**
* Cast map's element to a String, Object map.
* Return JSON node by key.
*
* <p>Used to retrieve the settings and mappings from the index templates, which are a JSON object
* themselves.
*
* @param map the index template as a map.
* @param map the parent JSON node where the key to retrieve is.
* @param key the element's key to retrieve and cast.
* @return a String, Object map
*/
@SuppressWarnings("unchecked")
public Map<String, Object> get(Map<String, Object> map, String key) {
return (Map<String, Object>) map.get(key);
}
/**
* Utility method to wrap up the call to {@link Thread#sleep(long)} on a try-catch block.
*
* @param millis sleep interval in milliseconds.
*/
public void sleep(long millis) {
try {
Thread.sleep(millis);
} catch (InterruptedException ex) {
throw new RuntimeException(ex);
}
}
}

18
plugins/setup/src/main/resources/templates/ism-config.json
View File

@ -2,6 +2,16 @@
"index_patterns": [
".opendistro-ism-config"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"refresh_interval": "5s",
"hidden": true
}
},
"mappings": {
"dynamic": "strict",
"properties": {
@ -1699,14 +1709,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"refresh_interval": "5s",
"hidden": true
}
}
}

4
plugins/setup/src/main/resources/templates/monitoring.json
View File

@ -1,8 +1,9 @@
{
"order": 1,
"index_patterns": [
"wazuh-monitoring*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
@ -41,3 +42,4 @@
}
}
}
}

84
plugins/setup/src/main/resources/templates/states/fim-files.json
View File

@ -2,6 +2,49 @@
"index_patterns": [
"wazuh-states-fim-files*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.groups ",
"agent.host.architecture ",
"agent.host.hostname ",
"agent.host.ip ",
"agent.host.os.name ",
"agent.host.os.platform ",
"agent.host.os.type ",
"agent.host.os.version ",
"agent.id ",
"agent.name ",
"agent.version ",
"checksum.hash.sha1",
"file.attributes",
"file.device",
"file.gid",
"file.group",
"file.hash.md5",
"file.hash.sha1",
"file.hash.sha256",
"file.inode",
"file.mtime",
"file.owner",
"file.path",
"file.path.fields.text",
"file.permissions",
"file.size",
"file.uid",
"state.document_version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -174,47 +217,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.groups ",
"agent.host.architecture ",
"agent.host.hostname ",
"agent.host.ip ",
"agent.host.os.name ",
"agent.host.os.platform ",
"agent.host.os.type ",
"agent.host.os.version ",
"agent.id ",
"agent.name ",
"agent.version ",
"checksum.hash.sha1",
"file.attributes",
"file.device",
"file.gid",
"file.group",
"file.hash.md5",
"file.hash.sha1",
"file.hash.sha256",
"file.inode",
"file.mtime",
"file.owner",
"file.path",
"file.path.fields.text",
"file.permissions",
"file.size",
"file.uid",
"state.document_version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

60
plugins/setup/src/main/resources/templates/states/fim-registry-keys.json
View File

@ -2,6 +2,37 @@
"index_patterns": [
"wazuh-states-fim-registry-keys*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"checksum.hash.sha1",
"registry.architecture",
"registry.gid",
"registry.group",
"registry.hive",
"registry.key",
"registry.mtime",
"registry.owner",
"registry.path",
"registry.permissions",
"registry.uid",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -155,35 +186,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"checksum.hash.sha1",
"registry.architecture",
"registry.gid",
"registry.group",
"registry.hive",
"registry.key",
"registry.mtime",
"registry.owner",
"registry.path",
"registry.permissions",
"registry.uid",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

60
plugins/setup/src/main/resources/templates/states/fim-registry-values.json
View File

@ -2,6 +2,37 @@
"index_patterns": [
"wazuh-states-fim-registry-values*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"checksum.hash.sha1",
"registry.architecture",
"registry.data.hash.md5",
"registry.data.hash.sha1",
"registry.data.hash.sha256",
"registry.data.type",
"registry.hive",
"registry.key",
"registry.path",
"registry.size",
"registry.value",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -163,35 +194,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"checksum.hash.sha1",
"registry.architecture",
"registry.data.hash.md5",
"registry.data.hash.sha1",
"registry.data.hash.sha256",
"registry.data.type",
"registry.hive",
"registry.key",
"registry.path",
"registry.size",
"registry.value",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

52
plugins/setup/src/main/resources/templates/states/inventory-browser-extensions.json
View File

@ -2,6 +2,33 @@
"index_patterns": [
"wazuh-states-inventory-browser-extensions*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"browser.name",
"user.id",
"package.name",
"package.id",
"package.version",
"package.vendor",
"package.type",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -217,31 +244,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"browser.name",
"user.id",
"package.name",
"package.id",
"package.version",
"package.vendor",
"package.type",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

32
plugins/setup/src/main/resources/templates/states/inventory-groups.json
View File

@ -2,6 +2,23 @@
"index_patterns": [
"wazuh-states-inventory-groups*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"group.name",
"group.description",
"group.id_signed",
"group.uuid",
"group.users",
"state.modified_at"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -140,21 +157,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"group.name",
"group.description",
"group.id_signed",
"group.uuid",
"group.users",
"state.modified_at"
],
"refresh_interval": "2s"
}
}
}

42
plugins/setup/src/main/resources/templates/states/inventory-hardware.json
View File

@ -2,6 +2,28 @@
"index_patterns": [
"wazuh-states-inventory-hardware*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"host.serial_number",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -151,26 +173,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"host.serial_number",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

40
plugins/setup/src/main/resources/templates/states/inventory-hotfixes.json
View File

@ -2,6 +2,27 @@
"index_patterns": [
"wazuh-states-inventory-hotfixes*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.hotfix.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -125,25 +146,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.hotfix.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

40
plugins/setup/src/main/resources/templates/states/inventory-interfaces.json
View File

@ -2,6 +2,27 @@
"index_patterns": [
"wazuh-states-inventory-interfaces*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"host.mac",
"interface.name",
"interface.alias",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -179,25 +200,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"host.mac",
"interface.name",
"interface.alias",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node"
],
"refresh_interval": "2s"
}
}
}

40
plugins/setup/src/main/resources/templates/states/inventory-networks.json
View File

@ -2,6 +2,27 @@
"index_patterns": [
"wazuh-states-inventory-networks*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"interface.name",
"network.ip",
"network.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -143,25 +164,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.name",
"agent.version",
"agent.host.ip",
"interface.name",
"network.ip",
"network.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node"
],
"refresh_interval": "2s"
}
}
}

64
plugins/setup/src/main/resources/templates/states/inventory-packages.json
View File

@ -2,6 +2,39 @@
"index_patterns": [
"wazuh-states-inventory-packages*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.architecture",
"package.category",
"package.description",
"package.installed",
"package.multiarch",
"package.name",
"package.path",
"package.priority",
"package.size",
"package.source",
"package.type",
"package.vendor",
"package.version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -166,37 +199,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.architecture",
"package.category",
"package.description",
"package.installed",
"package.multiarch",
"package.name",
"package.path",
"package.priority",
"package.size",
"package.source",
"package.type",
"package.vendor",
"package.version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

56
plugins/setup/src/main/resources/templates/states/inventory-ports.json
View File

@ -2,6 +2,35 @@
"index_patterns": [
"wazuh-states-inventory-ports*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"host.network.egress.queue",
"host.network.ingress.queue",
"file.inode",
"interface.state",
"network.transport",
"process.name",
"process.pid",
"source.ip",
"destination.ip",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -189,33 +218,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"host.network.egress.queue",
"host.network.ingress.queue",
"file.inode",
"interface.state",
"network.transport",
"process.name",
"process.pid",
"source.ip",
"destination.ip",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

58
plugins/setup/src/main/resources/templates/states/inventory-processes.json
View File

@ -2,6 +2,36 @@
"index_patterns": [
"wazuh-states-inventory-processes*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"process.args",
"process.args_count",
"process.command_line",
"process.name",
"process.parent.pid",
"process.pid",
"process.start",
"process.state",
"process.stime",
"process.utime",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -154,34 +184,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"process.args",
"process.args_count",
"process.command_line",
"process.name",
"process.parent.pid",
"process.pid",
"process.start",
"process.state",
"process.stime",
"process.utime",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

42
plugins/setup/src/main/resources/templates/states/inventory-protocols.json
View File

@ -2,6 +2,28 @@
"index_patterns": [
"wazuh-states-inventory-protocols*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"network.type",
"interface.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -137,26 +159,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"network.type",
"interface.name",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

72
plugins/setup/src/main/resources/templates/states/inventory-services.json
View File

@ -2,6 +2,43 @@
"index_patterns": [
"wazuh-states-inventory-services*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"service.name",
"service.id",
"service.description",
"service.enabled",
"service.restart",
"service.state",
"service.sub_state",
"service.type",
"service.start_type",
"process.executable",
"file.path",
"log.file.path",
"error.log.file.path",
"process.args",
"process.user.name",
"process.group.name",
"service.address",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -282,41 +319,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"service.name",
"service.id",
"service.description",
"service.enabled",
"service.restart",
"service.state",
"service.sub_state",
"service.type",
"service.start_type",
"process.executable",
"file.path",
"log.file.path",
"error.log.file.path",
"process.args",
"process.user.name",
"process.group.name",
"service.address",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

70
plugins/setup/src/main/resources/templates/states/inventory-system.json
View File

@ -2,6 +2,42 @@
"index_patterns": [
"wazuh-states-inventory-system*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"host.architecture",
"host.hostname",
"host.os.build",
"host.os.codename",
"host.os.distribution.release",
"host.os.full",
"host.os.kernel.name",
"host.os.kernel.release",
"host.os.kernel.version",
"host.os.major",
"host.os.minor",
"host.os.name",
"host.os.patch",
"host.os.platform",
"host.os.type",
"host.os.version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -192,40 +228,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"host.architecture",
"host.hostname",
"host.os.build",
"host.os.codename",
"host.os.distribution.release",
"host.os.full",
"host.os.kernel.name",
"host.os.kernel.release",
"host.os.kernel.version",
"host.os.major",
"host.os.minor",
"host.os.name",
"host.os.patch",
"host.os.platform",
"host.os.type",
"host.os.version",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

68
plugins/setup/src/main/resources/templates/states/inventory-users.json
View File

@ -2,6 +2,41 @@
"index_patterns": [
"wazuh-states-inventory-users*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"user.id",
"user.name",
"user.home",
"user.type",
"user.created",
"user.groups",
"user.uuid",
"user.auth_failures.timestamp",
"user.auth_failures.count",
"user.group.id_signed",
"user.login.status",
"user.login.type",
"user.login.tty",
"user.password.status",
"user.password.last_change",
"user.password.expiration_date",
"user.roles",
"user.last_login",
"process.pid",
"host.ip",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -249,39 +284,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"user.id",
"user.name",
"user.home",
"user.type",
"user.created",
"user.groups",
"user.uuid",
"user.auth_failures.timestamp",
"user.auth_failures.count",
"user.group.id_signed",
"user.login.status",
"user.login.type",
"user.login.tty",
"user.password.status",
"user.password.last_change",
"user.password.expiration_date",
"user.roles",
"user.last_login",
"process.pid",
"host.ip",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

50
plugins/setup/src/main/resources/templates/states/sca.json
View File

@ -2,6 +2,32 @@
"index_patterns": [
"wazuh-states-sca*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.groups",
"check.name",
"check.id",
"checksum.hash.sha1",
"host.name",
"host.os.type",
"host.os.version",
"policy.id",
"policy.name",
"policy.file",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "5s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -184,30 +210,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.id",
"agent.groups",
"check.name",
"check.id",
"checksum.hash.sha1",
"host.name",
"host.os.type",
"host.os.version",
"policy.id",
"policy.name",
"policy.file",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "5s"
}
}
}

74
plugins/setup/src/main/resources/templates/states/vulnerabilities.json
View File

@ -2,6 +2,44 @@
"index_patterns": [
"wazuh-states-vulnerabilities*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"codec": "best_compression",
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.architecture",
"package.category",
"package.description",
"package.installed",
"package.multiarch",
"package.name",
"package.path",
"package.priority",
"package.size",
"package.source",
"package.type",
"package.vendor",
"package.version",
"vulnerability.id",
"vulnerability.description",
"vulnerability.severity",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
},
"mappings": {
"date_detection": false,
"dynamic": "strict",
@ -296,42 +334,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"codec": "best_compression",
"number_of_replicas": "0",
"number_of_shards": "1",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"package.architecture",
"package.category",
"package.description",
"package.installed",
"package.multiarch",
"package.name",
"package.path",
"package.priority",
"package.size",
"package.source",
"package.type",
"package.vendor",
"package.version",
"vulnerability.id",
"vulnerability.description",
"vulnerability.severity",
"state.modified_at",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
}
}
}

4
plugins/setup/src/main/resources/templates/statistics.json
View File

@ -1,8 +1,9 @@
{
"order": 1,
"index_patterns": [
"wazuh-statistics*"
],
"priority": 1,
"template": {
"settings": {
"index": {
"number_of_replicas": "0",
@ -253,3 +254,4 @@
}
}
}
}

51
plugins/setup/src/main/resources/templates/streams/access-management.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-access-management-*"
"wazuh-events-v5-access-management*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-access-management"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -8273,28 +8298,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-access-management"
}
}
}

49
plugins/setup/src/main/resources/templates/streams/alerts.json
View File

@ -1,7 +1,31 @@
{
"index_patterns": [
"wazuh-alerts-v5-*"
"wazuh-alerts-v5*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-alerts-v5"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -8308,27 +8332,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-alerts-v5"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/applications.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-applications-*"
"wazuh-events-v5-applications*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-applications"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -10306,28 +10331,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-applications"
}
}
}

49
plugins/setup/src/main/resources/templates/streams/archives.json
View File

@ -1,7 +1,31 @@
{
"index_patterns": [
"wazuh-archives-v5-*"
"wazuh-archives-v5*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-archives-v5"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -8308,27 +8332,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-archives-v5"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/cloud-services-aws.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-aws-*"
"wazuh-events-v5-cloud-services-aws*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 200,
"mapping.total_fields.limit": 3500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-aws"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -12087,28 +12112,6 @@
}
}
}
},
"order": 10,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 200,
"mapping.total_fields.limit": 3500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-aws"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/cloud-services-azure.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-azure-*"
"wazuh-events-v5-cloud-services-azure*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-azure"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -10587,28 +10612,6 @@
}
}
}
},
"order": 10,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-azure"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/cloud-services-gcp.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-gcp-*"
"wazuh-events-v5-cloud-services-gcp*"
],
"priority": 10,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 100,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-gcp"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -9858,28 +9883,6 @@
}
}
}
},
"order": 10,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 100,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services-gcp"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/cloud-services.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-cloud-services-*"
"wazuh-events-v5-cloud-services*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -8464,28 +8489,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-cloud-services"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/network-activity.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-network-activity-*"
"wazuh-events-v5-network-activity*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 4000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-network-activity"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -13798,28 +13823,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 4000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-network-activity"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/other.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-other-*"
"wazuh-events-v5-other*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-other"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -8273,28 +8298,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 2500,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-other"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/security.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-security-*"
"wazuh-events-v5-security*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-security"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -10235,28 +10260,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-security"
}
}
}

51
plugins/setup/src/main/resources/templates/streams/system-activity.json
View File

@ -1,7 +1,32 @@
{
"index_patterns": [
"wazuh-events-v5-system-activity-*"
"wazuh-events-v5-system-activity*"
],
"priority": 1,
"data_stream": {},
"template": {
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-system-activity"
},
"mappings": {
"date_detection": false,
"dynamic": "false",
@ -10091,28 +10116,6 @@
}
}
}
},
"order": 1,
"settings": {
"index": {
"auto_expand_replicas": "0-1",
"max_docvalue_fields_search": 200,
"number_of_replicas": "0",
"number_of_shards": "3",
"query.default_field": [
"agent.host.architecture",
"agent.host.ip",
"agent.id",
"agent.name",
"agent.version",
"wazuh.cluster.name",
"wazuh.cluster.node",
"wazuh.schema.version"
],
"refresh_interval": "2s"
},
"mapping.nested_fields.limit": 50,
"mapping.total_fields.limit": 3000,
"plugins.index_state_management.rollover_alias": "wazuh-events-v5-system-activity"
}
}
}

53
plugins/setup/src/test/java/com/wazuh/setup/index/IndexStateManagementTests.java
View File

@ -29,10 +29,9 @@ import org.opensearch.transport.client.Client;
import org.opensearch.transport.client.IndicesAdminClient;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import com.wazuh.setup.utils.IndexUtils;
import com.wazuh.setup.utils.JsonUtils;
import static org.mockito.Mockito.*;
@ -42,7 +41,7 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
private IndexStateManagement ismIndex;
private Client client;
private IndicesAdminClient indicesAdminClient;
private IndexUtils indexUtils;
private JsonUtils jsonUtils;
@Override
public void setUp() throws Exception {
@ -51,7 +50,7 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
this.client = mock(Client.class);
AdminClient adminClient = mock(AdminClient.class);
this.indicesAdminClient = mock(IndicesAdminClient.class);
this.indexUtils = mock(IndexUtils.class);
this.jsonUtils = mock(JsonUtils.class);
// Default settings
ClusterService clusterService = mock(ClusterService.class);
@ -62,9 +61,9 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
doReturn(this.indicesAdminClient).when(adminClient).indices();
this.ismIndex =
spy(new IndexStateManagement(IndexStateManagement.ISM_INDEX_NAME, "ism-template"));
spy(new IndexStateManagement(IndexStateManagement.ISM_INDEX_NAME, "templates/ism-config"));
this.ismIndex.setClient(this.client);
this.ismIndex.setIndexUtils(this.indexUtils);
this.ismIndex.setUtils(this.jsonUtils);
this.ismIndex.setClusterService(clusterService);
}
@ -75,33 +74,33 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
* @throws IOException if an error occurs while reading the policy file
*/
public void testInitialize_CreatesIndexAndPolicies() throws IOException {
Map<String, Object> template = new HashMap<>();
template.put("settings", Settings.builder().build());
template.put("mappings", Map.of());
// Mock indexExists to return false so createIndex is called
doReturn(false).when(this.ismIndex).indexExists(IndexStateManagement.ISM_INDEX_NAME);
doReturn(template).when(this.indexUtils).fromFile("ism-template.json");
doReturn(template.get("mappings")).when(this.indexUtils).get(template, "mappings");
// Mock the CreateIndexResponse
CreateIndexResponse createResponse = mock(CreateIndexResponse.class);
doReturn(IndexStateManagement.ISM_INDEX_NAME).when(createResponse).index();
ActionFuture actionFuture = mock(ActionFuture.class);
doReturn(actionFuture).when(this.indicesAdminClient).create(any(CreateIndexRequest.class));
ActionFuture<CreateIndexResponse> createIndexFuture = mock(ActionFuture.class);
doReturn(createResponse).when(createIndexFuture).actionGet(anyLong());
doReturn(createIndexFuture).when(this.indicesAdminClient).create(any(CreateIndexRequest.class));
// Mock the policy file loading
Map<String, Object> policyFile = Map.of("policy", "definition");
doReturn(policyFile)
.when(this.indexUtils)
.when(this.jsonUtils)
.fromFile(IndexStateManagement.STREAM_ROLLOVER_POLICY_PATH);
doReturn(actionFuture).when(this.client).index(any(IndexRequest.class));
doReturn(createResponse).when(actionFuture).actionGet(anyLong());
// Mock the policy indexing
ActionFuture indexFuture = mock(ActionFuture.class);
doReturn(indexFuture).when(this.client).index(any(IndexRequest.class));
doReturn(null).when(indexFuture).actionGet(anyLong());
this.ismIndex.initialize();
// Verify that the index was created with the correct request
verify(this.indicesAdminClient).create(any(CreateIndexRequest.class));
// Verify that the policy was indexed
verify(this.client).index(any(IndexRequest.class));
}
@ -128,7 +127,7 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
public void testPolicyFileMissing_LogsError() throws IOException {
doReturn(true).when(this.ismIndex).indexExists(IndexStateManagement.ISM_INDEX_NAME);
doThrow(new IOException("file not found"))
.when(indexUtils)
.when(jsonUtils)
.fromFile(IndexStateManagement.STREAM_ROLLOVER_POLICY_PATH);
this.ismIndex.initialize();
@ -147,9 +146,7 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
doReturn(true).when(this.ismIndex).indexExists(IndexStateManagement.ISM_INDEX_NAME);
Map<String, Object> policyFile = Map.of("policy", "definition");
doReturn(policyFile)
.when(indexUtils)
.fromFile(IndexStateManagement.STREAM_ROLLOVER_POLICY_PATH);
doReturn(policyFile).when(jsonUtils).fromFile(IndexStateManagement.STREAM_ROLLOVER_POLICY_PATH);
doThrow(new ResourceAlreadyExistsException("already exists"))
.when(this.client)
.index(any(IndexRequest.class));
@ -158,4 +155,14 @@ public class IndexStateManagementTests extends OpenSearchTestCase {
// Verifies that exception is caught and logged
}
/**
* Verifies that IOException while reading a file is caught and logged.
*
* @throws IOException if there is an error reading the file
*/
public void testFileIOException() throws IOException {
doThrow(new IOException("Test failed successfully")).when(this.jsonUtils).fromFile(anyString());
this.ismIndex.indexPolicy("test-template");
}
}

60
plugins/setup/src/test/java/com/wazuh/setup/index/IndexTests.java
View File

@ -18,7 +18,6 @@ package com.wazuh.setup.index;
import org.opensearch.action.admin.indices.create.CreateIndexRequest;
import org.opensearch.action.admin.indices.create.CreateIndexResponse;
import org.opensearch.action.admin.indices.template.put.PutIndexTemplateRequest;
import org.opensearch.action.support.clustermanager.AcknowledgedResponse;
import org.opensearch.cluster.ClusterState;
import org.opensearch.cluster.routing.RoutingTable;
@ -30,11 +29,7 @@ import org.opensearch.transport.client.AdminClient;
import org.opensearch.transport.client.Client;
import org.opensearch.transport.client.IndicesAdminClient;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import com.wazuh.setup.utils.IndexUtils;
import com.wazuh.setup.utils.JsonUtils;
import static org.mockito.Mockito.*;
@ -42,21 +37,22 @@ import static org.mockito.Mockito.*;
public class IndexTests extends OpenSearchTestCase {
private Index index;
private Client client;
private IndicesAdminClient indicesAdminClient;
private RoutingTable routingTable;
private IndexUtils indexUtils;
private JsonUtils jsonUtils;
@Override
public void setUp() throws Exception {
super.setUp();
Client client = mock(Client.class);
this.client = mock(Client.class);
AdminClient adminClient = mock(AdminClient.class);
this.indicesAdminClient = mock(IndicesAdminClient.class);
ClusterService clusterService = mock(ClusterService.class);
this.routingTable = mock(RoutingTable.class);
ClusterState clusterState = mock(ClusterState.class);
this.indexUtils = mock(IndexUtils.class);
this.jsonUtils = mock(JsonUtils.class);
// Default settings
Settings settings = Settings.builder().build();
@ -64,11 +60,11 @@ public class IndexTests extends OpenSearchTestCase {
// Concrete implementation of abstract class
this.index = new Index("test-index", "test-template") {};
this.index.setClient(client);
this.index.setClient(this.client);
this.index.setClusterService(clusterService);
this.index.setIndexUtils(indexUtils);
this.index.setUtils(jsonUtils);
doReturn(adminClient).when(client).admin();
doReturn(adminClient).when(this.client).admin();
doReturn(this.indicesAdminClient).when(adminClient).indices();
doReturn(clusterState).when(clusterService).state();
doReturn(this.routingTable).when(clusterState).getRoutingTable();
@ -98,43 +94,21 @@ public class IndexTests extends OpenSearchTestCase {
verify(this.indicesAdminClient, never()).create(any());
}
/**
* Verifies that template creation is successful when valid data is returned from file.
*
* @throws IOException if there is an error reading the template file
*/
public void testCreateTemplateSuccess() throws IOException {
Map<String, Object> templateMap =
Map.of(
"settings", Settings.builder().build(),
"mappings", Map.of(),
"index_patterns", List.of("test-*"));
doReturn(templateMap).when(this.indexUtils).fromFile("test-template.json");
doReturn(templateMap.get("mappings")).when(this.indexUtils).get(templateMap, "mappings");
/** Verifies that template creation is successful when valid data is returned from file. */
public void testCreateTemplateSuccess() {
// Mock the response for client.execute()
AcknowledgedResponse ackResponse = mock(AcknowledgedResponse.class);
ActionFuture actionFuture = mock(ActionFuture.class);
doReturn(ackResponse).when(actionFuture).actionGet(anyLong());
doReturn(actionFuture)
.when(this.indicesAdminClient)
.putTemplate(any(PutIndexTemplateRequest.class));
this.index.createTemplate("test-template");
verify(this.indicesAdminClient).putTemplate(any(PutIndexTemplateRequest.class));
}
// Mock client.execute() which is what createTemplate actually uses
doReturn(actionFuture).when(this.client).execute(any(), any());
/**
* Verifies that IOException while reading template file is caught and logged.
*
* @throws IOException if there is an error reading the template file
*/
public void testCreateTemplateIOException() throws IOException {
doThrow(new IOException("test")).when(this.indexUtils).fromFile("test-template.json");
// Call createTemplate with a real template file that exists
this.index.createTemplate("templates/ism-config");
this.index.createTemplate("test-template");
// Expect error to be logged but not thrown
// Verify that client.execute was called
verify(this.client).execute(any(), any());
}
/** Verifies that initialize() invokes both createTemplate and createIndex in order. */

69
plugins/setup/src/test/java/com/wazuh/setup/index/StreamIndexTests.java
View File

@ -16,9 +16,7 @@
*/
package com.wazuh.setup.index;
import org.opensearch.action.admin.indices.alias.Alias;
import org.opensearch.action.admin.indices.create.CreateIndexRequest;
import org.opensearch.action.admin.indices.create.CreateIndexResponse;
import org.opensearch.action.support.clustermanager.AcknowledgedResponse;
import org.opensearch.cluster.ClusterState;
import org.opensearch.cluster.routing.RoutingTable;
import org.opensearch.cluster.service.ClusterService;
@ -29,7 +27,7 @@ import org.opensearch.transport.client.AdminClient;
import org.opensearch.transport.client.Client;
import org.opensearch.transport.client.IndicesAdminClient;
import com.wazuh.setup.utils.IndexUtils;
import com.wazuh.setup.utils.JsonUtils;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.*;
@ -37,9 +35,9 @@ import static org.mockito.Mockito.*;
/** Unit tests for the {@link StreamIndex} class. */
public class StreamIndexTests extends OpenSearchTestCase {
private static final String STREAM_INDEX = "stream-index";
private StreamIndex streamIndex;
private IndicesAdminClient indicesAdminClient;
private RoutingTable routingTable;
@Override
public void setUp() throws Exception {
@ -49,57 +47,56 @@ public class StreamIndexTests extends OpenSearchTestCase {
AdminClient adminClient = mock(AdminClient.class);
this.indicesAdminClient = mock(IndicesAdminClient.class);
ClusterService clusterService = mock(ClusterService.class);
this.routingTable = mock(RoutingTable.class);
RoutingTable routingTable = mock(RoutingTable.class);
ClusterState clusterState = mock(ClusterState.class);
// Default settings
Settings settings = Settings.builder().build();
doReturn(settings).when(clusterService).getSettings();
this.streamIndex = new StreamIndex("stream-index", "stream-template", "stream-alias");
this.streamIndex = new StreamIndex(STREAM_INDEX, "stream-template");
this.streamIndex.setClient(client);
this.streamIndex.setClusterService(clusterService);
this.streamIndex.setIndexUtils(mock(IndexUtils.class));
this.streamIndex.setUtils(mock(JsonUtils.class));
doReturn(adminClient).when(client).admin();
doReturn(this.indicesAdminClient).when(adminClient).indices();
doReturn(clusterState).when(clusterService).state();
doReturn(this.routingTable).when(clusterState).getRoutingTable();
doReturn(routingTable).when(clusterState).getRoutingTable();
}
/**
* Verifies that createIndex adds the alias and calls the create method when the index does not
* exist.
* Verifies that createIndex handles ResourceAlreadyExistsException gracefully when the data
* stream already exists.
*/
public void testCreateIndexWithAlias() {
doReturn(false).when(this.routingTable).hasIndex("stream-index");
public void testCreateIndexWhenAlreadyExists() {
// Mock createDataStream to throw ResourceAlreadyExistsException
ActionFuture actionFuture = mock(ActionFuture.class);
doThrow(new org.opensearch.ResourceAlreadyExistsException("Data stream already exists"))
.when(actionFuture)
.actionGet(anyLong());
doReturn(actionFuture).when(this.indicesAdminClient).createDataStream(any());
// Should not throw exception - it should be caught and logged
this.streamIndex.createIndex(STREAM_INDEX);
// Verify createDataStream was called once
verify(this.indicesAdminClient).createDataStream(any());
}
/** Verifies that createIndex successfully creates a data stream when it doesn't exist. */
public void testCreateIndexSuccess() {
// Mock successful data stream creation
AcknowledgedResponse response = mock(AcknowledgedResponse.class);
// doReturn(true).when(response).isAcknowledged();
CreateIndexResponse response = mock(CreateIndexResponse.class);
doReturn("stream-index").when(response).index();
ActionFuture actionFuture = mock(ActionFuture.class);
doReturn(response).when(actionFuture).actionGet(anyLong());
doReturn(actionFuture).when(this.indicesAdminClient).create(any(CreateIndexRequest.class));
doReturn(actionFuture).when(this.indicesAdminClient).createDataStream(any());
this.streamIndex.createIndex("stream-index");
this.streamIndex.createIndex(STREAM_INDEX);
verify(this.indicesAdminClient)
.create(
argThat(
req -> {
Alias alias = req.aliases().stream().findFirst().orElse(null);
return req.index().equals("stream-index")
&& alias != null
&& "stream-alias".equals(alias.name())
&& Boolean.TRUE.equals(alias.writeIndex());
}));
}
/** Verifies that createIndex skips index creation if the index already exists. */
public void testCreateIndexWhenAlreadyExists() {
doReturn(true).when(this.routingTable).hasIndex("stream-index");
this.streamIndex.createIndex("stream-index");
verify(this.indicesAdminClient, never()).create(any());
// Verify createDataStream was called
verify(this.indicesAdminClient).createDataStream(any());
}
}