wazuh-indexer-plugins

wazuh/wazuh-indexer-plugins

Fork 0

mirror of https://github.com/wazuh/wazuh-indexer-plugins.git synced 2025-12-11 18:44:00 -06:00

Commit Graph

Author	SHA1	Message	Date
Jorge Sánchez	c409e6465c	Fix verify_integrations script to read the integrations from module_list.txt (#640 ) * Fix verify_integrations script to read the integrations from module_list.txt * Add Changelog entry --------- Co-authored-by: Álex Ruiz Becerra <alejandro.ruiz.becerra@wazuh.com>	2025-11-05 15:01:09 +01:00
Kevin Ledesma	9de219fd97	Implement pre-processing ECS sources types sanitization (#628 ) * Add ECS source files sanitizer script * Simplify sanitizer and improve logging * Update schema_sanitizer to handle specific fields removal * Update the ecs generator Dockerfile to use the sanitizer * Improve sanitizer filtering * Remove --ref flag from ECS generator to force the tool to use local source * Update the Wazuh Common Schema * Fix multi-fields deletion on sanitizer script * Update the Wazuh Common Schema * Fix nested fields sanitization and remove multi-fields specific fields list * Update the Wazuh Common Schema * Update documentation from README and scripts Remove commented-out code Improve methods docstrings Add new script to README * Restore WCS mappings modification * Rename constants and fix styling * Update CHANGELOG * Revert changes on cloud-services-azure module * Re-build WCS mappings using sanitized ECS definitions (#636) * Re-build all the WCS index templates using the sanitized ECS * Do not remove @timestamp on stateless modules --------- Co-authored-by: Alex Ruiz <alejandro.ruiz.becerra@wazuh.com> * Replace leftovers from ecs folder restructuring Replace occurences of 'stateless-' with 'stateless/' --------- Co-authored-by: Wazuh Indexer Bot <github_devel_xdrsiem_indexer@wazuh.com> Co-authored-by: Alex Ruiz <alejandro.ruiz.becerra@wazuh.com>	2025-11-03 18:51:58 +01:00
Álex Ruiz Becerra	93ef5f1f67	Add tooling and index definitions for Wazuh decoders (#581 ) * Add initial WCS tooling for integrations and index definitions * Add changelog and index creation * Run mappings generator on pull request sync * Fix mappings generator * Bump versions - ECS: v8.11.0 -> v9.1.0 - Python image: 3.10 -> 3.13-slim * Apply rollover policy to wazuh-events indices * Use short descriptions * Add support for ECS v9.1.0 * Remove on_push trigger for the mappings generator Workflow * Update generate_and_push_templates.sh to commit new (untracked) files * Fix glob pattern * Set checkout branch in mappings generator workflow * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Extend mappings limits Remove synthetic_source_keep from the index templates * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Fix typo reading CSV column name Was using Wazuh Type instead of 'Wazuh type', causing all custom fields to use the default data type 'keyword' * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Fix fortinet.yml and add the new indices to the setup plugin * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Update azure fields csv to trigger index creation * Add azure template * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Create test tool * Add root: true to ignore the integration name as part of the field names * Add concurrency limit to the workflow to generate index templates * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Re-run with updated CSV * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Keep base fields for every stateless indices * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Upgrade test tool * Delete extra field from f5 fields * Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek * Add missing iss index creation * Add more integrations Replace underscore with dash on integrations names * Fix flaky integration tests Replace custom code to check cluster is green with the OpenSearchIntegTestCase::ensureGreen() method, which fits this purpose * Add more modules and enhance tooling * Add back path output on navigate_to_project_root function * Add ecs/scripts/module_list.txt to the files to commit * Update the Wazuh Common Schema * Re-structure tooling * Fix links and filter of modified files * Improve detecting of modified files to commit and push * Add creation of more indices Rename modsec to modsecurity * Update the Wazuh Common Schema * Fix Cisco IOS and AWS templates * Increase limit of nested fields for the AWS indices * Update the Wazuh Common Schema * Add suggestions from code review --------- Co-authored-by: Wazuh Indexer Bot <github_devel_xdrsiem_indexer@wazuh.com> Co-authored-by: Jorge Sanchez <jorge.sanchez@wazuh.com>	2025-10-14 16:39:11 +02:00

Author

SHA1

Message

Date

Jorge Sánchez

c409e6465c

Fix verify_integrations script to read the integrations from module_list.txt (#640 )

* Fix verify_integrations script to read the integrations from module_list.txt

* Add Changelog entry

---------

Co-authored-by: Álex Ruiz Becerra <alejandro.ruiz.becerra@wazuh.com>

2025-11-05 15:01:09 +01:00

Kevin Ledesma

9de219fd97

Implement pre-processing ECS sources types sanitization (#628 )

* Add ECS source files sanitizer script

* Simplify sanitizer and improve logging

* Update schema_sanitizer to handle specific fields removal

* Update the ecs generator Dockerfile to use the sanitizer

* Improve sanitizer filtering

* Remove --ref flag from ECS generator to force the tool to use local source

* Update the Wazuh Common Schema

* Fix multi-fields deletion on sanitizer script

* Update the Wazuh Common Schema

* Fix nested fields sanitization and remove multi-fields specific fields list

* Update the Wazuh Common Schema

* Update documentation from README and scripts

Remove commented-out code

Improve methods docstrings

Add new script to README

* Restore WCS mappings modification

* Rename constants and fix styling

* Update CHANGELOG

* Revert changes on cloud-services-azure module

* Re-build WCS mappings using sanitized ECS definitions (#636)

* Re-build all the WCS index templates using the sanitized ECS

* Do not remove @timestamp on stateless modules

---------

Co-authored-by: Alex Ruiz <alejandro.ruiz.becerra@wazuh.com>

* Replace leftovers from ecs folder restructuring

Replace occurences of 'stateless-' with 'stateless/'

---------

Co-authored-by: Wazuh Indexer Bot <github_devel_xdrsiem_indexer@wazuh.com>
Co-authored-by: Alex Ruiz <alejandro.ruiz.becerra@wazuh.com>

2025-11-03 18:51:58 +01:00

Álex Ruiz Becerra

93ef5f1f67

Add tooling and index definitions for Wazuh decoders (#581 )

* Add initial WCS tooling for integrations and index definitions

* Add changelog and index creation

* Run mappings generator on pull request sync

* Fix mappings generator

* Bump versions

- ECS: v8.11.0 -> v9.1.0
- Python image: 3.10 -> 3.13-slim

* Apply rollover policy to wazuh-events indices

* Use short descriptions

* Add support for ECS v9.1.0

* Remove on_push trigger for the mappings generator Workflow

* Update generate_and_push_templates.sh to commit new (untracked) files

* Fix glob pattern

* Set checkout branch in mappings generator workflow

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Extend mappings limits

Remove synthetic_source_keep from the index templates

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Fix typo reading CSV column name

Was using Wazuh Type instead of 'Wazuh type', causing all custom fields to use the default data type 'keyword'

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Fix fortinet.yml and add the new indices to the setup plugin

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Update azure fields csv to trigger index creation

* Add azure template

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Create test tool

* Add root: true to ignore the integration name as part of the field names

* Add concurrency limit to the workflow to generate index templates

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Re-run with updated CSV

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Keep base fields for every stateless indices

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Upgrade test tool

* Delete extra field from f5 fields

* Update ECS templates for modified modules: stateless-amazon-security-lake stateless-apache_tomcat stateless-audit stateless-azure-app-service stateless-azure-metrics stateless-checkpoint stateless-cisco-asa stateless-cisco_umbrella stateless-f5-bigip stateless-fortinet stateless-gcp stateless-iis stateless-iptables stateless-microsoft-dhcp stateless-microsoft-dnsserver stateless-microsoft-exchange-server stateless-modsec stateless-oracle_weblogic stateless-pfsense stateless-snort stateless-spring_boot stateless-squid stateless-suricata stateless-template stateless-unifiedlogs stateless-websphere stateless-windows stateless-zeek

* Add missing iss index creation

* Add more integrations

Replace underscore with dash on integrations names

* Fix flaky integration tests

Replace custom code to check cluster is green with the OpenSearchIntegTestCase::ensureGreen() method, which fits this purpose

* Add more modules and enhance tooling

* Add back path output on navigate_to_project_root function

* Add ecs/scripts/module_list.txt to the files to commit

* Update the Wazuh Common Schema

* Re-structure tooling

* Fix links and filter of modified files

* Improve detecting of modified files to commit and push

* Add creation of more indices

Rename modsec to modsecurity

* Update the Wazuh Common Schema

* Fix Cisco IOS and AWS templates

* Increase limit of nested fields for the AWS indices

* Update the Wazuh Common Schema

* Add suggestions from code review

---------

Co-authored-by: Wazuh Indexer Bot <github_devel_xdrsiem_indexer@wazuh.com>
Co-authored-by: Jorge Sanchez <jorge.sanchez@wazuh.com>

2025-10-14 16:39:11 +02:00

3 Commits