wazuh-indexer/libs/common/build.gradle
Andriy Redko 7c46f8f14e
Update to Apache Lucene 10 for 3.0.0 (#16366)
* Update to Apache Lucene 10 for 3.0.0

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* More Lucene 10 build fixes

Core compiles now. (Plugins/modules don't yet.)

Major areas covered:

1. MultiTermQuery now takes RewriteMethod via constructor instead of
   setter.
2. Scorable no longer has docId.
3. ScoreCachingWrappingScorer now takes + returns LeafCollector.
4. MMapDirectory allows configuring preload per file (so we don't have
   to).
5. Concurrent search can now operate at sub-segment level (but we don't
   yet).

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Replace DocValuesFieldExistsQuery with FieldExistsQuery

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix signature changes in o.a.l.search.BooleanQuery

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix more compilation failures, update shadow Gradle plugin and fix loggerUsageCheck Gradle task

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix analysis-common compilation issues

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* More compilation fixes

Signed-off-by: Andrew Ross <andrross@amazon.com>

* More compilation fixes

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix assemble tasks

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures, add CHANGELOG.md

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix more compilation failures

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixed Lucene latest codec name

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Refactor preload tests in FsDirectoryFactoryTests

Also more compilation fixes.

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix more compilation failures

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix precommit failures

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix analysis test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix FieldData & ParentJoinAggregator ordinal traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix IpFieldMapper & CollapsingDocValuesSource ordinal traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix MultiValueMode, GlobalOrdinalValuesSource, BinaryRangeAggregator, DiversifiedOrdinalsSamplerAggregator, GlobalOrdinalsStringTermsAggregator & CardinalityAggregator ordinal traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix MultiValueModeTests, BinaryRangeAggregatorTests, IncludeExcludeTests & MissingValuesTests ordinal traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix ICUCollationKeywordFieldMapper & VectorFieldTests tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.common.xcontent.support.XContentMapValuesTests test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix FunctionScoreQuery::CustomBoostFactorWeight::scorerSupplier that may throw NPE

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix CodecTests and non-concurrent ContextIndexSearcher::search implementation for aggregations

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixed GlobalOrdinalMapping ordinals iteration and SortedWiderNumericSortField implementation of missingValueAsComparableLong

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix FieldData and SortedSetDocValuesWriterWrapperTests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.search.aggregations.bucket.composite.CompositeAggregatorTests test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix MultiValueMode ordinal traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix GlobalOrdinalMapping and GlobalOrdinalsStringTermsAggregator ordinal traversals

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix QueryStringQueryBuilderTests and QueryStringQueryBuilder fuzzy query default rewrite method detection logic

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix StarTree*DocValueFormatTests

The Composite912DocValuesReader was not honoring the IOContext
correctly. The metadata file should be opened with READONCE and
immediately be closed (on the same thread). The data file needs
to be opened with ReadAdvice.NORMAL to ensure that it's
closeable on another thread if necessary.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix various MultiTermQuery-related tests

Now that we use CONSTANT_SCORE_BLENDED_REWRITE by default for most
MTQs, we need to update the tests. Also FuzzyQuery should default to
TopTermsBlendedFreqScoringRewrite (taken from Lucene).

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix Spotless

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix RegEx automaton processing using Operations.determinize(...)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix compilation issues (post merge)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix UnsignedLongComparator::sortableBytesToLong encoding

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix HalfFloatComparator::sortableBytesToLong and HalfFloatComparator::missingValueAsComparableLong

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix infinite recursion in CollapsingTopDocsCollectorTests

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix SingleDimensionValuesSourceTests

Can no longer mock IndexReader as it's sealed.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix a few more MultiTermQuery-related test failures

Signed-off-by: Michael Froh <froh@amazon.com>

* Fixed QueryPhaseTests#minScoreOptimisation by not delegating setWeight to underlying delegate

Signed-off-by: expani <anijainc@amazon.com>

* Fix DerivedFieldMapperQueryTests

Needed to add doc values to date and IP fields.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix SortedNumericDocValuesWriterWrapperTests

Lucene's SortedNumericDocValuesWriter no longer throws an exception if
you try to iterate past a document's last value. (Instead, it happily
leaks the next document's value.)

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix IpRangeAggregatorTests

Rewrote a for-loop in BinaryRangeAggregator to be less confusing. As a
side-effect, it fixed the test.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix IpFieldMapperTests

I broke these tests as a side-effect for my fix to
DerivedFieldMapperQueryTests.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix MissingValues ordinal traversal and MultiOrdinalsTests tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Prevent cascading test failures in SearchServiceTests

I'm not really fixing the failing tests, but ensuring that the two
tests that fail don't cause a bunch of other tests to fail by
leaving transient settings around. (Clean up the transient settings
even if those tests fail.)

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix DerivedFieldTypeTests

I broke these when fixing DerivedFieldMapperQueryTests.

Signed-off-by: Michael Froh <froh@amazon.com>

* Fix TransferManagerRemoteDirectoryReaderTests

Needed to update mocked calls.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add missing javadoc

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Uploaded Index generated from Lucene 8.x ( OpenSearch 1.3.0 ) to be used by UnitTests

Signed-off-by: expani <anijainc@amazon.com>

* Moved path to a constant

Signed-off-by: expani <anijainc@amazon.com>

* Update Apache Lucene to 10.1.0

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixed unit tests for lucene 10 upgrade

Signed-off-by: expani <anijainc@amazon.com>

* Fix posting format for CompletionFieldMapper

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.search.SearchServiceTests test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Removed extra iteration of DISI and added stored field option to bypass modified check in lucene Field

Signed-off-by: expani <anijainc@amazon.com>

* Changed to use advance and init to use docId() instead of nextDoc()

Signed-off-by: expani <anijainc@amazon.com>

* Increased number of indexed docs to match segment count

Signed-off-by: expani <anijainc@amazon.com>

* Reduced segment count to match assertion instead of increasing docs

Signed-off-by: expani <anijainc@amazon.com>

* Revert to make 90_search_after integ pass as hits are not shown

Signed-off-by: expani <anijainc@amazon.com>

* Making Half Float failures work with it's conversions

Signed-off-by: expani <anijainc@amazon.com>

* Changing max long to signed from unsigned

Signed-off-by: expani <anijainc@amazon.com>

* Addressed PR Comments

Signed-off-by: expani <anijainc@amazon.com>

* Trying another approach for gracefully handling unsigned long conversions

Signed-off-by: expani <anijainc@amazon.com>

* Simplified unsigned to signed long conversion

Signed-off-by: expani <anijainc@amazon.com>

* Reverted changes as need to think more on this

Signed-off-by: expani <anijainc@amazon.com>

* Fix azure-fixture & s3-fixture Docker images

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix 'search/90_search_after/numeric skipping logic with competitive missing value' test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix gcs-fixture Docker images

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Updated to use modern Romanian Unicode

Signed-off-by: expani <anijainc@amazon.com>

* Fix org.opensearch.wildfly.WildflyIT test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.search.aggregations.bucket.MinDocCountIT test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix CardinalityAggregator.OrdinalsCollector ordinals traversal

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.search.query.ScriptScoreQueryIT test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix org.opensearch.search.scriptfilter.ScriptQuerySearchIT test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix UnsignedLongComparator top/bottom as comparable long conversions

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix post-merge (with main) compilation issues

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix UnsignedLongComparator with respect to sortable bytes to long conversion

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix FlatObjectFieldTypeTests test cases

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: expani <anijainc@amazon.com>
Co-authored-by: Michael Froh <froh@amazon.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: expani <anijainc@amazon.com>
2025-01-22 16:12:41 -05:00

66 lines
1.8 KiB
Groovy

/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*/
import org.opensearch.gradle.info.BuildParams
apply plugin: 'opensearch.publish'
ext {
// Do not fail on 'warning: using incubating module(s): jdk.incubator.vector'
failOnJavadocWarning = false
}
base {
archivesName = 'opensearch-common'
}
dependencies {
// This dependency is used only by :libs:core for null-checking interop with other tools
compileOnly "com.google.code.findbugs:jsr305:3.0.2"
/*******
* !!!! NO THIRD PARTY DEPENDENCIES !!!!
*******/
testImplementation "com.carrotsearch.randomizedtesting:randomizedtesting-runner:${versions.randomizedrunner}"
testImplementation "junit:junit:${versions.junit}"
testImplementation "org.hamcrest:hamcrest:${versions.hamcrest}"
testImplementation(project(":test:framework")) {
exclude group: 'org.opensearch', module: 'opensearch-common'
}
}
tasks.withType(JavaCompile).configureEach {
options.compilerArgs -= '-Xlint:unchecked'
}
tasks.named('forbiddenApisMain').configure {
// :libs:opensearch-common does not depend on server
// TODO: Need to decide how we want to handle for forbidden signatures with the changes to server
replaceSignatureFiles 'jdk-signatures'
failOnMissingClasses = false
ignoreSignaturesOfMissingClasses = true
}
compileJava {
options.compilerArgs += ['--add-modules', 'jdk.incubator.vector']
options.compilerArgs -= '-Werror' // use of incubator modules is reported as a warning
}
test {
systemProperty 'opensearch.experimental.feature.simd.rounding.enabled', 'forced'
}
javadoc {
options.addStringOption("-add-modules", "jdk.incubator.vector")
}