This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new d8dc1f0f2ac7 test: add unit test coverage for secondary index,
expression, and bloom filter classes in hudi-common (#19204)
d8dc1f0f2ac7 is described below
commit d8dc1f0f2ac730fd293f738075986f12f616d081
Author: Rahil C <[email protected]>
AuthorDate: Wed Aug 5 20:35:29 2026 -0700
test: add unit test coverage for secondary index, expression, and bloom
filter classes in hudi-common (#19204)
* test: add unit test coverage for secondary index, expression, and bloom
filter classes in hudi-common
Adds JUnit tests for pure-logic classes that had 0% or low line coverage:
- core/index/secondary/SecondaryIndexManager: create/drop/show, duplicate
and
missing-index detection, schema validation (mocks HoodieTableMetaClient
since hudi-common has no filesystem-backed HoodieStorage impl available at
test time; intercepts the static HoodieTableConfig#update/#delete calls
and
applies them to an in-memory HoodieTableConfig to faithfully emulate
persistence)
- core/index/secondary/HoodieSecondaryIndex: builder, Lucene single-column
validation, toString, HoodieIndexCompactor
- core/index/secondary/SecondaryIndexUtils: JSON round trip, malformed
input,
unknown-property tolerance
- core/index/secondary/SecondaryIndexType: of(byte)/of(String) success and
failure paths
- common/bloom/Key: equals/hashCode/compareTo/serde
- common/bloom/SimpleBloomFilter: add/mightContain, serialize round trip,
or()
- common/bloom/InternalDynamicBloomFilter: growth across number of keys,
and/or/xor/not, write/readFields round trip (extends existing test class)
- common/expression/Predicates: eval and null-handling for each predicate
type
(extends existing test class)
- common/expression/PartialBindVisitor: missing-field branches for Not/In/
IsNull/IsNotNull/StringStartsWith/StringContains, unsupported predicate
type
(extends existing test class)
- common/expression/BindVisitor: full binding, missing-field errors, And/Or
short-circuit and simplification branches (new test class)
* test: apply self-review cleanup to secondary index and bloom filter tests
- Use assertFalse instead of assertTrue(!x) in TestSimpleBloomFilter
- Extract a shared SecondaryIndexTestUtils fixture helper to remove
duplicated columns-map/HoodieSecondaryIndex construction across
TestHoodieSecondaryIndex, TestSecondaryIndexManager, and
TestSecondaryIndexUtils
- Move the trivial getInstance() singleton test out of
TestSecondaryIndexManager into its own class so it no longer pays for
that class's mocked-metaClient fixture
* test: follow the SecondaryIndexUtils -> SecondaryIndexDefinitionUtils
rename
The class this test covers was renamed by #19212 while this PR was open.
Rename the test class and its references to match.
* test: address review feedback on secondary index and bloom filter tests
Addresses @voonhous's review comments:
- Drop TestSecondaryIndexManagerSingleton and inline its one check into
TestSecondaryIndexManager. Use assertSame rather than assertEquals, since
SecondaryIndexManager has no equals() override and assertEquals only reads
as value equality while silently degrading to reference equality.
- TestSimpleBloomFilter: drop testAddAndMightContain, testOrMergesTwoFilters
and testSerializeToStringRoundTrip, which duplicate what the parameterized
TestBloomFilter already runs for SIMPLE. The remaining cases are the ones
TestBloomFilter does not cover: negative lookups (its testAddKey only
asserts positive membership), null handling, the ByteBuffer constructor,
getBloomFilterTypeCode and or() type enforcement.
- TestHoodieSecondaryIndex: drop testLuceneIndexWithSingleColumnIsValid. Its
size() == 1 assertion could only fail if the fixture helper itself broke,
and every other test in the class already constructs a valid single-column
Lucene index through the validating constructor.
- Extract the write/readFields round trip into a package-local
BloomSerDeTestUtils, replacing the three copies in TestKey,
TestInternalDynamicBloomFilter and TestInternalBloomFilter. Key and
InternalFilter share no supertype declaring write(DataOutput), so the
helper takes the write call (serialize(original::write)) instead of the
object being written.
---
.../hudi/common/bloom/BloomSerDeTestUtils.java | 64 +++++
.../hudi/common/bloom/TestInternalBloomFilter.java | 14 +-
.../bloom/TestInternalDynamicBloomFilter.java | 161 +++++++++++
.../java/org/apache/hudi/common/bloom/TestKey.java | 138 ++++++++++
.../hudi/common/bloom/TestSimpleBloomFilter.java | 92 +++++++
.../hudi/common/expression/TestBindVisitor.java | 305 +++++++++++++++++++--
.../common/expression/TestPartialBindVisitor.java | 156 +++++++++++
.../hudi/common/expression/TestPredicates.java | 260 ++++++++++++++++++
.../index/secondary/SecondaryIndexTestUtils.java | 52 ++++
.../index/secondary/TestHoodieSecondaryIndex.java | 103 +++++++
.../TestSecondaryIndexDefinitionUtils.java | 110 ++++++++
.../index/secondary/TestSecondaryIndexManager.java | 218 +++++++++++++++
.../index/secondary/TestSecondaryIndexType.java | 62 +++++
13 files changed, 1709 insertions(+), 26 deletions(-)
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/BloomSerDeTestUtils.java
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/BloomSerDeTestUtils.java
new file mode 100644
index 000000000000..d52c4985482e
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/BloomSerDeTestUtils.java
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.bloom;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInput;
+import java.io.DataInputStream;
+import java.io.DataOutput;
+import java.io.DataOutputStream;
+import java.io.IOException;
+
+/**
+ * Write/readFields helpers shared by the {@code org.apache.hudi.common.bloom}
tests.
+ *
+ * <p>{@link Key} and {@link InternalFilter} both expose {@code
write(DataOutput)} /
+ * {@code readFields(DataInput)} but share no supertype declaring them, so the
helper takes the
+ * write call itself rather than the object being written.
+ */
+final class BloomSerDeTestUtils {
+
+ /**
+ * The {@code write(DataOutput)} method of the object under test, e.g.
{@code key::write}.
+ */
+ @FunctionalInterface
+ interface DataWriter {
+ void write(DataOutput out) throws IOException;
+ }
+
+ private BloomSerDeTestUtils() {
+ }
+
+ /**
+ * Serializes through {@code writer} and returns the bytes it produced.
+ */
+ static byte[] serialize(DataWriter writer) throws IOException {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ writer.write(new DataOutputStream(baos));
+ return baos.toByteArray();
+ }
+
+ /**
+ * Wraps {@code bytes} as a {@link DataInput} for the matching {@code
readFields} call.
+ */
+ static DataInput asDataInput(byte[] bytes) {
+ return new DataInputStream(new ByteArrayInputStream(bytes));
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalBloomFilter.java
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalBloomFilter.java
index 175fe725ba6c..20f6874b895b 100644
---
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalBloomFilter.java
+++
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalBloomFilter.java
@@ -22,10 +22,6 @@ import org.apache.hudi.common.util.hash.Hash;
import org.junit.jupiter.api.Test;
-import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
-import java.io.DataInputStream;
-import java.io.DataOutputStream;
import java.io.IOException;
import java.util.Arrays;
import java.util.BitSet;
@@ -89,7 +85,7 @@ public class TestInternalBloomFilter {
}
byte[] serialized = serialize(filter);
InternalBloomFilter deserialized = new InternalBloomFilter();
- deserialized.readFields(new DataInputStream(new
ByteArrayInputStream(serialized)));
+ deserialized.readFields(BloomSerDeTestUtils.asDataInput(serialized));
for (Key key : keys) {
assertTrue(deserialized.membershipTest(key));
}
@@ -113,7 +109,7 @@ public class TestInternalBloomFilter {
// The last byte carries bits 56..60; bits 61..63 are beyond vectorSize
and must be ignored.
mutated[mutated.length - 1] |= (byte) 0xE0;
InternalBloomFilter deserialized = new InternalBloomFilter();
- deserialized.readFields(new DataInputStream(new
ByteArrayInputStream(mutated)));
+ deserialized.readFields(BloomSerDeTestUtils.asDataInput(mutated));
for (Key key : keys) {
assertTrue(deserialized.membershipTest(key));
}
@@ -210,9 +206,7 @@ public class TestInternalBloomFilter {
}
private static byte[] serialize(InternalBloomFilter filter) throws
IOException {
- ByteArrayOutputStream baos = new ByteArrayOutputStream();
- filter.write(new DataOutputStream(baos));
- return baos.toByteArray();
+ return BloomSerDeTestUtils.serialize(filter::write);
}
/** Packs the oracle bits with the Hadoop BloomFilter byte layout: bit i at
byte i >> 3, mask 1 << (i & 7). */
@@ -228,7 +222,7 @@ public class TestInternalBloomFilter {
private static InternalBloomFilter copy(InternalBloomFilter filter) throws
IOException {
InternalBloomFilter copied = new InternalBloomFilter();
- copied.readFields(new DataInputStream(new
ByteArrayInputStream(serialize(filter))));
+ copied.readFields(BloomSerDeTestUtils.asDataInput(serialize(filter)));
return copied;
}
}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalDynamicBloomFilter.java
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalDynamicBloomFilter.java
index 888d9fb561d1..4e20a93581dc 100644
---
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalDynamicBloomFilter.java
+++
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestInternalDynamicBloomFilter.java
@@ -22,9 +22,12 @@ import org.apache.hudi.common.util.hash.Hash;
import org.junit.jupiter.api.Test;
+import java.io.IOException;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
@@ -71,4 +74,162 @@ public class TestInternalDynamicBloomFilter {
HoodieDynamicBoundedBloomFilter rescaledToSize4Filter =
rescaledToSize2Filter.rescaleFromTarget(4);
assertEquals(4, rescaledToSize4Filter.getMatrixLength());
}
+
+ @Test
+ public void testAddNullKeyThrows() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ assertThrows(NullPointerException.class, () -> filter.add((Key) null));
+ }
+
+ @Test
+ public void testMembershipTestNullKeyReturnsTrue() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ assertTrue(filter.membershipTest(null));
+ }
+
+ @Test
+ public void testAddGrowsMatrixAcrossNumberOfKeys() {
+ int keysPerRow = 10;
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, keysPerRow, 1000);
+ assertEquals(1, filter.getMatrixLength());
+
+ for (int i = 0; i < keysPerRow; i++) {
+ filter.add(new Key(("key" + i).getBytes()));
+ }
+ // The first row is now full but growth is deferred until the next add.
+ assertEquals(1, filter.getMatrixLength());
+
+ filter.add(new Key("one-more-key".getBytes()));
+ assertEquals(2, filter.getMatrixLength());
+ }
+
+ @Test
+ public void testMembershipTestFindsAddedKeyAcrossRows() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 2, 1000);
+ for (int i = 0; i < 20; i++) {
+ filter.add(new Key(("key" + i).getBytes()));
+ }
+ assertTrue(filter.getMatrixLength() > 1);
+ for (int i = 0; i < 20; i++) {
+ assertTrue(filter.membershipTest(new Key(("key" + i).getBytes())));
+ }
+ }
+
+ @Test
+ public void testAndThrowsForIncompatibleFilter() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ InternalDynamicBloomFilter differentVectorSize = new
InternalDynamicBloomFilter(500, 5, Hash.MURMUR_HASH, 10, 1000);
+ assertThrows(IllegalArgumentException.class, () ->
filter.and(differentVectorSize));
+ }
+
+ @Test
+ public void testAndKeepsCommonKey() {
+ InternalDynamicBloomFilter filter1 = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ InternalDynamicBloomFilter filter2 = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ Key commonKey = new Key("common-key".getBytes());
+ filter1.add(commonKey);
+ filter2.add(commonKey);
+
+ filter1.and(filter2);
+
+ assertTrue(filter1.membershipTest(commonKey));
+ }
+
+ @Test
+ public void testOrThrowsForIncompatibleFilter() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ InternalDynamicBloomFilter differentNbHash = new
InternalDynamicBloomFilter(1000, 6, Hash.MURMUR_HASH, 10, 1000);
+ assertThrows(IllegalArgumentException.class, () ->
filter.or(differentNbHash));
+ }
+
+ @Test
+ public void testOrMergesKeysFromBothFilters() {
+ InternalDynamicBloomFilter filter1 = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ InternalDynamicBloomFilter filter2 = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ Key key1 = new Key("key1".getBytes());
+ Key key2 = new Key("key2".getBytes());
+ filter1.add(key1);
+ filter2.add(key2);
+
+ filter1.or(filter2);
+
+ assertTrue(filter1.membershipTest(key1));
+ assertTrue(filter1.membershipTest(key2));
+ }
+
+ @Test
+ public void testXorThrowsForIncompatibleFilter() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ InternalDynamicBloomFilter differentNr = new
InternalDynamicBloomFilter(1000, 5, Hash.MURMUR_HASH, 20, 1000);
+ assertThrows(IllegalArgumentException.class, () ->
filter.xor(differentNr));
+ }
+
+ @Test
+ public void testXorWithSelfClearsAllBits() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ Key key = new Key("key1".getBytes());
+ filter.add(key);
+ assertTrue(filter.membershipTest(key));
+
+ filter.xor(filter);
+
+ assertFalse(filter.membershipTest(key));
+ }
+
+ @Test
+ public void testNotInvertsEmptyFilterToAlwaysContain() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ Key key = new Key("never-added".getBytes());
+ assertFalse(filter.membershipTest(key));
+
+ filter.not();
+
+ assertTrue(filter.membershipTest(key));
+ }
+
+ @Test
+ public void testAddRowsWithNonPositiveSizeIsNoOp() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ filter.addRows(0);
+ assertEquals(1, filter.getMatrixLength());
+ filter.addRows(-1);
+ assertEquals(1, filter.getMatrixLength());
+ }
+
+ @Test
+ public void testAddRowsWithMultipleRows() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ filter.addRows(3);
+ assertEquals(4, filter.getMatrixLength());
+ }
+
+ @Test
+ public void testToStringContainsOneEntryPerRow() {
+ InternalDynamicBloomFilter filter = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 10, 1000);
+ filter.addRows(2);
+ assertEquals(3, filter.getMatrixLength());
+
+ // Each row's underlying InternalBloomFilter renders its bit set as a
"{...}" block,
+ // so the number of opening braces reflects the number of rows toString()
iterated over.
+ String str = filter.toString();
+ long rowBlockCount = str.chars().filter(c -> c == '{').count();
+ assertEquals(3, rowBlockCount);
+ }
+
+ @Test
+ public void testWriteAndReadFieldsRoundTrip() throws IOException {
+ InternalDynamicBloomFilter original = new InternalDynamicBloomFilter(1000,
5, Hash.MURMUR_HASH, 2, 1000);
+ for (int i = 0; i < 10; i++) {
+ original.add(new Key(("key" + i).getBytes()));
+ }
+ assertTrue(original.getMatrixLength() > 1);
+
+ InternalDynamicBloomFilter deserialized = new InternalDynamicBloomFilter();
+
deserialized.readFields(BloomSerDeTestUtils.asDataInput(BloomSerDeTestUtils.serialize(original::write)));
+
+ assertEquals(original.getMatrixLength(), deserialized.getMatrixLength());
+ for (int i = 0; i < 10; i++) {
+ assertTrue(deserialized.membershipTest(new Key(("key" + i).getBytes())));
+ }
+ }
}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestKey.java
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestKey.java
new file mode 100644
index 000000000000..c438f6408ef0
--- /dev/null
+++ b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestKey.java
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.bloom;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.IOException;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Tests {@link Key}.
+ */
+public class TestKey {
+
+ @Test
+ public void testDefaultWeightIsOne() {
+ Key key = new Key("abc".getBytes());
+ assertEquals(1.0, key.getWeight());
+ }
+
+ @Test
+ public void testConstructorWithExplicitWeight() {
+ Key key = new Key("abc".getBytes(), 2.5);
+ assertEquals(2.5, key.getWeight());
+ assertArrayEquals("abc".getBytes(), key.getBytes());
+ }
+
+ @Test
+ public void testSetWithNullValueThrows() {
+ Key key = new Key();
+ assertThrows(IllegalArgumentException.class, () -> key.set(null, 1.0));
+ }
+
+ @Test
+ public void testIncrementWeightByAmount() {
+ Key key = new Key("abc".getBytes(), 1.0);
+ key.incrementWeight(2.0);
+ assertEquals(3.0, key.getWeight());
+ }
+
+ @Test
+ public void testIncrementWeightByOne() {
+ Key key = new Key("abc".getBytes(), 1.0);
+ key.incrementWeight();
+ assertEquals(2.0, key.getWeight());
+ }
+
+ @Test
+ public void testEqualsAndHashCodeForIdenticalKeys() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ Key key2 = new Key("abc".getBytes(), 1.0);
+ assertTrue(key1.equals(key2));
+ assertEquals(key1.hashCode(), key2.hashCode());
+ }
+
+ @Test
+ public void testEqualsIsFalseForDifferentBytes() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ Key key2 = new Key("xyz".getBytes(), 1.0);
+ assertFalse(key1.equals(key2));
+ }
+
+ @Test
+ public void testEqualsIsFalseForDifferentWeight() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ Key key2 = new Key("abc".getBytes(), 2.0);
+ assertFalse(key1.equals(key2));
+ }
+
+ @Test
+ public void testEqualsIsFalseForNonKeyObject() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ assertFalse(key1.equals("abc"));
+ }
+
+ @Test
+ public void testCompareToDifferentLength() {
+ Key shortKey = new Key("ab".getBytes());
+ Key longKey = new Key("abc".getBytes());
+ assertTrue(shortKey.compareTo(longKey) < 0);
+ assertTrue(longKey.compareTo(shortKey) > 0);
+ }
+
+ @Test
+ public void testCompareToSameLengthDifferentBytes() {
+ Key key1 = new Key("aac".getBytes());
+ Key key2 = new Key("abc".getBytes());
+ assertTrue(key1.compareTo(key2) < 0);
+ }
+
+ @Test
+ public void testCompareToSameBytesDifferentWeight() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ Key key2 = new Key("abc".getBytes(), 2.0);
+ assertTrue(key1.compareTo(key2) < 0);
+ assertTrue(key2.compareTo(key1) > 0);
+ }
+
+ @Test
+ public void testCompareToEqualKeys() {
+ Key key1 = new Key("abc".getBytes(), 1.0);
+ Key key2 = new Key("abc".getBytes(), 1.0);
+ assertEquals(0, key1.compareTo(key2));
+ }
+
+ @Test
+ public void testWriteAndReadFieldsRoundTrip() throws IOException {
+ Key original = new Key("hello-world".getBytes(), 3.5);
+
+ Key deserialized = new Key();
+
deserialized.readFields(BloomSerDeTestUtils.asDataInput(BloomSerDeTestUtils.serialize(original::write)));
+
+ assertArrayEquals(original.getBytes(), deserialized.getBytes());
+ assertEquals(original.getWeight(), deserialized.getWeight());
+ assertEquals(original, deserialized);
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestSimpleBloomFilter.java
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestSimpleBloomFilter.java
new file mode 100644
index 000000000000..834d7a35c250
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/common/bloom/TestSimpleBloomFilter.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.bloom;
+
+import org.apache.hudi.common.util.hash.Hash;
+
+import org.junit.jupiter.api.Test;
+
+import java.nio.ByteBuffer;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Tests the {@link SimpleBloomFilter} behaviour that the parameterized {@code
TestBloomFilter}
+ * suite does not already cover for {@link BloomFilterTypeCode#SIMPLE}:
negative lookups, null
+ * handling, the {@link java.nio.ByteBuffer} constructor and {@code or()} type
enforcement.
+ */
+public class TestSimpleBloomFilter {
+
+ @Test
+ public void testMightContainReturnsFalseForKeyNeverAdded() {
+ SimpleBloomFilter filter = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ filter.add("key1");
+ // With a low error rate and small key set, an unrelated key should not be
reported as present.
+ assertFalse(filter.mightContain("totally-different-key-xyz"));
+ }
+
+ @Test
+ public void testAddNullBytesThrows() {
+ SimpleBloomFilter filter = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ assertThrows(NullPointerException.class, () -> filter.add((byte[]) null));
+ }
+
+ @Test
+ public void testMightContainNullThrows() {
+ SimpleBloomFilter filter = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ assertThrows(NullPointerException.class, () -> filter.mightContain(null));
+ }
+
+ @Test
+ public void testByteBufferConstructorRoundTrip() {
+ SimpleBloomFilter filter = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ filter.add("key1");
+
+ String serialized = filter.serializeToString();
+ ByteBuffer byteBuffer = ByteBuffer.wrap(serialized.getBytes());
+ SimpleBloomFilter deserialized = new SimpleBloomFilter(byteBuffer);
+
+ assertTrue(deserialized.mightContain("key1"));
+ }
+
+ @Test
+ public void testGetBloomFilterTypeCode() {
+ SimpleBloomFilter filter = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ assertEquals(BloomFilterTypeCode.SIMPLE, filter.getBloomFilterTypeCode());
+ }
+
+ @Test
+ public void testOrWithNullFilterIsNoOp() {
+ SimpleBloomFilter filter1 = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ filter1.add("key1");
+ filter1.or(null);
+ assertTrue(filter1.mightContain("key1"));
+ }
+
+ @Test
+ public void testOrWithIncompatibleFilterTypeThrows() {
+ SimpleBloomFilter filter1 = new SimpleBloomFilter(1000, 0.000001,
Hash.MURMUR_HASH);
+ BloomFilter dynamicFilter = new HoodieDynamicBoundedBloomFilter(1000,
0.000001, Hash.MURMUR_HASH, 10000);
+
+ assertThrows(IllegalArgumentException.class, () ->
filter1.or(dynamicFilter));
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestBindVisitor.java
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestBindVisitor.java
index ca29eada7b8e..907fcc1e5e5e 100644
---
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestBindVisitor.java
+++
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestBindVisitor.java
@@ -20,34 +20,307 @@ package org.apache.hudi.common.expression;
import org.apache.hudi.common.schema.internal.Types;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collections;
-import static org.junit.jupiter.api.Assertions.assertFalse;
-import static org.junit.jupiter.api.Assertions.assertThrows;
-import static org.junit.jupiter.api.Assertions.assertTrue;
+/**
+ * Tests {@link BindVisitor}.
+ */
+public class TestBindVisitor {
-class TestBindVisitor {
+ private static Types.RecordType schema;
- private static Types.RecordType schema() {
- ArrayList<Types.Field> fields = new ArrayList<>(1);
+ @BeforeAll
+ public static void init() {
+ ArrayList<Types.Field> fields = new ArrayList<>(3);
fields.add(Types.Field.get(0, true, "a", Types.StringType.get()));
- return Types.RecordType.get(fields, "schema");
+ fields.add(Types.Field.get(1, true, "c", Types.IntType.get()));
+ fields.add(Types.Field.get(2, true, "d", Types.LongType.get()));
+ schema = Types.RecordType.get(fields, "schema");
+ }
+
+ @Test
+ public void testVisitNameReferenceBindsExistingField() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Expression bound = new NameReference("a").accept(bindVisitor);
+
+ Assertions.assertTrue(bound instanceof BoundReference);
+ Assertions.assertEquals(Types.StringType.get(), bound.getDataType());
+ }
+
+ @Test
+ public void testVisitNameReferenceThrowsForMissingField() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ NameReference unbound = new NameReference("unknown");
+
+ IllegalArgumentException e =
Assertions.assertThrows(IllegalArgumentException.class,
+ () -> unbound.accept(bindVisitor));
+ Assertions.assertTrue(e.getMessage().contains("cannot be bound from
schema"));
+ }
+
+ @Test
+ public void testVisitNameReferenceCaseSensitiveFailsOnDifferentCase() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ NameReference upperCase = new NameReference("A");
+
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
upperCase.accept(bindVisitor));
+ }
+
+ @Test
+ public void testVisitNameReferenceCaseInsensitiveBindsDifferentCase() {
+ BindVisitor bindVisitor = new BindVisitor(schema, false);
+ Expression bound = new NameReference("A").accept(bindVisitor);
+
+ Assertions.assertTrue(bound instanceof BoundReference);
+ }
+
+ @Test
+ public void testVisitLiteralReturnsSameInstance() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Literal<String> literal = Literal.from("value");
+
+ Assertions.assertSame(literal, literal.accept(bindVisitor));
+ }
+
+ @Test
+ public void testVisitBoundReferenceReturnsSameInstance() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ BoundReference boundReference = new BoundReference(0,
Types.StringType.get());
+
+ Assertions.assertSame(boundReference, boundReference.accept(bindVisitor));
+ }
+
+ @Test
+ public void testVisitAndBothOperandsBindAndEval() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.And and = Predicates.and(
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")),
+ Predicates.gt(new NameReference("c"), Literal.from(10)));
+ Expression bound = and.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 5, 5L))));
+ }
+
+ @Test
+ public void testVisitAndShortCircuitsWhenLeftIsFalseExpression() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.And and = Predicates.and(Predicates.alwaysFalse(),
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")));
+
+ Assertions.assertTrue(and.accept(bindVisitor) instanceof
Predicates.FalseExpression);
+ }
+
+ @Test
+ public void testVisitAndShortCircuitsWhenRightIsFalseExpression() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.And and = Predicates.and(
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")),
Predicates.alwaysFalse());
+
+ Assertions.assertTrue(and.accept(bindVisitor) instanceof
Predicates.FalseExpression);
+ }
+
+ @Test
+ public void testVisitAndSimplifiesWhenBothBoundOperandsAreAlwaysTrue() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.And and = Predicates.and(Predicates.alwaysTrue(),
Predicates.alwaysTrue());
+
+ Assertions.assertTrue(and.accept(bindVisitor) instanceof
Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testVisitAndSimplifiesWhenLeftBoundOperandIsAlwaysTrue() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison rightPredicate = Predicates.eq(new
NameReference("a"), Literal.from("Jane"));
+ Predicates.And and = Predicates.and(Predicates.alwaysTrue(),
rightPredicate);
+
+ Expression bound = and.accept(bindVisitor);
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitAndSimplifiesWhenRightBoundOperandIsAlwaysTrue() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison leftPredicate = Predicates.eq(new
NameReference("a"), Literal.from("Jane"));
+ Predicates.And and = Predicates.and(leftPredicate,
Predicates.alwaysTrue());
+
+ Expression bound = and.accept(bindVisitor);
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitOrBothOperandsBindAndEval() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Or or = Predicates.or(
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")),
+ Predicates.gt(new NameReference("c"), Literal.from(10)));
+ Expression bound = or.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 5, 5L))));
+ }
+
+ @Test
+ public void testVisitOrShortCircuitsWhenLeftIsTrueExpression() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Or or = Predicates.or(Predicates.alwaysTrue(),
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")));
+
+ Assertions.assertTrue(or.accept(bindVisitor) instanceof
Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testVisitOrShortCircuitsWhenRightIsTrueExpression() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Or or = Predicates.or(
+ Predicates.eq(new NameReference("a"), Literal.from("Jane")),
Predicates.alwaysTrue());
+
+ Assertions.assertTrue(or.accept(bindVisitor) instanceof
Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testVisitOrSimplifiesWhenBothBoundOperandsAreAlwaysFalse() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Or or = Predicates.or(Predicates.alwaysFalse(),
Predicates.alwaysFalse());
+
+ Assertions.assertTrue(or.accept(bindVisitor) instanceof
Predicates.FalseExpression);
+ }
+
+ @Test
+ public void testVisitOrSimplifiesWhenLeftBoundOperandIsAlwaysFalse() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison rightPredicate = Predicates.eq(new
NameReference("a"), Literal.from("Jane"));
+ Predicates.Or or = Predicates.or(Predicates.alwaysFalse(), rightPredicate);
+
+ Expression bound = or.accept(bindVisitor);
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitOrSimplifiesWhenRightBoundOperandIsAlwaysFalse() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison leftPredicate = Predicates.eq(new
NameReference("a"), Literal.from("Jane"));
+ Predicates.Or or = Predicates.or(leftPredicate, Predicates.alwaysFalse());
+
+ Expression bound = or.accept(bindVisitor);
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateNot() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Not not = Predicates.not(Predicates.eq(new NameReference("a"),
Literal.from("Jane")));
+ Expression bound = not.accept(bindVisitor);
+
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateNotOfAlwaysTrueChildIsAlwaysFalse() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Not not = Predicates.not(Predicates.alwaysTrue());
+
+ Assertions.assertTrue(not.accept(bindVisitor) instanceof
Predicates.FalseExpression);
+ }
+
+ @Test
+ public void testVisitPredicateNotOfAlwaysFalseChildIsAlwaysTrue() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.Not not = Predicates.not(Predicates.alwaysFalse());
+
+ Assertions.assertTrue(not.accept(bindVisitor) instanceof
Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testVisitPredicateBinaryComparison() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison eq = Predicates.eq(new NameReference("c"),
Literal.from(15));
+ Expression bound = eq.accept(bindVisitor);
+
+ Assertions.assertTrue(bound instanceof Predicates.BinaryComparison);
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateIn() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.In in = Predicates.in(new NameReference("c"),
+ Arrays.asList(Literal.from(10), Literal.from(15)));
+ Expression bound = in.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 99, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateIsNull() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.IsNull isNull = Predicates.isNull(new NameReference("a"));
+ Expression bound = isNull.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList(null, 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateIsNotNull() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.IsNotNull isNotNull = Predicates.isNotNull(new
NameReference("a"));
+ Expression bound = isNotNull.accept(bindVisitor);
+
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList(null, 15, 5L))));
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateStringStartsWith() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.StringStartsWith startsWith = Predicates.startsWith(new
NameReference("a"), Literal.from("Ja"));
+ Expression bound = startsWith.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateStringContains() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.StringContains contains = Predicates.contains(new
NameReference("a"), Literal.from("an"));
+ Expression bound = contains.accept(bindVisitor);
+
+ Assertions.assertTrue((Boolean) bound.eval(new
ArrayData(Arrays.asList("Jane", 15, 5L))));
+ Assertions.assertFalse((Boolean) bound.eval(new
ArrayData(Arrays.asList("Lone", 15, 5L))));
+ }
+
+ @Test
+ public void testVisitPredicateThrowsForMissingFieldInsideBinaryComparison() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.BinaryComparison eq = Predicates.eq(new
NameReference("unknown"), Literal.from("Jane"));
+
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
eq.accept(bindVisitor));
}
@Test
- void testUnsupportedPredicateErrorNamesTheExpression() {
- BindVisitor bindVisitor = new BindVisitor(schema(), true);
- Predicates.StringStartsWithAny startsWithAny =
- Predicates.startsWithAny(new NameReference("a"),
Collections.singletonList(Literal.from("Ja")));
+ public void testUnsupportedPredicateErrorNamesTheExpression() {
+ BindVisitor bindVisitor = new BindVisitor(schema, true);
+ Predicates.StringStartsWithAny startsWithAny =
Predicates.startsWithAny(new NameReference("a"),
+ Collections.singletonList(Literal.from("Ja")));
- IllegalArgumentException e =
- assertThrows(IllegalArgumentException.class, () ->
startsWithAny.accept(bindVisitor));
+ IllegalArgumentException e =
Assertions.assertThrows(IllegalArgumentException.class,
+ () -> startsWithAny.accept(bindVisitor));
-
assertTrue(e.getMessage().contains("NameReference(name=a).startsWithAny(Ja)"),
e::getMessage);
- assertFalse(e.getMessage().contains(BindVisitor.class.getName()),
e::getMessage);
- assertTrue(e.getMessage().contains(" cannot be visited as predicate"),
e::getMessage);
+
Assertions.assertTrue(e.getMessage().contains("NameReference(name=a).startsWithAny(Ja)"),
e::getMessage);
+
Assertions.assertFalse(e.getMessage().contains(BindVisitor.class.getName()),
e::getMessage);
+ Assertions.assertTrue(e.getMessage().contains(" cannot be visited as
predicate"), e::getMessage);
}
}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPartialBindVisitor.java
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPartialBindVisitor.java
index bffb2a6ca504..443375a7edd1 100644
---
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPartialBindVisitor.java
+++
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPartialBindVisitor.java
@@ -81,4 +81,160 @@ public class TestPartialBindVisitor {
Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Lone", "2023-04-02", 15, 5L, false))));
Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Lone", "2023-04-02", 10, 5L, false))));
}
+
+ @Test
+ public void testPartialBindNotWithAllFieldsPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.Not expr = Predicates.not(Predicates.eq(new NameReference("a"),
Literal.from("Jane")));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList("Lone", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindNotWithMissingFieldIsAlwaysFalse() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.Not expr = Predicates.not(Predicates.eq(new NameReference("m"),
Literal.from(5)));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.FalseExpression);
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindInWithAllValuesPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.In expr = Predicates.in(new NameReference("c"),
+ Arrays.asList(Literal.from(1), Literal.from(2)));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 1, 5L, false))));
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 99, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindInWithMissingValueExpressionIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.In expr = Predicates.in(new NameReference("m"),
+ Arrays.asList(Literal.from(1), Literal.from(2)));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindInWithMissingValidValueIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.In expr = Predicates.in(new NameReference("c"),
+ Arrays.asList(new NameReference("m")));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindIsNullWithFieldPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.IsNull expr = Predicates.isNull(new NameReference("a"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList(null, "2023-04-02", 15, 5L, false))));
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindIsNullWithMissingFieldIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.IsNull expr = Predicates.isNull(new NameReference("m"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindIsNotNullWithFieldPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.IsNotNull expr = Predicates.isNotNull(new NameReference("a"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList(null, "2023-04-02", 15, 5L, false))));
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindIsNotNullWithMissingFieldIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.IsNotNull expr = Predicates.isNotNull(new NameReference("m"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindStringStartsWithBothFieldsPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringStartsWith expr = Predicates.startsWith(new
NameReference("a"), Literal.from("Ja"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Lone", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindStringStartsWithMissingLeftIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringStartsWith expr = Predicates.startsWith(new
NameReference("m"), Literal.from("Ja"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindStringStartsWithMissingRightIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringStartsWith expr = Predicates.startsWith(new
NameReference("a"), new NameReference("m"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindStringContainsBothFieldsPresent() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringContains expr = Predicates.contains(new
NameReference("a"), Literal.from("an"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue((Boolean) binded.eval(new
ArrayData(Arrays.asList("Jane", "2023-04-02", 15, 5L, false))));
+ Assertions.assertFalse((Boolean) binded.eval(new
ArrayData(Arrays.asList("Lone", "2023-04-02", 15, 5L, false))));
+ }
+
+ @Test
+ public void testPartialBindStringContainsMissingLeftIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringContains expr = Predicates.contains(new
NameReference("m"), Literal.from("an"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindStringContainsMissingRightIsAlwaysTrue() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringContains expr = Predicates.contains(new
NameReference("a"), new NameReference("m"));
+ Expression binded = expr.accept(partialBindVisitor);
+
+ Assertions.assertTrue(binded instanceof Predicates.TrueExpression);
+ }
+
+ @Test
+ public void testPartialBindThrowsForUnsupportedPredicateType() {
+ PartialBindVisitor partialBindVisitor = new PartialBindVisitor(schema,
false);
+ Predicates.StringStartsWithAny expr = Predicates.startsWithAny(new
NameReference("a"),
+ Arrays.asList(Literal.from("Ja")));
+
+ IllegalArgumentException e =
Assertions.assertThrows(IllegalArgumentException.class,
+ () -> expr.accept(partialBindVisitor));
+ Assertions.assertTrue(e.getMessage().contains("cannot be visited as
predicate"));
+ }
}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPredicates.java
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPredicates.java
index 3c5b9f1c24c4..40ef988cacd6 100644
---
a/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPredicates.java
+++
b/hudi-common/src/test/java/org/apache/hudi/common/expression/TestPredicates.java
@@ -19,14 +19,18 @@
package org.apache.hudi.common.expression;
+import org.apache.hudi.common.schema.internal.Types;
+
import org.junit.jupiter.api.Test;
+import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
class TestPredicates {
@@ -73,4 +77,260 @@ class TestPredicates {
Predicates.startsWithAny(null,
Collections.singletonList(Literal.from("key1")));
assertEquals("null.startsWithAny(key1)", predicate.toString());
}
+
+ @Test
+ void testStringStartsWithAnyGetChildrenIncludesLeftAndAllRightValues() {
+ Expression left = Literal.from("key");
+ List<Expression> right = Arrays.asList(Literal.from("k1"),
Literal.from("k2"));
+ Predicates.StringStartsWithAny predicate = Predicates.startsWithAny(left,
right);
+
+ assertEquals(Arrays.asList(left, right.get(0), right.get(1)),
predicate.getChildren());
+ assertEquals(right, predicate.getRightChildren());
+ }
+
+ @Test
+ void testTrueExpressionEvalAndDispatch() {
+ Predicates.TrueExpression trueExpr = Predicates.alwaysTrue();
+ assertTrue(trueExpr.eval(null));
+ assertEquals(Expression.Operator.TRUE, trueExpr.getOperator());
+ assertEquals("TRUE", trueExpr.toString());
+ assertEquals("visitedAlwaysTrue", trueExpr.accept(new TaggingVisitor()));
+ }
+
+ @Test
+ void testFalseExpressionEvalAndDispatch() {
+ Predicates.FalseExpression falseExpr = Predicates.alwaysFalse();
+ assertFalse(falseExpr.eval(null));
+ assertEquals(Expression.Operator.FALSE, falseExpr.getOperator());
+ assertEquals("FALSE", falseExpr.toString());
+ assertEquals("visitedAlwaysFalse", falseExpr.accept(new TaggingVisitor()));
+ }
+
+ @Test
+ void testAndEvalBothTrue() {
+ assertTrue(Predicates.and(Literal.from(true),
Literal.from(true)).eval(null));
+ }
+
+ @Test
+ void testAndEvalLeftFalse() {
+ assertFalse(Predicates.and(Literal.from(false),
Literal.from(true)).eval(null));
+ }
+
+ @Test
+ void testAndEvalRightFalse() {
+ assertFalse(Predicates.and(Literal.from(true),
Literal.from(false)).eval(null));
+ }
+
+ @Test
+ void testAndEvalShortCircuitsWhenLeftIsFalseExpression() {
+ assertFalse(Predicates.and(Predicates.alwaysFalse(), new
NameReference("unbound")).eval(null));
+ }
+
+ @Test
+ void testAndEvalShortCircuitsWhenRightIsFalseExpression() {
+ assertFalse(Predicates.and(new NameReference("unbound"),
Predicates.alwaysFalse()).eval(null));
+ }
+
+ @Test
+ void testAndEvalWithNullLeftIsFalse() {
+ Literal<Boolean> nullBool = new Literal<>(null, Types.BooleanType.get());
+ assertFalse(Predicates.and(nullBool, Literal.from(true)).eval(null));
+ }
+
+ @Test
+ void testAndEvalWithNullRightIsFalse() {
+ Literal<Boolean> nullBool = new Literal<>(null, Types.BooleanType.get());
+ assertFalse(Predicates.and(Literal.from(true), nullBool).eval(null));
+ }
+
+ @Test
+ void testOrEvalShortCircuitsWhenLeftIsTrueExpression() {
+ assertTrue(Predicates.or(Predicates.alwaysTrue(), new
NameReference("unbound")).eval(null));
+ }
+
+ @Test
+ void testOrEvalShortCircuitsWhenRightIsTrueExpression() {
+ assertTrue(Predicates.or(new NameReference("unbound"),
Predicates.alwaysTrue()).eval(null));
+ }
+
+ @Test
+ void testOrEvalShortCircuitsWhenLeftIsTrueLiteral() {
+ // once left evaluates to true, right should never be evaluated
+ assertTrue(Predicates.or(Literal.from(true), new
NameReference("unbound")).eval(null));
+ }
+
+ @Test
+ void testOrEvalWithNullLeftIsFalseEvenIfRightIsTrue() {
+ Literal<Boolean> nullBool = new Literal<>(null, Types.BooleanType.get());
+ assertFalse(Predicates.or(nullBool, Literal.from(true)).eval(null));
+ }
+
+ @Test
+ void testOrEvalWithFalseLeftAndNullRightIsFalse() {
+ Literal<Boolean> nullBool = new Literal<>(null, Types.BooleanType.get());
+ assertFalse(Predicates.or(Literal.from(false), nullBool).eval(null));
+ }
+
+ @Test
+ void testOrEvalWithFalseLeftAndTrueRight() {
+ assertTrue(Predicates.or(Literal.from(false),
Literal.from(true)).eval(null));
+ }
+
+ @Test
+ void testNotEval() {
+ assertFalse(Predicates.not(Literal.from(true)).eval(null));
+ assertTrue(Predicates.not(Literal.from(false)).eval(null));
+ }
+
+ @Test
+ void testNotGetChildren() {
+ Expression child = Literal.from(true);
+ assertEquals(java.util.Collections.singletonList(child),
Predicates.not(child).getChildren());
+ }
+
+ @Test
+ void testIsNullEvalTrueForNullValue() {
+ Literal<String> nullValue = new Literal<>(null, Types.StringType.get());
+ assertTrue(Predicates.isNull(nullValue).eval(null));
+ }
+
+ @Test
+ void testIsNullEvalFalseForNonNullValue() {
+ assertFalse(Predicates.isNull(Literal.from("value")).eval(null));
+ }
+
+ @Test
+ void testIsNotNullEvalFalseForNullValue() {
+ Literal<String> nullValue = new Literal<>(null, Types.StringType.get());
+ assertFalse(Predicates.isNotNull(nullValue).eval(null));
+ }
+
+ @Test
+ void testIsNotNullEvalTrueForNonNullValue() {
+ assertTrue(Predicates.isNotNull(Literal.from("value")).eval(null));
+ }
+
+ @Test
+ void testInEvalTrueWhenValueMatches() {
+ Predicates.In in = Predicates.in(Literal.from("b"),
+ Arrays.asList(Literal.from("a"), Literal.from("b"),
Literal.from("c")));
+ assertTrue(in.eval(null));
+ }
+
+ @Test
+ void testInEvalFalseWhenValueDoesNotMatch() {
+ Predicates.In in = Predicates.in(Literal.from("z"),
+ Arrays.asList(Literal.from("a"), Literal.from("b"),
Literal.from("c")));
+ assertFalse(in.eval(null));
+ }
+
+ @Test
+ void testInGetChildrenIncludesValueAndValidValues() {
+ Expression value = Literal.from("b");
+ List<Expression> validValues = Arrays.asList(Literal.from("a"),
Literal.from("b"));
+ Predicates.In in = Predicates.in(value, validValues);
+
+ List<Expression> expected = new ArrayList<>();
+ expected.add(value);
+ expected.addAll(validValues);
+ assertEquals(expected, in.getChildren());
+ assertEquals(validValues, in.getRightChildren());
+ }
+
+ @Test
+ void testStringStartsWithEval() {
+ Predicates.StringStartsWith startsWith =
Predicates.startsWith(Literal.from("hoodie"), Literal.from("hoo"));
+ assertTrue((boolean) startsWith.eval(null));
+
+ Predicates.StringStartsWith noMatch =
Predicates.startsWith(Literal.from("hoodie"), Literal.from("bar"));
+ assertFalse((boolean) noMatch.eval(null));
+ }
+
+ @Test
+ void testStringContainsEval() {
+ Predicates.StringContains contains =
Predicates.contains(Literal.from("hoodie"), Literal.from("ood"));
+ assertTrue((boolean) contains.eval(null));
+
+ Predicates.StringContains noMatch =
Predicates.contains(Literal.from("hoodie"), Literal.from("xyz"));
+ assertFalse((boolean) noMatch.eval(null));
+ }
+
+ @Test
+ void testBinaryComparisonEvalForEachOperator() {
+ assertTrue(Predicates.eq(Literal.from(5), Literal.from(5)).eval(null));
+ assertTrue(Predicates.gt(Literal.from(5), Literal.from(3)).eval(null));
+ assertTrue(Predicates.gteq(Literal.from(5), Literal.from(5)).eval(null));
+ assertTrue(Predicates.lt(Literal.from(3), Literal.from(5)).eval(null));
+ assertTrue(Predicates.lteq(Literal.from(5), Literal.from(5)).eval(null));
+
+ assertFalse(Predicates.eq(Literal.from(5), Literal.from(6)).eval(null));
+ assertFalse(Predicates.gt(Literal.from(3), Literal.from(5)).eval(null));
+ assertFalse(Predicates.lt(Literal.from(5), Literal.from(3)).eval(null));
+ }
+
+ @Test
+ void testBinaryComparisonThrowsForNestedLeftType() {
+ ArrayList<Types.Field> fields = new ArrayList<>();
+ fields.add(Types.Field.get(0, true, "a", Types.StringType.get()));
+ Types.RecordType nestedType = Types.RecordType.get(fields, "nested");
+ BoundReference nestedRef = new BoundReference(0, nestedType);
+
+ Predicates.BinaryComparison comparison = Predicates.eq(nestedRef,
Literal.from("value"));
+ IllegalArgumentException e = assertThrows(IllegalArgumentException.class,
() -> comparison.eval(null));
+ assertTrue(e.getMessage().contains("nested type doesn't support binary
comparison"));
+ }
+
+ @Test
+ void testBinaryComparisonThrowsForUnsupportedOperator() {
+ Predicates.BinaryComparison comparison =
+ new Predicates.BinaryComparison(Literal.from(1),
Expression.Operator.AND, Literal.from(1));
+ IllegalArgumentException e = assertThrows(IllegalArgumentException.class,
() -> comparison.eval(null));
+ assertTrue(e.getMessage().contains("doesn't support binary comparison"));
+ }
+
+ /**
+ * Minimal visitor used to confirm that {@code accept} dispatches to the
expected
+ * {@link ExpressionVisitor} callback rather than some other overload.
+ */
+ private static class TaggingVisitor implements ExpressionVisitor<String> {
+ @Override
+ public String alwaysTrue() {
+ return "visitedAlwaysTrue";
+ }
+
+ @Override
+ public String alwaysFalse() {
+ return "visitedAlwaysFalse";
+ }
+
+ @Override
+ public String visitAnd(Predicates.And and) {
+ return "visitedAnd";
+ }
+
+ @Override
+ public String visitOr(Predicates.Or or) {
+ return "visitedOr";
+ }
+
+ @Override
+ public String visitLiteral(Literal literal) {
+ return "visitedLiteral";
+ }
+
+ @Override
+ public String visitNameReference(NameReference attribute) {
+ return "visitedNameReference";
+ }
+
+ @Override
+ public String visitBoundReference(BoundReference boundReference) {
+ return "visitedBoundReference";
+ }
+
+ @Override
+ public String visitPredicate(Predicate predicate) {
+ return "visitedPredicate";
+ }
+ }
}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/SecondaryIndexTestUtils.java
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/SecondaryIndexTestUtils.java
new file mode 100644
index 000000000000..d8d280116e42
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/SecondaryIndexTestUtils.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.core.index.secondary;
+
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+/**
+ * Shared fixtures for secondary index tests in this package.
+ */
+final class SecondaryIndexTestUtils {
+
+ private SecondaryIndexTestUtils() {
+ }
+
+ static LinkedHashMap<String, Map<String, String>> singleColumn(String
columnName) {
+ LinkedHashMap<String, Map<String, String>> columns = new LinkedHashMap<>();
+ columns.put(columnName, Collections.emptyMap());
+ return columns;
+ }
+
+ static HoodieSecondaryIndex newLuceneIndex(String indexName, String
columnName) {
+ return newLuceneIndex(indexName, columnName, Collections.emptyMap());
+ }
+
+ static HoodieSecondaryIndex newLuceneIndex(String indexName, String
columnName, Map<String, String> options) {
+ return HoodieSecondaryIndex.builder()
+ .setIndexName(indexName)
+ .setIndexType("lucene")
+ .setColumns(singleColumn(columnName))
+ .setOptions(options)
+ .build();
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestHoodieSecondaryIndex.java
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestHoodieSecondaryIndex.java
new file mode 100644
index 000000000000..9fdf92cb97aa
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestHoodieSecondaryIndex.java
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.core.index.secondary;
+
+import org.apache.hudi.exception.HoodieSecondaryIndexException;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Tests {@link HoodieSecondaryIndex}.
+ */
+public class TestHoodieSecondaryIndex {
+
+ @Test
+ public void testBuilderPopulatesAllFields() {
+ LinkedHashMap<String, Map<String, String>> columns = new LinkedHashMap<>();
+ columns.put("name", Collections.singletonMap("order", "1"));
+
+ HoodieSecondaryIndex index = HoodieSecondaryIndex.builder()
+ .setIndexName("idx_name")
+ .setIndexType("lucene")
+ .setColumns(columns)
+ .setOptions(Collections.singletonMap("k", "v"))
+ .build();
+
+ assertEquals("idx_name", index.getIndexName());
+ assertEquals(SecondaryIndexType.LUCENE, index.getIndexType());
+ assertEquals(columns, index.getColumns());
+ assertEquals(Collections.singletonMap("k", "v"), index.getOptions());
+ }
+
+ @Test
+ public void testBuilderIndexTypeIsCaseInsensitive() {
+ HoodieSecondaryIndex index = HoodieSecondaryIndex.builder()
+ .setIndexName("idx_name")
+ .setIndexType("LUCENE")
+ .setColumns(SecondaryIndexTestUtils.singleColumn("name"))
+ .setOptions(Collections.emptyMap())
+ .build();
+
+ assertEquals(SecondaryIndexType.LUCENE, index.getIndexType());
+ }
+
+ @Test
+ public void testLuceneIndexWithMultipleColumnsThrows() {
+ LinkedHashMap<String, Map<String, String>> columns = new LinkedHashMap<>();
+ columns.put("name", Collections.emptyMap());
+ columns.put("city", Collections.emptyMap());
+
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> new HoodieSecondaryIndex("idx_name", SecondaryIndexType.LUCENE,
columns, Collections.emptyMap()));
+ assertTrue(e.getMessage().contains("Lucene index only support single
column"));
+ }
+
+ @Test
+ public void testToStringContainsAllFields() {
+ HoodieSecondaryIndex index =
SecondaryIndexTestUtils.newLuceneIndex("idx_name", "name");
+
+ String str = index.toString();
+ assertTrue(str.contains("idx_name"));
+ assertTrue(str.contains("LUCENE"));
+ assertTrue(str.contains("name"));
+ }
+
+ @Test
+ public void testHoodieIndexCompactorSortsByIndexName() {
+ HoodieSecondaryIndex idxB =
SecondaryIndexTestUtils.newLuceneIndex("idx_b", "name");
+ HoodieSecondaryIndex idxA =
SecondaryIndexTestUtils.newLuceneIndex("idx_a", "name");
+
+ List<HoodieSecondaryIndex> sorted = Arrays.asList(idxB, idxA);
+ sorted.sort(new HoodieSecondaryIndex.HoodieIndexCompactor());
+
+ assertEquals("idx_a", sorted.get(0).getIndexName());
+ assertEquals("idx_b", sorted.get(1).getIndexName());
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexDefinitionUtils.java
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexDefinitionUtils.java
new file mode 100644
index 000000000000..c1e2130d7d9a
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexDefinitionUtils.java
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.core.index.secondary;
+
+import org.apache.hudi.common.table.HoodieTableConfig;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieSecondaryIndexException;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * Tests {@link SecondaryIndexDefinitionUtils}.
+ */
+public class TestSecondaryIndexDefinitionUtils {
+
+ private static HoodieSecondaryIndex newIndex(String name) {
+ return SecondaryIndexTestUtils.newLuceneIndex(name, "name",
Collections.singletonMap("k", "v"));
+ }
+
+ private static HoodieTableMetaClient
metaClientWithTableConfig(HoodieTableConfig tableConfig) {
+ HoodieTableMetaClient metaClient = mock(HoodieTableMetaClient.class);
+ when(metaClient.getTableConfig()).thenReturn(tableConfig);
+ return metaClient;
+ }
+
+ @Test
+ public void testToJsonStringAndFromJsonStringRoundTrip() {
+ List<HoodieSecondaryIndex> original = Arrays.asList(newIndex("idx_1"),
newIndex("idx_2"));
+ String json = SecondaryIndexDefinitionUtils.toJsonString(original);
+
+ List<HoodieSecondaryIndex> parsed =
SecondaryIndexDefinitionUtils.fromJsonString(json);
+ assertEquals(2, parsed.size());
+ assertEquals("idx_1", parsed.get(0).getIndexName());
+ assertEquals(SecondaryIndexType.LUCENE, parsed.get(0).getIndexType());
+ assertEquals(original.get(0).getColumns(), parsed.get(0).getColumns());
+ assertEquals(original.get(0).getOptions(), parsed.get(0).getOptions());
+ }
+
+ @Test
+ public void testFromJsonStringThrowsOnMalformedJson() {
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> SecondaryIndexDefinitionUtils.fromJsonString("not a valid
json"));
+ assertTrue(e.getMessage().contains("Fail to get secondary indexes"));
+ }
+
+ @Test
+ public void testFromJsonStringWithTypeReferenceReturnsNullForEmptyInput()
throws Exception {
+ assertNull(SecondaryIndexDefinitionUtils.fromJsonString(null, new
TypeReference<List<HoodieSecondaryIndex>>() { }));
+ assertNull(SecondaryIndexDefinitionUtils.fromJsonString("", new
TypeReference<List<HoodieSecondaryIndex>>() { }));
+ }
+
+ @Test
+ public void testObjectMapperIgnoresUnknownProperties() throws Exception {
+ String jsonWithUnknownField =
"[{\"indexName\":\"idx_1\",\"indexType\":\"LUCENE\","
+ +
"\"columns\":{\"name\":{}},\"options\":{},\"unknownField\":\"shouldBeIgnored\"}]";
+
+ List<HoodieSecondaryIndex> parsed =
SecondaryIndexDefinitionUtils.fromJsonString(jsonWithUnknownField);
+ assertEquals(1, parsed.size());
+ assertEquals("idx_1", parsed.get(0).getIndexName());
+ }
+
+ @Test
+ public void testGetSecondaryIndexesReturnsEmptyWhenNotSet() {
+ HoodieTableMetaClient metaClient = metaClientWithTableConfig(new
HoodieTableConfig());
+
assertFalse(SecondaryIndexDefinitionUtils.getSecondaryIndexes(metaClient).isPresent());
+ }
+
+ @Test
+ public void testGetSecondaryIndexesReturnsParsedListWhenSet() {
+ List<HoodieSecondaryIndex> original =
Collections.singletonList(newIndex("idx_1"));
+ HoodieTableConfig tableConfig = new HoodieTableConfig();
+ tableConfig.setValue(HoodieTableConfig.SECONDARY_INDEXES_METADATA,
SecondaryIndexDefinitionUtils.toJsonString(original));
+ HoodieTableMetaClient metaClient = metaClientWithTableConfig(tableConfig);
+
+ Option<List<HoodieSecondaryIndex>> indexes =
SecondaryIndexDefinitionUtils.getSecondaryIndexes(metaClient);
+ assertTrue(indexes.isPresent());
+ assertEquals("idx_1", indexes.get().get(0).getIndexName());
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexManager.java
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexManager.java
new file mode 100644
index 000000000000..175fcb9ba14b
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexManager.java
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.core.index.secondary;
+
+import org.apache.hudi.common.table.HoodieTableConfig;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieSecondaryIndexException;
+import org.apache.hudi.storage.StoragePath;
+
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertSame;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * Tests {@link SecondaryIndexManager}.
+ *
+ * <p>{@link HoodieTableMetaClient} is mocked since {@code hudi-common} does
not have access to a
+ * concrete, filesystem-backed {@link org.apache.hudi.storage.HoodieStorage}
implementation at test
+ * time. The static {@link HoodieTableConfig#update} / {@link
HoodieTableConfig#delete} calls are
+ * intercepted and applied directly to an in-memory {@link HoodieTableConfig},
faithfully mirroring
+ * what the real implementation persists to disk.
+ */
+public class TestSecondaryIndexManager {
+
+ private static final String TABLE_SCHEMA =
"{\"type\":\"record\",\"name\":\"trip\",\"fields\":["
+ + "{\"name\":\"id\",\"type\":\"string\"},"
+ + "{\"name\":\"name\",\"type\":\"string\"},"
+ + "{\"name\":\"city\",\"type\":\"string\"}]}";
+
+ private final SecondaryIndexManager manager =
SecondaryIndexManager.getInstance();
+
+ private HoodieTableConfig tableConfig;
+ private HoodieTableMetaClient metaClient;
+ private MockedStatic<HoodieTableConfig> tableConfigStatic;
+
+ @BeforeEach
+ public void setUp() {
+ tableConfig = new HoodieTableConfig();
+ tableConfig.setValue(HoodieTableConfig.CREATE_SCHEMA, TABLE_SCHEMA);
+
+ HoodieActiveTimeline activeTimeline = mock(HoodieActiveTimeline.class);
+
when(activeTimeline.getLastCommitMetadataWithValidSchema(false)).thenReturn(Option.empty());
+
+ metaClient = mock(HoodieTableMetaClient.class);
+ when(metaClient.getTableConfig()).thenReturn(tableConfig);
+ when(metaClient.getActiveTimeline()).thenReturn(activeTimeline);
+ when(metaClient.getMetaPath()).thenReturn(new
StoragePath("/tmp/dummy/.hoodie"));
+
+ tableConfigStatic = Mockito.mockStatic(HoodieTableConfig.class,
Mockito.CALLS_REAL_METHODS);
+ tableConfigStatic.when(() -> HoodieTableConfig.update(any(), any(), any()))
+ .thenAnswer(invocation -> {
+ tableConfig.setAll(invocation.getArgument(2));
+ return null;
+ });
+ tableConfigStatic.when(() -> HoodieTableConfig.delete(any(), any(), any()))
+ .thenAnswer(invocation -> {
+ Set<String> propsToDelete = invocation.getArgument(2);
+ propsToDelete.forEach(key ->
tableConfig.getProps(false).remove(key));
+ return null;
+ });
+ }
+
+ @AfterEach
+ public void tearDown() {
+ tableConfigStatic.close();
+ }
+
+ @Test
+ public void testGetInstanceReturnsSingleton() {
+ assertSame(SecondaryIndexManager.getInstance(),
SecondaryIndexManager.getInstance());
+ }
+
+ @Test
+ public void testCreateAddsSecondaryIndexMetadata() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("name");
+ manager.create(metaClient, "idx_name", "lucene", false, columns,
Collections.emptyMap());
+
+ Option<List<HoodieSecondaryIndex>> indexes = manager.show(metaClient);
+ assertTrue(indexes.isPresent());
+ assertEquals(1, indexes.get().size());
+ HoodieSecondaryIndex index = indexes.get().get(0);
+ assertEquals("idx_name", index.getIndexName());
+ assertEquals(SecondaryIndexType.LUCENE, index.getIndexType());
+ assertEquals(columns.keySet(), index.getColumns().keySet());
+ }
+
+ @Test
+ public void testCreateThrowsWhenColumnNotInSchema() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("unknown_col");
+
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> manager.create(metaClient, "idx_unknown", "lucene", false,
columns, Collections.emptyMap()));
+ assertTrue(e.getMessage().contains("Field not exists"));
+ }
+
+ @Test
+ public void testCreateThrowsWhenIndexNameAlreadyExists() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("name");
+ manager.create(metaClient, "idx_name", "lucene", false, columns,
Collections.emptyMap());
+
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> manager.create(metaClient, "idx_name", "lucene", false, columns,
Collections.emptyMap()));
+ assertTrue(e.getMessage().contains("already exists"));
+ }
+
+ @Test
+ public void testCreateIgnoresWhenIndexNameAlreadyExistsAndIgnoreFlagSet() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("name");
+ manager.create(metaClient, "idx_name", "lucene", false, columns,
Collections.emptyMap());
+
+ assertDoesNotThrow(() -> manager.create(metaClient, "idx_name", "lucene",
true, columns, Collections.emptyMap()));
+ assertEquals(1, manager.show(metaClient).get().size());
+ }
+
+ @Test
+ public void testCreateThrowsWhenSameTypeAndColumnsUnderDifferentName() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("name");
+ manager.create(metaClient, "idx_1", "lucene", false, columns,
Collections.emptyMap());
+
+ // Same index type and columns, but a different name: should be treated as
a duplicate.
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> manager.create(metaClient, "idx_2", "lucene", false, columns,
Collections.emptyMap()));
+ assertTrue(e.getMessage().contains("already exists"));
+ }
+
+ @Test
+ public void testCreateMultipleDistinctIndexesAreSortedByName() {
+ LinkedHashMap<String, Map<String, String>> nameCol =
SecondaryIndexTestUtils.singleColumn("name");
+ LinkedHashMap<String, Map<String, String>> cityCol =
SecondaryIndexTestUtils.singleColumn("city");
+
+ manager.create(metaClient, "idx_z", "lucene", false, nameCol,
Collections.emptyMap());
+ manager.create(metaClient, "idx_a", "lucene", false, cityCol,
Collections.emptyMap());
+
+ List<HoodieSecondaryIndex> indexes = manager.show(metaClient).get();
+ assertEquals(2, indexes.size());
+ assertEquals("idx_a", indexes.get(0).getIndexName());
+ assertEquals("idx_z", indexes.get(1).getIndexName());
+ }
+
+ @Test
+ public void testShowReturnsEmptyWhenNoSecondaryIndexes() {
+ assertFalse(manager.show(metaClient).isPresent());
+ }
+
+ @Test
+ public void testDropThrowsWhenIndexNotExists() {
+ HoodieSecondaryIndexException e =
assertThrows(HoodieSecondaryIndexException.class,
+ () -> manager.drop(metaClient, "idx_missing", false));
+ assertTrue(e.getMessage().contains("not exists"));
+ }
+
+ @Test
+ public void testDropIgnoresWhenIndexNotExistsAndIgnoreFlagSet() {
+ assertDoesNotThrow(() -> manager.drop(metaClient, "idx_missing", true));
+ }
+
+ @Test
+ public void
testDropRemovesSecondaryIndexMetadataEntirelyWhenLastIndexDropped() {
+ LinkedHashMap<String, Map<String, String>> columns =
SecondaryIndexTestUtils.singleColumn("name");
+ manager.create(metaClient, "idx_name", "lucene", false, columns,
Collections.emptyMap());
+
+ manager.drop(metaClient, "idx_name", false);
+
+ assertFalse(manager.show(metaClient).isPresent());
+ }
+
+ @Test
+ public void testDropKeepsRemainingIndexesWhenOtherIndexesExist() {
+ LinkedHashMap<String, Map<String, String>> nameCol =
SecondaryIndexTestUtils.singleColumn("name");
+ LinkedHashMap<String, Map<String, String>> cityCol =
SecondaryIndexTestUtils.singleColumn("city");
+
+ manager.create(metaClient, "idx_name", "lucene", false, nameCol,
Collections.emptyMap());
+ manager.create(metaClient, "idx_city", "lucene", false, cityCol,
Collections.emptyMap());
+
+ manager.drop(metaClient, "idx_name", false);
+
+ List<HoodieSecondaryIndex> remaining = manager.show(metaClient).get();
+ assertEquals(1, remaining.size());
+ assertEquals("idx_city", remaining.get(0).getIndexName());
+ }
+}
diff --git
a/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexType.java
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexType.java
new file mode 100644
index 000000000000..07ef437ea681
--- /dev/null
+++
b/hudi-common/src/test/java/org/apache/hudi/core/index/secondary/TestSecondaryIndexType.java
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.core.index.secondary;
+
+import org.apache.hudi.exception.HoodieIndexException;
+
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+/**
+ * Tests {@link SecondaryIndexType}.
+ */
+public class TestSecondaryIndexType {
+
+ @Test
+ public void testGetValueReturnsByteCode() {
+ assertEquals((byte) 1, SecondaryIndexType.LUCENE.getValue());
+ }
+
+ @Test
+ public void testOfByteReturnsMatchingType() {
+ assertEquals(SecondaryIndexType.LUCENE, SecondaryIndexType.of((byte) 1));
+ }
+
+ @Test
+ public void testOfByteThrowsForUnknownType() {
+ HoodieIndexException e = assertThrows(HoodieIndexException.class, () ->
SecondaryIndexType.of((byte) 99));
+ assertEquals("Unknown hoodie index type:99", e.getMessage());
+ }
+
+ @Test
+ public void testOfStringReturnsMatchingTypeCaseInsensitively() {
+ assertEquals(SecondaryIndexType.LUCENE, SecondaryIndexType.of("lucene"));
+ assertEquals(SecondaryIndexType.LUCENE, SecondaryIndexType.of("LUCENE"));
+ assertEquals(SecondaryIndexType.LUCENE, SecondaryIndexType.of("Lucene"));
+ }
+
+ @Test
+ public void testOfStringThrowsForUnknownType() {
+ HoodieIndexException e = assertThrows(HoodieIndexException.class, () ->
SecondaryIndexType.of("bloom"));
+ assertEquals("Unknown hoodie index type:bloom", e.getMessage());
+ }
+}
