This is an automated email from the ASF dual-hosted git repository.
LuciferYang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d9c08e5f159b [SPARK-57639][CORE] Fix NPE in `InMemoryStore.count` for
a type with no instances
d9c08e5f159b is described below
commit d9c08e5f159be8a8391250e83fdf5e01ed9ef3ca
Author: YangJie <[email protected]>
AuthorDate: Wed Jun 24 11:14:49 2026 +0800
[SPARK-57639][CORE] Fix NPE in `InMemoryStore.count` for a type with no
instances
### What changes were proposed in this pull request?
This PR fixes a `NullPointerException` in `InMemoryStore.count(Class,
String, Object)` and makes its behavior consistent with the other `KVStore`
implementations.
`inMemoryLists.get(type)` returns `null` for a type that has never been
written, and the method then dereferenced it via
`list.getIndexAccessor(index)`, throwing an NPE. The fix adds an early branch
for the never-written type that returns `0`. It still validates the index name
first (via a `KVTypeInfo` built from the type, mirroring how
`LevelDB`/`RocksDB` lazily build their type info), so an unknown index name
throws `IllegalArgumentException` exactly as it does for a populated type [...]
### Why are the changes needed?
It is a bug: counting by an indexed value for a type with no instances
should return `0`, not throw. The sibling one-arg `count(Class)` already
returns `0` in this situation, and both `LevelDB.count` and `RocksDB.count`
return `0` for an absent type, so `InMemoryStore` was inconsistent with its
siblings. The 3-arg `count` was also previously uncovered by
`InMemoryStoreSuite`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added `InMemoryStoreSuite.testCountByIndexValue`, which covers an absent
type returning `0`, an unknown index name throwing `IllegalArgumentException`
for both the absent-type and populated-type paths, and the match-counting cases
for a populated type. Ran `build/sbt 'kvstore/testOnly *InMemoryStoreSuite'`
and all tests pass.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #56701 from LuciferYang/SPARK-kvstore-inmemory-count-npe.
Authored-by: YangJie <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.../apache/spark/util/kvstore/InMemoryStore.java | 8 +++++
.../spark/util/kvstore/InMemoryStoreSuite.java | 35 ++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git
a/common/kvstore/src/main/java/org/apache/spark/util/kvstore/InMemoryStore.java
b/common/kvstore/src/main/java/org/apache/spark/util/kvstore/InMemoryStore.java
index 9a45a10532de..bb9c68bc8a96 100644
---
a/common/kvstore/src/main/java/org/apache/spark/util/kvstore/InMemoryStore.java
+++
b/common/kvstore/src/main/java/org/apache/spark/util/kvstore/InMemoryStore.java
@@ -65,6 +65,14 @@ public class InMemoryStore implements KVStore {
@Override
public long count(Class<?> type, String index, Object indexedValue) throws
Exception {
InstanceList<?> list = inMemoryLists.get(type);
+ if (list == null) {
+ // The type has never been written, so there are no matching entities.
Still validate
+ // the index name (via reflection on the type) before returning, so that
an unknown
+ // index fails the same way it does for a populated type and for the
LevelDB/RocksDB
+ // stores, instead of being silently swallowed.
+ new KVTypeInfo(type).getAccessor(index);
+ return 0;
+ }
int count = 0;
Object comparable = asKey(indexedValue);
KVTypeInfo.Accessor accessor = list.getIndexAccessor(index);
diff --git
a/common/kvstore/src/test/java/org/apache/spark/util/kvstore/InMemoryStoreSuite.java
b/common/kvstore/src/test/java/org/apache/spark/util/kvstore/InMemoryStoreSuite.java
index c7ae03f07829..ed4e58c974af 100644
---
a/common/kvstore/src/test/java/org/apache/spark/util/kvstore/InMemoryStoreSuite.java
+++
b/common/kvstore/src/test/java/org/apache/spark/util/kvstore/InMemoryStoreSuite.java
@@ -237,4 +237,39 @@ public class InMemoryStoreSuite {
store.delete(t4.getClass(), t4.key);
assertEquals(0, store.count(CustomType2.class));
}
+
+ @Test
+ public void testCountByIndexValue() throws Exception {
+ KVStore store = new InMemoryStore();
+
+ // Counting by an indexed value for a type that was never written must
return 0 instead
+ // of throwing, consistent with count(Class) and the LevelDB/RocksDB
implementations.
+ assertEquals(0, store.count(CustomType1.class, "id", "id"));
+
+ // An unknown index name must be rejected even when the type has no rows,
matching both
+ // the populated-type path and the LevelDB/RocksDB stores
(IllegalArgumentException),
+ // rather than being silently swallowed into a 0.
+ assertThrows(IllegalArgumentException.class,
+ () -> store.count(CustomType1.class, "nonexistentIndex", "id"));
+
+ CustomType1 t1 = new CustomType1();
+ t1.key = "key1";
+ t1.id = "id";
+ t1.name = "name1";
+ store.write(t1);
+
+ CustomType1 t2 = new CustomType1();
+ t2.key = "key2";
+ t2.id = "id";
+ t2.name = "name2";
+ store.write(t2);
+
+ assertEquals(2, store.count(CustomType1.class, "id", "id"));
+ assertEquals(1, store.count(CustomType1.class, "name", "name1"));
+ assertEquals(0, store.count(CustomType1.class, "name", "nonexistent"));
+
+ // Same rejection for a populated type, confirming the absent/present
paths agree.
+ assertThrows(IllegalArgumentException.class,
+ () -> store.count(CustomType1.class, "nonexistentIndex", "id"));
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
