This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 89d2f3675012 [SPARK-54395][SHUFFLE] RemoteBlockPushResolver class
repeatedly initializes ObjectMapper
89d2f3675012 is described below
commit 89d2f3675012a7aca781fdd5627b9f4da78fbf1d
Author: cuiyanxiang <[email protected]>
AuthorDate: Fri Nov 21 13:32:02 2025 -0800
[SPARK-54395][SHUFFLE] RemoteBlockPushResolver class repeatedly initializes
ObjectMapper
### What changes were proposed in this pull request?
RemoteBlockPushResolver class repeatedly initializes ObjectMapper
`ObjectMapper mapper = new ObjectMapper();`
### Why are the changes needed?
repeatedly initializes ObjectMapper
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
verified
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53110 from CuiYanxiang/SPARK-54395.
Authored-by: cuiyanxiang <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../org/apache/spark/network/shuffle/RemoteBlockPushResolver.java | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git
a/common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
b/common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
index a48208bad5b8..43f5e9c530e5 100644
---
a/common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
+++
b/common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
@@ -119,6 +119,9 @@ public class RemoteBlockPushResolver implements
MergedShuffleFileManager {
private static final ObjectMapper mapper = new ObjectMapper();
+ private static final TypeReference<Map<String, String>>
SHUFFLE_MANAGER_META_TYPE_REF =
+ new TypeReference<Map<String, String>>(){};
+
/**
* This a common prefix to the key for each app shuffle partition we add to
leveldb, so they
* are easy to find, since leveldb lets you search based on prefix.
@@ -881,10 +884,7 @@ public class RemoteBlockPushResolver implements
MergedShuffleFileManager {
String mergeDirInfo =
shuffleManagerMeta.substring(shuffleManagerMeta.indexOf(SHUFFLE_META_DELIMITER)
+ 1);
try {
- ObjectMapper mapper = new ObjectMapper();
- TypeReference<Map<String, String>> typeRef
- = new TypeReference<Map<String, String>>(){};
- Map<String, String> metaMap = mapper.readValue(mergeDirInfo, typeRef);
+ Map<String, String> metaMap = mapper.readValue(mergeDirInfo,
SHUFFLE_MANAGER_META_TYPE_REF);
String mergeDir = metaMap.get(MERGE_DIR_KEY);
int attemptId = Integer.valueOf(
metaMap.getOrDefault(ATTEMPT_ID_KEY,
String.valueOf(UNDEFINED_ATTEMPT_ID)));
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
