This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 7d73a2d [SPARK-57871] Exclude Hadoop MapReduce classes from shadow jar
7d73a2d is described below
commit 7d73a2d12fe59d42598f6a5446bc19655c607610
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jul 1 17:44:35 2026 -0700
[SPARK-57871] Exclude Hadoop MapReduce classes from shadow jar
### What changes were proposed in this pull request?
This PR excludes Hadoop MapReduce classes (`org/apache/hadoop/mapred/**` and
`org/apache/hadoop/mapreduce/**`) from the operator shadow jar by adding
class-path filters to the `shadowJar` block in
`spark-operator/build.gradle`.
### Why are the changes needed?
The operator manages Spark applications on Kubernetes and never runs the
MapReduce
framework. The bundled MapReduce classes are dead weight in the runtime
classpath
and shaded jar. Removing them drops 403 classes (~1.7 MB uncompressed) that
originate from `hadoop-client-api`, shrinking the shadow jar by ~774 KB.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #732 from dongjoon-hyun/SPARK-57871.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
spark-operator/build.gradle | 3 +++
1 file changed, 3 insertions(+)
diff --git a/spark-operator/build.gradle b/spark-operator/build.gradle
index 2c3861b..f0af025 100644
--- a/spark-operator/build.gradle
+++ b/spark-operator/build.gradle
@@ -101,6 +101,9 @@ shadowJar {
exclude("webapps/**")
// Hadoop YARN classes - not used for K8s deployment
exclude("org/apache/hadoop/yarn/**")
+ // Hadoop MapReduce classes - not used for K8s deployment
+ exclude("org/apache/hadoop/mapred/**")
+ exclude("org/apache/hadoop/mapreduce/**")
// Exclude 3rd party metadata to avoid conflicts
exclude("META-INF/maven/**")
// Netty QUIC native libraries - not needed for K8s API server communication
(HTTP/1.1 & HTTP/2)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
