[spark] branch master updated: [SPARK-41458][BUILD][YARN][SHUFFLE] Correctly transform the SPI services for Yarn Shuffle Service

[srowen]

This is an automated email from the ASF dual-hosted git repository.

srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new be52d67fbe9 [SPARK-41458][BUILD][YARN][SHUFFLE] Correctly transform 
the SPI services for Yarn Shuffle Service
be52d67fbe9 is described below

commit be52d67fbe98110eeabf1b2a7c16741dceefdca6
Author: Cheng Pan <cheng...@apache.org>
AuthorDate: Fri Dec 9 08:14:17 2022 -0600

    [SPARK-41458][BUILD][YARN][SHUFFLE] Correctly transform the SPI services 
for Yarn Shuffle Service
    
    ### What changes were proposed in this pull request?
    
    Correctly transform the SPI services for Yarn Shuffle Service by 
configuring `ServicesResourceTransformer`.
    
    ### Why are the changes needed?
    
    SPARK-12807 relocated the Jackson classes, but did not handle SPI services 
properly.
    
    It affects Spark 2.0 and above, so this PR is for 3.2/3.3/master.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Manually checked the output jar.
    
    Before:
    
    <img width="772" alt="Xnip2022-12-09_12-49-31" 
src="https://user-images.githubusercontent.com/26535726/206632421-acbec562-c600-4497-83a3-f9b2f6caba74.png";>
    
    After:
    
    <img width="1209" alt="Xnip2022-12-09_12-53-39" 
src="https://user-images.githubusercontent.com/26535726/206632440-4c8ed745-dbc8-4b6e-a9e7-f285521aa8b4.png";>
    
    Closes #38989 from pan3793/SPARK-41458.
    
    Authored-by: Cheng Pan <cheng...@apache.org>
    Signed-off-by: Sean Owen <sro...@gmail.com>
---
 common/network-yarn/pom.xml | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/common/network-yarn/pom.xml b/common/network-yarn/pom.xml
index a4073969dbf..a77732bb8b8 100644
--- a/common/network-yarn/pom.xml
+++ b/common/network-yarn/pom.xml
@@ -136,6 +136,11 @@
             <goals>
               <goal>shade</goal>
             </goals>
+            <configuration>
+              <transformers>
+                <transformer 
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
+              </transformers>
+            </configuration>
           </execution>
         </executions>
       </plugin>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org