[
https://issues.apache.org/jira/browse/PIG-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869452#comment-17869452
]
Koji Noguchi commented on PIG-5456:
-----------------------------------
In summary, changes to classloading is, for bin/pig and unit tests,
* MR/Tez jobs will stop using jars from spark directory.
* For Spark3, it would stop using reload4j (and orc-core after PIG-5457)
Former led to PIG-5458 where I noticed Pig on MR/Tez were relying on metrics
jar from Spark.
> Upgrade Spark to 3.4.3
> ----------------------
>
> Key: PIG-5456
> URL: https://issues.apache.org/jira/browse/PIG-5456
> Project: Pig
> Issue Type: Improvement
> Components: spark
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Major
> Fix For: 0.19.0
>
> Attachments: pig-5456-v01.patch, pig-5456-v02.patch
>
>
> Major blocker for upgrading to Spark 3.4.3 was Spark started using log4j2.
> Simple upgrade failing a lot of tests with
> {noformat}
> java.lang.VerifyError: class org.apache.log4j.bridge.LogEventAdapter
> overrides final method getTimeStamp.()J {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
