[
https://issues.apache.org/jira/browse/PIG-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844427#comment-17844427
]
Rohini Palaniswamy commented on PIG-5439:
-----------------------------------------
+1
> Support Spark 3 and drop SparkShim
> ----------------------------------
>
> Key: PIG-5439
> URL: https://issues.apache.org/jira/browse/PIG-5439
> Project: Pig
> Issue Type: Improvement
> Components: spark
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Major
> Fix For: 0.19.0
>
> Attachments: pig-5439-v01.patch, pig-5439-v02.patch
>
>
> Support Pig-on-Spark to run on spark3.
> Initial version would only run up to Spark 3.2.4 and not on 3.3 or 3.4.
> This is due to log4j mismatch.
> After moving to log4j2 (PIG-5426), we can move Spark to 3.3 or higher.
> So far, not all unit/e2e tests pass with the proposed patch but at least
> compilation goes through.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
