[ https://issues.apache.org/jira/browse/PIG-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844427#comment-17844427 ]
Rohini Palaniswamy commented on PIG-5439: ----------------------------------------- +1 > Support Spark 3 and drop SparkShim > ---------------------------------- > > Key: PIG-5439 > URL: https://issues.apache.org/jira/browse/PIG-5439 > Project: Pig > Issue Type: Improvement > Components: spark > Reporter: Koji Noguchi > Assignee: Koji Noguchi > Priority: Major > Fix For: 0.19.0 > > Attachments: pig-5439-v01.patch, pig-5439-v02.patch > > > Support Pig-on-Spark to run on spark3. > Initial version would only run up to Spark 3.2.4 and not on 3.3 or 3.4. > This is due to log4j mismatch. > After moving to log4j2 (PIG-5426), we can move Spark to 3.3 or higher. > So far, not all unit/e2e tests pass with the proposed patch but at least > compilation goes through. -- This message was sent by Atlassian Jira (v8.20.10#820010)