[
https://issues.apache.org/jira/browse/PIG-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152753#comment-14152753
]
Praveen Rachabattuni commented on PIG-4173:
-------------------------------------------
Looks fine on my end.
Had to add three more excludes with Spark dependency to get it working.
<dependency org="org.apache.spark" name="spark-core_2.10"
rev="${spark.version}" conf="compile->default">
<exclude org="org.eclipse.jetty.orbit" module="javax.servlet"/>
<exclude org="org.eclipse.jetty.orbit" module="javax.transaction"/>
<exclude org="org.eclipse.jetty.orbit" module="javax.mail.glassfish"/>
<exclude org="org.eclipse.jetty.orbit" module="javax.activation"/>
</dependency>
Shall be making this change and merge it.
JavaRDD weren't used, observing some of the operations not synced from RDD
earlier. We could however use JavaRDD as needed, shouldn't think it should have
any performance issues.
> Move to Spark 1.x
> -----------------
>
> Key: PIG-4173
> URL: https://issues.apache.org/jira/browse/PIG-4173
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: bc Wong
> Assignee: Richard Ding
> Attachments: PIG-4173.patch, PIG-4173_2.patch
>
>
> The Spark branch is using Spark 0.9:
> https://github.com/apache/pig/blob/spark/ivy.xml#L438. We should probably
> switch to Spark 1.x asap, due to Spark interface changes since 1.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
