[
https://issues.apache.org/jira/browse/PIG-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140146#comment-16140146
]
Nandor Kollar commented on PIG-5080:
------------------------------------
[~jeffzhang] it seems your patch requires some adjustments: in Spark 2.x
DataFrames are replaced with Dataset<Row> in StoreConverter, and the ivy.xml
also requires modifications, instead of single spark configuration, we now have
two different config for spark1 and spark2.
> Support store alias as spark table
> ----------------------------------
>
> Key: PIG-5080
> URL: https://issues.apache.org/jira/browse/PIG-5080
> Project: Pig
> Issue Type: New Feature
> Components: spark
> Affects Versions: 0.16.0
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Fix For: 0.17.1
>
> Attachments: PIG-5080-1.patch, PIG-5080-2.patch
>
>
> The purpose is that I'd like to take advantage of both pig and hive.
> Pig-latin has powerful data flow expression ability which is useful for ETL
> while hive is good at query.
> The scenario is that I'd like to store pig alias as spark temporary table
> (cache can be optional). And I have an another spark engine which share the
> same SparkContext (in the same JVM) to query the table.
> Please close this ticket if it is already supported. I didn't go through all
> the features of pig-spark.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
