[
https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170463#comment-14170463
]
Rohini Palaniswamy commented on PIG-4168:
-----------------------------------------
Sorry Liyun about being late to comment on this one.
Few comments:
- Why do you need a LowercaseUDF udf? There is already a LOWER builtin udf.
- Shouldn't the getExecType() in SparkMiniCluster return spark ?
- As far TestSpark.java - instead of creating new unit tests for Spark, try
getting the existing unit tests to work with Spark unless there is something
Spark specific like CACHE. For reference look at tez-tests. I did already
mention this in the last week's call when Praveen asked how to go about unit
tests, but you were not in that call.
> Initial implementation of unit tests for Pig on Spark
> -----------------------------------------------------
>
> Key: PIG-4168
> URL: https://issues.apache.org/jira/browse/PIG-4168
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Praveen Rachabattuni
> Assignee: liyunzhang_intel
> Attachments: PIG-4168.patch, PIG-4168_1.patch, PIG-4168_2.patch,
> PIG-4168_3.patch
>
>
> 1.ant clean jar; pig-0.14.0-SNAPSHOT-core-h1.jar will be generated by the
> command
> 2.export SPARK_PIG_JAR=$PIG_HOME/pig-0.14.0-SNAPSHOT-core-h1.jar
> 3.build hadoop1 and spark env.spark run in local mode
> jps:
> 11647 Master #spark master runs
> 6457 DataNode #hadoop datanode runs
> 22374 Jps
> 11705 Worker# spark worker runs
> 27009 JobTracker #hadoop jobtracker runs
> 26602 NameNode #hadoop namenode runs
> 29486 org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar
> 19692 Main
>
> 4.ant test-spark
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
