[jira] [Updated] (PIG-4797) Optimization for join/group case for spark mode

[ https://issues.apache.org/jira/browse/PIG-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4797: -- Summary: Optimization for join/group case for spark mode (was: JoinOptimization for spark mode)

[jira] [Updated] (PIG-4797) JoinOptimization for spark mode

[ https://issues.apache.org/jira/browse/PIG-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4797: -- Summary: JoinOptimization for spark mode (was: Analyze JOIN performance and improve the same.) >

[jira] [Updated] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4937: -- Attachment: pigmix2.PNG > Pigmix hangs when generating data after rows is set as 62500 in >

[jira] [Commented] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351312#comment-15351312 ] liyunzhang_intel commented on PIG-4937: --- [~rohini]: As there are 3 nodes( each has 56

[jira] [Updated] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4937: -- Attachment: pigmix1.PNG > Pigmix hangs when generating data after rows is set as 62500 in >

[jira] [Updated] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4937: -- Description: use the default setting in test/perf/pigmix/conf/config.sh, generate data by "ant -v

[jira] [Commented] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350514#comment-15350514 ] liyunzhang_intel commented on PIG-4937: --- [~rohini]: Can you help view the problem? Can

[jira] [Created] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

liyunzhang_intel created PIG-4937: - Summary: Pigmix hangs when generating data after rows is set as 62500 in test/perf/pigmix/conf/config.sh Key: PIG-4937 URL: https://issues.apache.org/jira/browse/PIG-4937

[jira] [Updated] (PIG-4936) Fix NPE exception in TestCustomPartitioner#testCustomPartitionerParseJoins

[ https://issues.apache.org/jira/browse/PIG-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4936: -- Attachment: PIG-4936.patch [~mohitsabharwal],[~kexianda] and [~pallavi.rao]: please help review. T

[jira] [Created] (PIG-4936) Fix NPE exception in TestCustomPartitioner#testCustomPartitionerParseJoins

liyunzhang_intel created PIG-4936: - Summary: Fix NPE exception in TestCustomPartitioner#testCustomPartitionerParseJoins Key: PIG-4936 URL: https://issues.apache.org/jira/browse/PIG-4936 Project: Pig

[jira] [Commented] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350452#comment-15350452 ] liyunzhang_intel commented on PIG-4927: --- [~xuefuz]: Please commit PIG-4927_1.patch to

[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4927: -- Status: Patch Available (was: Open) > Support stop.on.failure in spark mode > ---

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Status: Patch Available (was: Open) [~xuefuz]: Please merge it to the branch. > Avoid add all sp

[jira] [Commented] (PIG-4890) Run pigmix on spark on yarn with multiple nodes

[ https://issues.apache.org/jira/browse/PIG-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347615#comment-15347615 ] liyunzhang_intel commented on PIG-4890: --- Following is the result of pigmix between spa

[jira] [Commented] (PIG-4846) Use pigmix to test the performance of pig on spark

[ https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347613#comment-15347613 ] liyunzhang_intel commented on PIG-4846: --- [~pallavi.rao],[~mohitsabharwal]: Following i

[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4927: -- Attachment: PIG-4927_1.patch [~kexianda]: thanks for review and have submitted PIG-4927_1.patch fo

[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4927: -- Attachment: PIG-4927.patch > Support stop.on.failure in spark mode > -

[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4927: -- Attachment: (was: PIG-4927.patch) > Support stop.on.failure in spark mode > --

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903_5.patch [~sriksun]: Sorry for reply late. have submitted PIG-4903_5.patch acc

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Attachment: PIG-4893_2.patch [~pallavi.rao]: PIG-4893_2.patch is for latest comments on the review

[jira] [Commented] (PIG-4810) Implement Merge join for spark engine

[ https://issues.apache.org/jira/browse/PIG-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335526#comment-15335526 ] liyunzhang_intel commented on PIG-4810: --- [~kexianda]:LGTM +1 [~xuefuz]: Please merge

[jira] [Commented] (PIG-4871) Not use OperatorPlan#forceConnect in MultiQueryOptimizationSpark

[ https://issues.apache.org/jira/browse/PIG-4871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335514#comment-15335514 ] liyunzhang_intel commented on PIG-4871: --- [~xuefuz]: please merge PIG-4871_2.patch to b

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Attachment: PIG-4893_1.patch [~pallavi.rao]: upload latest patch and add review board for it, pl

[jira] [Commented] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334954#comment-15334954 ] liyunzhang_intel commented on PIG-4893: --- [~pallavi.rao]: before we append all jars und

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1595#comment-1595 ] liyunzhang_intel commented on PIG-4903: --- [~sriksun]: {quote} would this print a 0/1 on

[jira] [Commented] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1584#comment-1584 ] liyunzhang_intel commented on PIG-4927: --- [~pallavi.rao],[~mohitsabharwal] and [~kexian

[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

[ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4927: -- Attachment: PIG-4927.patch Before we skip TestGrunt#testStopOnFailure in spark mode {code} @Test

[jira] [Updated] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4926: -- Status: Patch Available (was: Open) > Modify the content of start.xml for spark mode > --

[jira] [Created] (PIG-4927) Support stop.on.failure in spark mode

liyunzhang_intel created PIG-4927: - Summary: Support stop.on.failure in spark mode Key: PIG-4927 URL: https://issues.apache.org/jira/browse/PIG-4927 Project: Pig Issue Type: Sub-task

[jira] [Comment Edited] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331232#comment-15331232 ] liyunzhang_intel edited comment on PIG-4926 at 6/15/16 6:34 AM: --

[jira] [Updated] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4926: -- Attachment: PIG-4926.patch [~rohini] and [~xuefu]: submit patch for this jira. While it is diff

[jira] [Comment Edited] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331165#comment-15331165 ] liyunzhang_intel edited comment on PIG-4903 at 6/15/16 5:36 AM: --

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903_4.patch [~sriksun] and [~rohini]: PIG-4903_4.patch includes the check of whe

[jira] [Commented] (PIG-4810) Implement Merge join for spark engine

[ https://issues.apache.org/jira/browse/PIG-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328915#comment-15328915 ] liyunzhang_intel commented on PIG-4810: --- [~kexianda]: Ok, after PIG-4856 is resolved,

[jira] [Commented] (PIG-4919) Upgrade spark.version to 1.6.1

[ https://issues.apache.org/jira/browse/PIG-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328874#comment-15328874 ] liyunzhang_intel commented on PIG-4919: --- [~xuefuz]: please merge it spark branch, than

[jira] [Updated] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4926: -- Attachment: pig.start.html.screenshot.png > Modify the content of start.xml for spark mode > -

[jira] [Updated] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4926: -- Attachment: (was: pig.start.html.screenshot.png) > Modify the content of start.xml for spark m

[jira] [Updated] (PIG-4926) Modify the content of start.xml for spark mode

[ https://issues.apache.org/jira/browse/PIG-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4926: -- Attachment: pig.start.html.screenshot.png [~rohini] and [~xuefuz]: Please help review the content

[jira] [Created] (PIG-4926) Modify the content of start.xml for spark mode

liyunzhang_intel created PIG-4926: - Summary: Modify the content of start.xml for spark mode Key: PIG-4926 URL: https://issues.apache.org/jira/browse/PIG-4926 Project: Pig Issue Type: Sub-task

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Parent Issue: PIG-4854 (was: PIG-4856) > Task deserialization time is too long for spark on yarn

[jira] [Updated] (PIG-4919) Upgrade spark.version to 1.6.1

[ https://issues.apache.org/jira/browse/PIG-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4919: -- Status: Patch Available (was: Open) > Upgrade spark.version to 1.6.1 > --

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903_3.patch [~sriksun] and [~rohini]: thanks for your comment. I have submitte

[jira] [Assigned] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned PIG-4893: - Assignee: liyunzhang_intel > Task deserialization time is too long for spark on yarn mode >

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Status: Patch Available (was: Open) > Task deserialization time is too long for spark on yarn mod

[jira] [Commented] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320250#comment-15320250 ] liyunzhang_intel commented on PIG-4893: --- [~sriksun] and [~pallavi.rao]: In current

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Attachment: PIG-4893.patch > Task deserialization time is too long for spark on yarn mode > --

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320244#comment-15320244 ] liyunzhang_intel commented on PIG-4903: --- [~sriksun]: Only 1 point is left to be discu

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903_2.patch > Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and

[jira] [Comment Edited] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318146#comment-15318146 ] liyunzhang_intel edited comment on PIG-4903 at 6/7/16 8:44 AM: ---

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318146#comment-15318146 ] liyunzhang_intel commented on PIG-4903: --- [~rohini] and [~sriksun]: sorry for reply lat

[jira] [Commented] (PIG-4923) Drop Hadoop 1.x support in Pig 0.17

[ https://issues.apache.org/jira/browse/PIG-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317681#comment-15317681 ] liyunzhang_intel commented on PIG-4923: --- [~daijy] and [~rohini]: Before [~rohini] sugg

[jira] [Created] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

liyunzhang_intel created PIG-4920: - Summary: Fail to use Javascript UDF in spark yarn client mode Key: PIG-4920 URL: https://issues.apache.org/jira/browse/PIG-4920 Project: Pig Issue Type: Su

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903_1.patch [~rohini]: I have submitted PIG-4903_1.patch. In this patch, it req

[jira] [Updated] (PIG-4919) Upgrade spark.version to 1.6.1

[ https://issues.apache.org/jira/browse/PIG-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4919: -- Attachment: PIG-4919.patch Changes in PIG-4919.patch 1. upgrade spark.version to 1.6.1 2. upgrade

[jira] [Created] (PIG-4919) Upgrade spark.version to 1.6.1

liyunzhang_intel created PIG-4919: - Summary: Upgrade spark.version to 1.6.1 Key: PIG-4919 URL: https://issues.apache.org/jira/browse/PIG-4919 Project: Pig Issue Type: Sub-task Rep

[jira] [Commented] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309940#comment-15309940 ] liyunzhang_intel commented on PIG-4893: --- Here summary the reason why task deserializat

[jira] [Commented] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

[ https://issues.apache.org/jira/browse/PIG-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309255#comment-15309255 ] liyunzhang_intel commented on PIG-4898: --- [~xuefuz]: please merge this patch to branch

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309248#comment-15309248 ] liyunzhang_intel commented on PIG-4903: --- [~sriksun]: Thanks for your comment! 1. for

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Attachment: PIG-4903.patch [~sriksun],[~xuefuz] ,[~mohitsabharwal] , [~pallavi.rao] and [~kexianda

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309149#comment-15309149 ] liyunzhang_intel commented on PIG-4903: --- [~sriksun]: After investigating the spark cod

[jira] [Comment Edited] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303337#comment-15303337 ] liyunzhang_intel edited comment on PIG-4903 at 5/27/16 2:08 AM: --

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303337#comment-15303337 ] liyunzhang_intel commented on PIG-4903: --- [~sriksun]: thanks for your reply, here is my

[jira] [Commented] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299670#comment-15299670 ] liyunzhang_intel commented on PIG-4903: --- [~xuefuz] and [~sriksun]: As before you wor

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Description: There are some comments about bin/pig on https://reviews.apache.org/r/45667/#comment

[jira] [Updated] (PIG-4903) Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH

[ https://issues.apache.org/jira/browse/PIG-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4903: -- Summary: Avoid add all spark dependency jars to SPARK_YARN_DIST_FILES and SPARK_DIST_CLASSPATH (

[jira] [Commented] (PIG-4667) Enable Pig on Spark to run on Yarn Client mode

[ https://issues.apache.org/jira/browse/PIG-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297618#comment-15297618 ] liyunzhang_intel commented on PIG-4667: --- [~sriksun]: Now community is reviewing the p

[jira] [Commented] (PIG-4904) Test spark branch with hadoop 1.x

[ https://issues.apache.org/jira/browse/PIG-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295943#comment-15295943 ] liyunzhang_intel commented on PIG-4904: --- [~rohini]: ok, i will fix all in one jira.

[jira] [Created] (PIG-4904) Test spark branch with hadoop 1.x

liyunzhang_intel created PIG-4904: - Summary: Test spark branch with hadoop 1.x Key: PIG-4904 URL: https://issues.apache.org/jira/browse/PIG-4904 Project: Pig Issue Type: Sub-task C

[jira] [Created] (PIG-4903) Modify bin/pig according to comments on the review board

liyunzhang_intel created PIG-4903: - Summary: Modify bin/pig according to comments on the review board Key: PIG-4903 URL: https://issues.apache.org/jira/browse/PIG-4903 Project: Pig Issue Type

[jira] [Updated] (PIG-4854) Merge spark branch to trunk

[ https://issues.apache.org/jira/browse/PIG-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4854: -- Issue Type: Bug (was: Sub-task) Parent: (was: PIG-4059) > Merge spark branch to trunk

[jira] [Comment Edited] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

[ https://issues.apache.org/jira/browse/PIG-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290271#comment-15290271 ] liyunzhang_intel edited comment on PIG-4898 at 5/19/16 2:03 AM: --

[jira] [Commented] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

[ https://issues.apache.org/jira/browse/PIG-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290271#comment-15290271 ] liyunzhang_intel commented on PIG-4898: --- [~mohitsabharwal],[~kexianda] and [~pallavi.r

[jira] [Updated] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

[ https://issues.apache.org/jira/browse/PIG-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4898: -- Attachment: PIG-4898.patch > Fix unit test failure after PIG-4771's patch was checked in > ---

[jira] [Created] (PIG-4899) The number of records of input file is calculated wrongly in spark mode in multiquery case

liyunzhang_intel created PIG-4899: - Summary: The number of records of input file is calculated wrongly in spark mode in multiquery case Key: PIG-4899 URL: https://issues.apache.org/jira/browse/PIG-4899

[jira] [Commented] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

[ https://issues.apache.org/jira/browse/PIG-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288630#comment-15288630 ] liyunzhang_intel commented on PIG-4898: --- the reason why following unit tests fail is 1

[jira] [Created] (PIG-4898) Fix unit test failure after PIG-4771's patch was checked in

liyunzhang_intel created PIG-4898: - Summary: Fix unit test failure after PIG-4771's patch was checked in Key: PIG-4898 URL: https://issues.apache.org/jira/browse/PIG-4898 Project: Pig Issue

[jira] [Commented] (PIG-4771) Implement FR Join for spark engine

[ https://issues.apache.org/jira/browse/PIG-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286007#comment-15286007 ] liyunzhang_intel commented on PIG-4771: --- [~xuefuz]: please commit it to branch, thank

[jira] [Updated] (PIG-4771) Implement FR Join for spark engine

[ https://issues.apache.org/jira/browse/PIG-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4771: -- Attachment: PIG-4771_3.patch [~mohitSabharwal]: changes in PIG-4771_3.patch 1. modify the replicat

[jira] [Commented] (PIG-4876) OutputConsumeIterator can't handle the last buffered tuples for some Operators

[ https://issues.apache.org/jira/browse/PIG-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285907#comment-15285907 ] liyunzhang_intel commented on PIG-4876: --- [~xuefuz]: please commit this patch by "patc

[jira] [Commented] (PIG-4854) Merge spark branch to trunk

[ https://issues.apache.org/jira/browse/PIG-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285825#comment-15285825 ] liyunzhang_intel commented on PIG-4854: --- [~rohini],[~xuefuz] and [~daijy]: Any commen

[jira] [Commented] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282540#comment-15282540 ] liyunzhang_intel commented on PIG-4893: --- I gave a simple case to descibe it: join.pig

[jira] [Commented] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282531#comment-15282531 ] liyunzhang_intel commented on PIG-4893: --- [~mohitsabharwal] and [~xuefuz]: Can you help

[jira] [Updated] (PIG-4883) MapKeyType of splitter was set wrongly in specific multiquery case

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4883: -- Attachment: PIG-4883_1.patch [~rohini]:Thanks for your review. Have submitted PIG-4883_1.patch for

[jira] [Updated] (PIG-4893) Task deserialization time is too long for spark on yarn mode

[ https://issues.apache.org/jira/browse/PIG-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4893: -- Attachment: time.PNG > Task deserialization time is too long for spark on yarn mode >

[jira] [Created] (PIG-4893) Task deserialization time is too long for spark on yarn mode

liyunzhang_intel created PIG-4893: - Summary: Task deserialization time is too long for spark on yarn mode Key: PIG-4893 URL: https://issues.apache.org/jira/browse/PIG-4893 Project: Pig Issue

[jira] [Commented] (PIG-4883) MapKeyType of splitter was set wrongly in specific multiquery case

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281096#comment-15281096 ] liyunzhang_intel commented on PIG-4883: --- [~rohini]: The problem is because for multiq

[jira] [Commented] (PIG-4771) Implement FR Join for spark engine

[ https://issues.apache.org/jira/browse/PIG-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279645#comment-15279645 ] liyunzhang_intel commented on PIG-4771: --- [~mohitsabharwal], [~pallavi.rao], [~kexianda

[jira] [Created] (PIG-4891) Implement FR join by broadcasting small rdd not making more copys of data

liyunzhang_intel created PIG-4891: - Summary: Implement FR join by broadcasting small rdd not making more copys of data Key: PIG-4891 URL: https://issues.apache.org/jira/browse/PIG-4891 Project: Pig

[jira] [Updated] (PIG-4890) Run pigmix on spark on yarn with multiple nodes

[ https://issues.apache.org/jira/browse/PIG-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4890: -- Issue Type: Sub-task (was: Bug) Parent: PIG-4856 > Run pigmix on spark on yarn with multi

[jira] [Updated] (PIG-4883) MapKeyType of splitter was set wrongly in specific multiquery case

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4883: -- Status: Patch Available (was: Open) > MapKeyType of splitter was set wrongly in specific multique

[jira] [Updated] (PIG-4886) Add PigSplit#getLocationInfo to fix the NPE found in log in spark mode

[ https://issues.apache.org/jira/browse/PIG-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4886: -- Status: Patch Available (was: Open) > Add PigSplit#getLocationInfo to fix the NPE found in log in

[jira] [Updated] (PIG-4883) MapKeyType of splitter was set wrongly in specific multiquery case

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4883: -- Summary: MapKeyType of splitter was set wrongly in specific multiquery case (was: java.lang.Clas

[jira] [Created] (PIG-4890) Run pigmix on spark on yarn with multiple nodes

liyunzhang_intel created PIG-4890: - Summary: Run pigmix on spark on yarn with multiple nodes Key: PIG-4890 URL: https://issues.apache.org/jira/browse/PIG-4890 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-4883) java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to java.lang.Long

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4883: -- Attachment: PIG-4883.patch in PIG-4883.patch: 1. add an attribute "mapKeyTypeForSplitter" for MapR

[jira] [Commented] (PIG-4883) java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to java.lang.Long

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277804#comment-15277804 ] liyunzhang_intel commented on PIG-4883: --- the problem in the above case is because i

[jira] [Assigned] (PIG-4883) java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to java.lang.Long

[ https://issues.apache.org/jira/browse/PIG-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reassigned PIG-4883: - Assignee: liyunzhang_intel > java.lang.ClassCastException: org.apache.pig.data.BinSedesTup

[jira] [Updated] (PIG-4553) Implement secondary sort using 1 shuffle not twice

[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4553: -- Attachment: PIG-4553_1.patch > Implement secondary sort using 1 shuffle not twice > --

[jira] [Updated] (PIG-4553) Implement secondary sort using 1 shuffle not twice

[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4553: -- Attachment: (was: PIG-4553_1.patch) > Implement secondary sort using 1 shuffle not twice > ---

[jira] [Updated] (PIG-4553) Implement secondary sort using 1 shuffle not twice

[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4553: -- Attachment: PIG-4553_1.patch PIG-4553_1.patch is based on PIG-4797. Once PIG-4797 is merged to bra

[jira] [Updated] (PIG-4553) Implement secondary sort using 1 shuffle not twice

[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4553: -- Attachment: (was: PIG-4553.patch) > Implement secondary sort using 1 shuffle not twice > -

<    2   3   4   5   6   7   8   9   10   11   >