[jira] [Commented] (PIG-4941) TestRank3#testRankWithSplitInMap hangs after upgrade to spark 1.6.1

2016-07-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381662#comment-15381662 ] liyunzhang_intel commented on PIG-4941: --- [~xuefuz]:please commit PIG-4941.patch to bra

[jira] [Comment Edited] (PIG-4553) Implement secondary sort using one shuffle

2016-07-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381663#comment-15381663 ] liyunzhang_intel edited comment on PIG-4553 at 7/18/16 3:28 AM: --

[jira] [Commented] (PIG-4553) Implement secondary sort using one shuffle

2016-07-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381663#comment-15381663 ] liyunzhang_intel commented on PIG-4553: --- [~xuefuz]: please commit it to branch, thanks

[jira] [Resolved] (PIG-4797) Optimization for join/group case for spark mode

2016-07-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved PIG-4797. --- Resolution: Fixed > Optimization for join/group case for spark mode > --

[jira] [Created] (PIG-4952) Parallism is not set as correct value in some physical operator like GlobalRearrageParallism is not set as correct value in some physical operator like POGlobalRearrange

2016-07-19 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-4952: - Summary: Parallism is not set as correct value in some physical operator like GlobalRearrageParallism is not set as correct value in some physical operator like POGlobalRearrange Key: PIG-4952

[jira] [Commented] (PIG-4952) Parallism is not set as correct value in some physical operator like GlobalRearrageParallism is not set as correct value in some physical operator like POGlobalRearrange

2016-07-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391286#comment-15391286 ] liyunzhang_intel commented on PIG-4952: --- After investigating code, the value of parall

[jira] [Updated] (PIG-4952) Calculate the value of parallism for spark mode

2016-07-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4952: -- Summary: Calculate the value of parallism for spark mode (was: Parallism is not set as correct va

[jira] [Updated] (PIG-4952) Calculate the value of parallism for spark mode

2016-07-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4952: -- Description: Calculate the value of parallism for spark mode like what org.apache.pig.backend.hado

[jira] [Commented] (PIG-4937) Pigmix hangs when generating data after rows is set as 625000000 in test/perf/pigmix/conf/config.sh

2016-07-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393408#comment-15393408 ] liyunzhang_intel commented on PIG-4937: --- [~rohini] and [~daijy]: After generating all

[jira] [Resolved] (PIG-4236) Avoid packaging spark specific jars into pig fat jar

2016-08-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved PIG-4236. --- Resolution: Fixed > Avoid packaging spark specific jars into pig fat jar > -

[jira] [Created] (PIG-4965) Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test

2016-08-04 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-4965: - Summary: Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test Key: PIG-4965 URL: https://issues.apache.org/jira/br

[jira] [Commented] (PIG-4965) Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test

2016-08-04 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408890#comment-15408890 ] liyunzhang_intel commented on PIG-4965: --- [~daijy] and [~rohini]: As you are familiar

[jira] [Commented] (PIG-4965) Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test

2016-08-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411279#comment-15411279 ] liyunzhang_intel commented on PIG-4965: --- [~daijy]: In PIG-200, we really want to dele

[jira] [Commented] (PIG-4965) Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test

2016-08-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411435#comment-15411435 ] liyunzhang_intel commented on PIG-4965: --- [~daijy]: After investigating the code, i fo

[jira] [Updated] (PIG-4965) Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of single test case if we enable cleanup_after_test

2016-08-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4965: -- Attachment: PIG-4965.patch > Refactor test/perf/pigmix/bin/runpigmix.pl to delete the output of si

[jira] [Created] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-09 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-4968: - Summary: Fix unit test failure of TestLogicalPlanBuilder#testQuery90 Key: PIG-4968 URL: https://issues.apache.org/jira/browse/PIG-4968 Project: Pig Issue

[jira] [Updated] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4968: -- Attachment: PIG-4968.patch [~daijy] and [~knoguchi]: Can you help review this patch as you are f

[jira] [Updated] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4968: -- Attachment: PIG-4968.patch [~daijy] and [~knoguchi]: Can you help review this patch as you are f

[jira] [Updated] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4968: -- Attachment: (was: PIG-4968.patch) > Fix unit test failure of TestLogicalPlanBuilder#testQuery9

[jira] [Commented] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414574#comment-15414574 ] liyunzhang_intel commented on PIG-4968: --- [~knoguchi]: got it and hope all tests pass

[jira] [Created] (PIG-4969) Optimize combine case for spark mode

2016-08-10 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-4969: - Summary: Optimize combine case for spark mode Key: PIG-4969 URL: https://issues.apache.org/jira/browse/PIG-4969 Project: Pig Issue Type: Bug Comp

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-08-10 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Issue Type: Sub-task (was: Bug) Parent: PIG-4856 > Optimize combine case for spark mode >

[jira] [Updated] (PIG-4886) Add PigSplit#getLocationInfo to fix the NPE found in log in spark mode

2016-08-15 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4886: -- Resolution: Duplicate Status: Resolved (was: Patch Available) Duplicated with PIG-4941 >

[jira] [Commented] (PIG-4968) Fix unit test failure of TestLogicalPlanBuilder#testQuery90

2016-08-16 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423720#comment-15423720 ] liyunzhang_intel commented on PIG-4968: --- [~knoguchi]: Sorry reply late. Actually Tes

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Description: Now we use KryoSerializer to serialize the jobConf in [SparkLauncher|https://github.

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970.patch [~kexianda]: changes in PIG-4970.patch, please help review, thanks: 1

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Description: udf.pig {code} register '/home/zly/prj/oss/merge.pig/pig/bin/udf.js' using javascrip

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920.patch [~mohitsabharwal], please help review: UDFContext.getUDFContext() retur

[jira] [Commented] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432343#comment-15432343 ] liyunzhang_intel commented on PIG-4970: --- [~kexianda]: Yes, in previous code, we set "j

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_2.patch [~kexianda]: changes: 1. remove redudant imports > Remove the dese

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_3.patch [~kexianda]: changes in PIG-4970_3.patch, please help review: 1. rem

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_4.patch [~kexianda]: changes in PIG-4970_4.patch, please help review: 1. r

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_2.patch [~mohitsabharwal]: As PIG-4970 has been resolved. repatch after that.

[jira] [Comment Edited] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436277#comment-15436277 ] liyunzhang_intel edited comment on PIG-4920 at 8/25/16 4:51 AM: --

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_3.patch [~mohitsabharwal]: PIG-4920_3.patch is used after spark branch import

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436324#comment-15436324 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: As you are reviewing Pig on Spar

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-08-30 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969.patch > Optimize combine case for spark mode > --

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-31 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454188#comment-15454188 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: {quote} You should be doing MapRe

[jira] [Reopened] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-08-31 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reopened PIG-4295: --- After PIG-4866, we don't serialize PigContext in configuration to the backend. So we can not serial

[jira] [Updated] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-08-31 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4295: -- Attachment: PIG-4295_5.patch [~mohitsabharwal]: Changes in PIG-4295.patch: 1. serialize and deseri

[jira] [Comment Edited] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-08-31 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454514#comment-15454514 ] liyunzhang_intel edited comment on PIG-4295 at 9/1/16 6:56 AM: ---

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch [~mohitsabharwal]: Changes in PIG-4920_4.patch As we can not seriali

[jira] [Comment Edited] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454794#comment-15454794 ] liyunzhang_intel edited comment on PIG-4920 at 9/1/16 8:51 AM: ---

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode > -

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode > -

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode > -

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-04 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464268#comment-15464268 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: I have update PIG-4920_4.patch an

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: (was: PIG-4969.patch) > Optimize combine case for spark mode > ---

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_2.patch [~pallavi.rao]: as you are familiar with combine feature in the spar

[jira] [Comment Edited] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466133#comment-15466133 ] liyunzhang_intel edited comment on PIG-4969 at 9/6/16 1:52 AM: ---

[jira] [Commented] (PIG-4870) Enable MergeJoin testcase in TestCollectedGroup for spark engine

2016-09-06 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469357#comment-15469357 ] liyunzhang_intel commented on PIG-4870: --- [~kexianda]: +1, [~xuefuz]: please help check

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469918#comment-15469918 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: comments for PIG-5024.patch 1.

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472907#comment-15472907 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: LGTM except some code style prob

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475907#comment-15475907 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: PIG-5024_6.patch is LGTM, [~xuef

[jira] [Created] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5029: - Summary: Optimize sort case when data is skewed Key: PIG-5029 URL: https://issues.apache.org/jira/browse/PIG-5029 Project: Pig Issue Type: Sub-task

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: SkewedData_L9.docx SkewedData_L9.docx describes the skewed problem detaily and attachs

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Description: In PigMix L9.pig {code} register $PIGMIX_JAR A = load '$HDFS_ROOT/page_views' using

[jira] [Comment Edited] (PIG-4846) Use pigmix to test the performance of pig on spark

2016-09-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483372#comment-15483372 ] liyunzhang_intel edited comment on PIG-4846 at 9/12/16 7:36 AM: --

[jira] [Commented] (PIG-4846) Use pigmix to test the performance of pig on spark

2016-09-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483372#comment-15483372 ] liyunzhang_intel commented on PIG-4846: --- [~rohini],[~xuefuz], [~mohitsabharwal],[~pall

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: PIG-5029.patch > Optimize sort case when data is skewed >

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486676#comment-15486676 ] liyunzhang_intel commented on PIG-5029: --- The solution to solve the skewed data sort in

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489116#comment-15489116 ] liyunzhang_intel edited comment on PIG-5029 at 9/14/16 2:18 AM: --

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489116#comment-15489116 ] liyunzhang_intel commented on PIG-5029: --- [~rohini] and [~knoguchi]: Thanks for intere

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_3.patch [~pallavi.rao]: thanks for review, changes in PIG-4969_3.patch 1. cre

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489116#comment-15489116 ] liyunzhang_intel edited comment on PIG-5029 at 9/14/16 2:48 AM: --

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489215#comment-15489215 ] liyunzhang_intel commented on PIG-4969: --- [~pallavi.rao]: thanks for review. [~xuefuz]:

[jira] [Commented] (PIG-5025) Improve TestLoad.java: use own separated folder under /tmp

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489298#comment-15489298 ] liyunzhang_intel commented on PIG-5025: --- [~szita] and [~daijy]: I also found that Te

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15500022#comment-15500022 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: I will review in the next 2 days.

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15500402#comment-15500402 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: Thanks for your reply. Here is

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502764#comment-15502764 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: can you also file a review board to

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505256#comment-15505256 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: wondering if you get a chance to g

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505358#comment-15505358 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: if this has nothing to do with s

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505394#comment-15505394 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: {quote} If node goes down after

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505394#comment-15505394 ] liyunzhang_intel edited comment on PIG-5029 at 9/20/16 4:54 AM: --

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505394#comment-15505394 ] liyunzhang_intel edited comment on PIG-5029 at 9/20/16 5:39 AM: --

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515605#comment-15515605 ] liyunzhang_intel commented on PIG-5029: --- [~kexianda], [~mohitsabharwal] , [~pallavi.ra

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515605#comment-15515605 ] liyunzhang_intel edited comment on PIG-5029 at 9/23/16 6:54 AM: --

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515725#comment-15515725 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: Left some comments on review board

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15521883#comment-15521883 ] liyunzhang_intel commented on PIG-5029: --- [~vanzin]: Thanks for your comment, here i h

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524780#comment-15524780 ] liyunzhang_intel commented on PIG-5029: --- [~rohini] and [~knoguchi]: thanks for your pa

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524845#comment-15524845 ] liyunzhang_intel commented on PIG-5029: --- [~vanzin]: I guess what [~rohini] and [~knog

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524780#comment-15524780 ] liyunzhang_intel edited comment on PIG-5029 at 9/27/16 2:45 AM: --

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528532#comment-15528532 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: thanks for patience on this jira

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-28 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: PIG-5029_2.patch Changes in PIG-5029_2.patch: Force to set "spark.shuffle.service.enab

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556910#comment-15556910 ] liyunzhang_intel commented on PIG-4969: --- [~bertpassek]:thanks for reporting this jira,

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-10-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556913#comment-15556913 ] liyunzhang_intel commented on PIG-4815: --- [~daijy]: would you please help [~szita] to

[jira] [Resolved] (PIG-4904) Test spark branch with hadoop 1.x

2016-10-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved PIG-4904. --- Resolution: Won't Fix As Pig will not support hadoop1 anymore. Spark branch will not support had

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568030#comment-15568030 ] liyunzhang_intel commented on PIG-4969: --- [~bertpassek]: uploaded new patches PIG-4969_

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_3.patch > Optimize combine case for spark mode >

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570443#comment-15570443 ] liyunzhang_intel commented on PIG-4969: --- [~kexianda]: All unit tests pass after PIG-49

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: (was: PIG-4969_3.patch) > Optimize combine case for spark mode > -

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_3.patch > Optimize combine case for spark mode >

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: (was: PIG-4969_3.patch) > Optimize combine case for spark mode > -

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570555#comment-15570555 ] liyunzhang_intel commented on PIG-4969: --- [~kexianda]: the difference between PIG-4696

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: (was: PIG-4969_3.patch) > Optimize combine case for spark mode > -

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-10-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_3.patch > Optimize combine case for spark mode >

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573776#comment-15573776 ] liyunzhang_intel commented on PIG-4969: --- [~xuefuz]: PIG-4969_2.patch made the jenkins

<    1   2   3   4   5   6   7   8   9   10   >