[jira] [Commented] (PIG-5145) fix TestLineageFindRelVisitor unit test failure after PIG-5132

2017-02-28 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887546#comment-15887546 ] liyunzhang_intel commented on PIG-5145: --- [~szita]: it is better to change the code like {code}

[jira] [Created] (PIG-5174) TestGrunt#testGruntUtf8 fails

2017-03-01 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5174: - Summary: TestGrunt#testGruntUtf8 fails Key: PIG-5174 URL: https://issues.apache.org/jira/browse/PIG-5174 Project: Pig Issue Type: Sub-task

[jira] [Created] (PIG-5161) Include all unit tests for spark branch

2017-02-28 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5161: - Summary: Include all unit tests for spark branch Key: PIG-5161 URL: https://issues.apache.org/jira/browse/PIG-5161 Project: Pig Issue Type: Bug

[jira] [Commented] (PIG-5157) Upgrade to Spark 2.0

2017-02-28 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889267#comment-15889267 ] liyunzhang_intel commented on PIG-5157: --- [~zjffdu]: thanks your suggestion. > Upgrade to Spark 2.0 >

[jira] [Commented] (PIG-5174) TestGrunt#testGruntUtf8 fails

2017-03-02 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891813#comment-15891813 ] liyunzhang_intel commented on PIG-5174: --- [~daijy]: thanks, I have created a new round of test on

[jira] [Updated] (PIG-5133) Commit changes from last round of review on rb

2017-03-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5133: -- Attachment: PIG-5133.patch [~szita] and [~nkollar]: merge changes from last review proposed by

[jira] [Commented] (PIG-5132) Merge from trunk (5) [Spark Branch]

2017-02-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886594#comment-15886594 ] liyunzhang_intel commented on PIG-5132: --- [~szita]: [~rohini] suggested to remove all these jars in

[jira] [Commented] (PIG-5157) Upgrade to Spark 2.0

2017-02-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886570#comment-15886570 ] liyunzhang_intel commented on PIG-5157: --- [~nkollar]: it is sensible to upgrade to latest spark

[jira] [Updated] (PIG-5132) Merge from trunk (5) [Spark Branch]

2017-02-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5132: -- Attachment: buid.xml.from.rb.PNG > Merge from trunk (5) [Spark Branch] >

[jira] [Commented] (PIG-5132) Merge from trunk (5) [Spark Branch]

2017-02-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885381#comment-15885381 ] liyunzhang_intel commented on PIG-5132: --- [~szita]: in PIG-5132.aftermerge.0.patch. most accept except

[jira] [Commented] (PIG-5132) Merge from trunk (5) [Spark Branch]

2017-03-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889733#comment-15889733 ] liyunzhang_intel commented on PIG-5132: --- [~szita]: commit PIG-5132-aftermerge.1.patch, please verify

[jira] [Commented] (PIG-5155) Fix FRJoin related issues after trunk merge

2017-03-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889754#comment-15889754 ] liyunzhang_intel commented on PIG-5155: --- [~nkollar]: commit to spark branch, thanks! > Fix FRJoin

[jira] [Commented] (PIG-5140) fix TestEmptyInputDir unit test failure after PIG-5132

2017-03-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889769#comment-15889769 ] liyunzhang_intel commented on PIG-5140: --- [~szita]: commit to spark branch, thanks > fix

[jira] [Commented] (PIG-5145) fix TestLineageFindRelVisitor unit test failure after PIG-5132

2017-03-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889776#comment-15889776 ] liyunzhang_intel commented on PIG-5145: --- [~szita]: commit to spark branch, thanks > fix

[jira] [Comment Edited] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436277#comment-15436277 ] liyunzhang_intel edited comment on PIG-4920 at 8/25/16 4:51 AM:

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_3.patch [~mohitsabharwal]: PIG-4920_3.patch is used after spark branch

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_2.patch [~mohitsabharwal]: As PIG-4970 has been resolved. repatch after

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436324#comment-15436324 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: As you are reviewing Pig on Spark, here say more

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-31 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454188#comment-15454188 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: {quote} You should be doing

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_2.patch [~pallavi.rao]: as you are familiar with combine feature in the

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: (was: PIG-4969.patch) > Optimize combine case for spark mode >

[jira] [Comment Edited] (PIG-4969) Optimize combine case for spark mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466133#comment-15466133 ] liyunzhang_intel edited comment on PIG-4969 at 9/6/16 1:52 AM: ---

[jira] [Commented] (PIG-4870) Enable MergeJoin testcase in TestCollectedGroup for spark engine

2016-09-06 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469357#comment-15469357 ] liyunzhang_intel commented on PIG-4870: --- [~kexianda]: +1, [~xuefuz]: please help checkin this patch >

[jira] [Comment Edited] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454514#comment-15454514 ] liyunzhang_intel edited comment on PIG-4295 at 9/1/16 6:56 AM: ---

[jira] [Comment Edited] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454794#comment-15454794 ] liyunzhang_intel edited comment on PIG-4920 at 9/1/16 8:51 AM: ---

[jira] [Updated] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4295: -- Attachment: PIG-4295_5.patch [~mohitsabharwal]: Changes in PIG-4295.patch: 1. serialize and

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch [~mohitsabharwal]: Changes in PIG-4920_4.patch As we can not

[jira] [Reopened] (PIG-4295) Enable unit test "TestPigContext" for spark

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel reopened PIG-4295: --- After PIG-4866, we don't serialize PigContext in configuration to the backend. So we can not

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_4.patch > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-01 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_4.patch) > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469918#comment-15469918 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: comments for PIG-5024.patch 1. rename

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472907#comment-15472907 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: LGTM except some code style problem. > add a

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-05 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464268#comment-15464268 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: I have update PIG-4920_4.patch and not serialize

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-08-30 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969.patch > Optimize combine case for spark mode >

[jira] [Created] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5029: - Summary: Optimize sort case when data is skewed Key: PIG-5029 URL: https://issues.apache.org/jira/browse/PIG-5029 Project: Pig Issue Type: Sub-task

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: SkewedData_L9.docx SkewedData_L9.docx describes the skewed problem detaily and

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Description: In PigMix L9.pig {code} register $PIGMIX_JAR A = load '$HDFS_ROOT/page_views' using

[jira] [Commented] (PIG-4846) Use pigmix to test the performance of pig on spark

2016-09-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483372#comment-15483372 ] liyunzhang_intel commented on PIG-4846: --- [~rohini],[~xuefuz], [~mohitsabharwal],[~pallavi.rao]: Here

[jira] [Comment Edited] (PIG-4846) Use pigmix to test the performance of pig on spark

2016-09-12 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483372#comment-15483372 ] liyunzhang_intel edited comment on PIG-4846 at 9/12/16 7:36 AM:

[jira] [Commented] (PIG-5024) add a physical operator to broadcast small RDDs

2016-09-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475907#comment-15475907 ] liyunzhang_intel commented on PIG-5024: --- [~kexianda]: PIG-5024_6.patch is LGTM, [~xuefuz]: please help

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489116#comment-15489116 ] liyunzhang_intel commented on PIG-5029: --- [~rohini] and [~knoguchi]: Thanks for interest in this

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489116#comment-15489116 ] liyunzhang_intel edited comment on PIG-5029 at 9/14/16 2:18 AM: [~rohini]

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489215#comment-15489215 ] liyunzhang_intel commented on PIG-4969: --- [~pallavi.rao]: thanks for review. [~xuefuz]: please check in

[jira] [Updated] (PIG-4969) Optimize combine case for spark mode

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4969: -- Attachment: PIG-4969_3.patch [~pallavi.rao]: thanks for review, changes in PIG-4969_3.patch 1.

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489116#comment-15489116 ] liyunzhang_intel edited comment on PIG-5029 at 9/14/16 2:48 AM: [~rohini]

[jira] [Commented] (PIG-5025) Improve TestLoad.java: use own separated folder under /tmp

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489298#comment-15489298 ] liyunzhang_intel commented on PIG-5025: --- [~szita] and [~daijy]: I also found that

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486676#comment-15486676 ] liyunzhang_intel commented on PIG-5029: --- The solution to solve the skewed data sort in PIG-5029.patch

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-13 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: PIG-5029.patch > Optimize sort case when data is skewed >

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521883#comment-15521883 ] liyunzhang_intel commented on PIG-5029: --- [~vanzin]: Thanks for your comment, here i have a question

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524780#comment-15524780 ] liyunzhang_intel commented on PIG-5029: --- [~rohini] and [~knoguchi]: thanks for your patience on this

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556910#comment-15556910 ] liyunzhang_intel commented on PIG-4969: --- [~bertpassek]:thanks for reporting this jira, will fix in

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-10-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556913#comment-15556913 ] liyunzhang_intel commented on PIG-4815: --- [~daijy]: would you please help [~szita] to review the

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15500022#comment-15500022 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: I will review in the next 2 days. > Add xml format

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15500402#comment-15500402 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: Thanks for your reply. Here is a question about

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505256#comment-15505256 ] liyunzhang_intel commented on PIG-4920: --- [~rohini]: wondering if you get a chance to give us some

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505394#comment-15505394 ] liyunzhang_intel edited comment on PIG-5029 at 9/20/16 5:39 AM: [~knoguchi]:

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505394#comment-15505394 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: {quote} If node goes down after

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505394#comment-15505394 ] liyunzhang_intel edited comment on PIG-5029 at 9/20/16 4:54 AM: [~knoguchi]:

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-19 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502764#comment-15502764 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: can you also file a review board to help review?

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515605#comment-15515605 ] liyunzhang_intel edited comment on PIG-5029 at 9/23/16 6:54 AM: [~kexianda],

[jira] [Commented] (PIG-4815) Add xml format support for 'explain' in spark engine

2016-09-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515725#comment-15515725 ] liyunzhang_intel commented on PIG-4815: --- [~szita]: Left some comments on review board. LGTM for

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515605#comment-15515605 ] liyunzhang_intel commented on PIG-5029: --- [~kexianda], [~mohitsabharwal] , [~pallavi.rao] and

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970.patch [~kexianda]: changes in PIG-4970.patch, please help review, thanks:

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Description: Now we use KryoSerializer to serialize the jobConf in

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Description: udf.pig {code} register '/home/zly/prj/oss/merge.pig/pig/bin/udf.js' using

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-08-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920.patch [~mohitsabharwal], please help review: UDFContext.getUDFContext()

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_3.patch [~kexianda]: changes in PIG-4970_3.patch, please help review: 1.

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_4.patch [~kexianda]: changes in PIG-4970_4.patch, please help review: 1.

[jira] [Commented] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432343#comment-15432343 ] liyunzhang_intel commented on PIG-4970: --- [~kexianda]: Yes, in previous code, we set

[jira] [Updated] (PIG-4970) Remove the deserialize and serialization of JobConf in code for spark mode

2016-08-23 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4970: -- Attachment: PIG-4970_2.patch [~kexianda]: changes: 1. remove redudant imports > Remove the

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-28 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528532#comment-15528532 ] liyunzhang_intel commented on PIG-5029: --- [~knoguchi]: thanks for patience on this jira. I will enable

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-09-28 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: PIG-5029_2.patch Changes in PIG-5029_2.patch: Force to set

[jira] [Comment Edited] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524780#comment-15524780 ] liyunzhang_intel edited comment on PIG-5029 at 9/27/16 2:45 AM: [~rohini]

[jira] [Commented] (PIG-5029) Optimize sort case when data is skewed

2016-09-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524845#comment-15524845 ] liyunzhang_intel commented on PIG-5029: --- [~vanzin]: I guess what [~rohini] and [~knoguchi] worries is

[jira] [Updated] (PIG-5051) Initialize PigContants.TASK_INDEX in spark mode correctly

2016-10-27 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5051: -- Attachment: PIG-5051.patch [~kexianda]: before we set PigConstants.TASK_INDEX in

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610287#comment-15610287 ] liyunzhang_intel commented on PIG-4920: --- [~mohitsabharwal] and [~xuefuz]: thanks for your review and

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607049#comment-15607049 ] liyunzhang_intel commented on PIG-4920: --- [~mohitsabharwal]: Would you take time to review

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Assignee: liyunzhang_intel Status: Patch Available (was: Open) > Fail to use Javascript

[jira] [Commented] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-25 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607244#comment-15607244 ] liyunzhang_intel commented on PIG-4920: --- [~xuefuz]: please checked in PIG-4920_6.patch. thanks! >

[jira] [Updated] (PIG-5029) Optimize sort case when data is skewed

2016-10-26 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5029: -- Attachment: PIG-5029_3.patch > Optimize sort case when data is skewed >

[jira] [Commented] (PIG-5052) Initialize MRConfiguration.JOB_ID in spark mode correctly

2016-11-07 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15644944#comment-15644944 ] liyunzhang_intel commented on PIG-5052: --- [~szita]: Sorry for reply a bit late. {quote} This can be

[jira] [Commented] (PIG-4899) The number of records of input file is calculated wrongly in spark mode in multiquery case

2016-11-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652651#comment-15652651 ] liyunzhang_intel commented on PIG-4899: --- [~szita]: i will try to reply you by the end of Friday if

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_5.patch [~mohitsabharwal]:submit PIG-4920_5.patch, just modify comments and

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-20 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_6.patch [~mohitsabharwal]: upload PIG-4920_6.patch, modification: 1. add

[jira] [Comment Edited] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-20 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590984#comment-15590984 ] liyunzhang_intel edited comment on PIG-4920 at 10/20/16 6:38 AM: -

[jira] [Created] (PIG-5044) Create SparlCompiler#getSamplingJob in spark mode

2016-10-14 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5044: - Summary: Create SparlCompiler#getSamplingJob in spark mode Key: PIG-5044 URL: https://issues.apache.org/jira/browse/PIG-5044 Project: Pig Issue Type:

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: PIG-4920_5.patch [~mohitsabharwal]: delete PIG-4920_5.patch because of missing

[jira] [Updated] (PIG-5044) Create SparlCompiler#getSamplingJob in spark mode

2016-10-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5044: -- Attachment: PIG-5044.patch [~kexianda]: Help review PIG-5044.patch > Create

[jira] [Comment Edited] (PIG-5044) Create SparlCompiler#getSamplingJob in spark mode

2016-10-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585999#comment-15585999 ] liyunzhang_intel edited comment on PIG-5044 at 10/18/16 5:11 PM: -

[jira] [Updated] (PIG-4920) Fail to use Javascript UDF in spark yarn client mode

2016-10-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4920: -- Attachment: (was: PIG-4920_5.patch) > Fail to use Javascript UDF in spark yarn client mode >

[jira] [Commented] (PIG-4969) Optimize combine case for spark mode

2016-10-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584275#comment-15584275 ] liyunzhang_intel commented on PIG-4969: --- [~xuefuz]: thanks for check in this patch,

[jira] [Resolved] (PIG-4969) Optimize combine case for spark mode

2016-10-17 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved PIG-4969. --- Resolution: Fixed > Optimize combine case for spark mode >

[jira] [Resolved] (PIG-4890) Run pigmix on spark on yarn with multiple nodes

2016-10-21 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel resolved PIG-4890. --- Resolution: Duplicate Duplicated with PIG-4846 > Run pigmix on spark on yarn with multiple

[jira] [Commented] (PIG-5054) Initialize SchemaTupleBackend correctly in backend in spark mode if spark job has more than 1 stage

2016-11-11 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657620#comment-15657620 ] liyunzhang_intel commented on PIG-5054: --- [~szita]: the issue is about PIG-4970, i think this will be

[jira] [Updated] (PIG-5068) Set SPARK_REDUCERS by pig.properties not by system configuration

2016-11-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5068: -- Component/s: spark > Set SPARK_REDUCERS by pig.properties not by system configuration >

[jira] [Updated] (PIG-5068) Set SPARK_REDUCERS by pig.properties not by system configuration

2016-11-22 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-5068: -- Issue Type: Sub-task (was: Bug) Parent: PIG-4856 > Set SPARK_REDUCERS by pig.properties

[jira] [Created] (PIG-5068) Set SPARK_REDUCERS by pig.properties not by system configuration

2016-11-22 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-5068: - Summary: Set SPARK_REDUCERS by pig.properties not by system configuration Key: PIG-5068 URL: https://issues.apache.org/jira/browse/PIG-5068 Project: Pig

<    5   6   7   8   9   10   11   12   13   14   >