[jira] [Commented] (PIG-3453) Implement a Storm backend to Pig

2014-09-24 Thread Mridul Jain (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147446#comment-14147446 ] Mridul Jain commented on PIG-3453: -- This bug seems to be closed. I will open a new Jira soo

[jira] [Commented] (PIG-4168) Initial implementation of unit tests for Pig on Spark

2014-09-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147233#comment-14147233 ] liyunzhang_intel commented on PIG-4168: --- Hi [~praveenr019] : i add SparkMiniCluster in

[jira] [Updated] (PIG-4168) Initial implementation of unit tests for Pig on Spark

2014-09-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4168: -- Attachment: PIG-4168_3.patch > Initial implementation of unit tests for Pig on Spark > ---

[jira] [Updated] (PIG-4050) HadoopShims.getTaskReports() can cause OOM with Hadoop 2

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4050: Assignee: Rohini Palaniswamy Status: Patch Available (was: Open) [~cheolsoo], With

[jira] [Updated] (PIG-4050) HadoopShims.getTaskReports() can cause OOM with Hadoop 2

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4050: Attachment: PIG-4050-1.patch > HadoopShims.getTaskReports() can cause OOM with Hadoop 2 >

[jira] [Commented] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script

2014-09-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147128#comment-14147128 ] Daniel Dai commented on PIG-4160: - hcat/hive jars are known to be locally, so bin/pig will p

[jira] [Updated] (PIG-4160) -forcelocaljars / -j flag when using a remote url for a script

2014-09-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4160: Attachment: PIG-4160-2.patch Thinking that again, I feel we might need to deal with mixed jar origin in one j

[jira] [Updated] (PIG-4080) Add Preprocessor commands and more to the black/whitelisting feature

2014-09-24 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-4080: - Patch Info: Patch Available > Add Preprocessor commands and more to the black/whitelisting f

[jira] [Resolved] (PIG-4162) Intermediate reducer parallelism in Tez should be higher

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy resolved PIG-4162. - Resolution: Fixed Hadoop Flags: Reviewed > Intermediate reducer parallelism in Tez sho

[jira] [Updated] (PIG-4162) Intermediate reducer parallelism in Tez should be higher

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4162: Attachment: PIG-4162-3.patch Committed to trunk (0.14). Thanks for the review Daniel > Interm

[jira] [Commented] (PIG-4199) Mapreduce ACLs should be translated to Tez ACLs

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146574#comment-14146574 ] Rohini Palaniswamy commented on PIG-4199: - [~sseth], What is the reason for separ

[jira] [Created] (PIG-4199) Mapreduce ACLs should be translated to Tez ACLs

2014-09-24 Thread Rohini Palaniswamy (JIRA)
Rohini Palaniswamy created PIG-4199: --- Summary: Mapreduce ACLs should be translated to Tez ACLs Key: PIG-4199 URL: https://issues.apache.org/jira/browse/PIG-4199 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-4199) Mapreduce ACLs should be translated to Tez ACLs

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-4199: Issue Type: Sub-task (was: Bug) Parent: PIG-3446 > Mapreduce ACLs should be translate

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

2014-09-24 Thread Daniel Dai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25912/#review54427 --- Ship it! Ship It! - Daniel Dai On Sept. 24, 2014, 3:27 p.m., Roh

Re: Branching Pig 0.14

2014-09-24 Thread Daniel Dai
@Prashant, I just posted my review comments @Russell, Pig on Spark is still in development and will not be in 0.14 On Wed, Sep 24, 2014 at 12:06 AM, Russell Jurney wrote: > Anyone know if pig on spark will be included in 0.14? > > On Tuesday, September 23, 2014, Rohini Palaniswamy > wrote: > >>

[jira] [Commented] (PIG-4193) Make collected group work with Spark

2014-09-24 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146527#comment-14146527 ] Praveen Rachabattuni commented on PIG-4193: --- [~rohini] Thanks for checking. Will m

[jira] [Commented] (PIG-4193) Make collected group work with Spark

2014-09-24 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146434#comment-14146434 ] Rohini Palaniswamy commented on PIG-4193: - [~daijy], Can you review this one? [~

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

2014-09-24 Thread Rohini Palaniswamy
> On Sept. 24, 2014, 5:32 a.m., Daniel Dai wrote: > > I think we might need a flag to enable/disable this feature. This feature > > might confuse some users when Pig override parallelism statement. In > > addition, our auto-parallel algorithm is not sophisticated enough, we shall > > provide a

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

2014-09-24 Thread Rohini Palaniswamy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25912/ --- (Updated Sept. 24, 2014, 3:27 p.m.) Review request for pig, Cheolsoo Park and D

[jira] [Updated] (PIG-4198) Add operator to pig latin for performing caching

2014-09-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4198: -- Description: how to use: CACHE alias You can use "CACHE" liking following way: 1. hadoop fs -cat

[jira] [Updated] (PIG-4193) Make collected group work with Spark

2014-09-24 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni updated PIG-4193: -- Attachment: PIG-4193-1.patch Made some changes in POCollectedGroup class, since there was

[jira] [Assigned] (PIG-4193) Make collected group work with Spark

2014-09-24 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni reassigned PIG-4193: - Assignee: Praveen Rachabattuni > Make collected group work with Spark > ---

[jira] [Commented] (PIG-4174) e2e tests for Spark

2014-09-24 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146272#comment-14146272 ] Praveen Rachabattuni commented on PIG-4174: --- Currently, I see a conflict with hado

[jira] [Commented] (PIG-4168) Initial implementation of unit tests for Pig on Spark

2014-09-24 Thread Praveen Rachabattuni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146186#comment-14146186 ] Praveen Rachabattuni commented on PIG-4168: --- Looks fine on my end. Should SparkMi

[jira] [Created] (PIG-4198) Add operator to pig latin for performing caching

2014-09-24 Thread liyunzhang_intel (JIRA)
liyunzhang_intel created PIG-4198: - Summary: Add operator to pig latin for performing caching Key: PIG-4198 URL: https://issues.apache.org/jira/browse/PIG-4198 Project: Pig Issue Type: New Fe

[jira] [Commented] (PIG-4168) Initial implementation of unit tests for Pig on Spark

2014-09-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146024#comment-14146024 ] liyunzhang_intel commented on PIG-4168: --- update PIG-4168_2.patch. Removed comment line

[jira] [Updated] (PIG-4168) Initial implementation of unit tests for Pig on Spark

2014-09-24 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated PIG-4168: -- Attachment: PIG-4168_2.patch > Initial implementation of unit tests for Pig on Spark > ---

Re: Branching Pig 0.14

2014-09-24 Thread Russell Jurney
Anyone know if pig on spark will be included in 0.14? On Tuesday, September 23, 2014, Rohini Palaniswamy wrote: > Please mark jira Patch available if ready for review and needs to go into > 0.14. I miss the "Patch available" notifications. Not sure when INFRA will > fix them. I will see if I can