[jira] [Updated] (PIG-3540) e2e test for tez

2013-10-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3540: Description: Need to add e2e tests for tez. Should be able to compare the tez run with mapreduce run. > e2e

[jira] [Updated] (PIG-3540) e2e test for tez

2013-10-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3540: Attachment: PIG-3540-1.patch Here is the command line I use to run e2e tests: ant -Dharness.cluster.conf=/Us

[jira] [Assigned] (PIG-2674) Document how parameter values that contain spaces can be passed to Pig

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi reassigned PIG-2674: --- Assignee: Aniket Mokashi > Document how parameter values that contain spaces can be passed to P

[jira] [Assigned] (PIG-3463) Pig should use hadoop local mode for small jobs

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi reassigned PIG-3463: --- Assignee: Aniket Mokashi > Pig should use hadoop local mode for small jobs > --

[jira] [Assigned] (PIG-3473) org.apache.pig.Expression should support "is null" and "not" operations

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi reassigned PIG-3473: --- Assignee: Aniket Mokashi > org.apache.pig.Expression should support "is null" and "not" operati

[jira] [Work started] (PIG-3368) doc pig flatten operator applied to empty vs null bag

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on PIG-3368 started by Aniket Mokashi. > doc pig flatten operator applied to empty vs null bag > - > >

[jira] [Updated] (PIG-3368) doc pig flatten operator applied to empty vs null bag

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3368: Fix Version/s: 0.13.0 > doc pig flatten operator applied to empty vs null bag > -

[jira] [Created] (PIG-3541) Add diagnostic information to TezStats

2013-10-22 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3541: -- Summary: Add diagnostic information to TezStats Key: PIG-3541 URL: https://issues.apache.org/jira/browse/PIG-3541 Project: Pig Issue Type: Sub-task Com

[jira] Subscription: PIG patch available

2013-10-22 Thread jira
Issue Subscription Filter: PIG patch available (15 issues) Subscriber: pigdaily Key Summary PIG-3535Add test-tez target to build.xml https://issues.apache.org/jira/browse/PIG-3535 PIG-3531Add TezMiniCluster for unit tests https://issues.apache.org/jira/bro

[jira] [Created] (PIG-3540) e2e test for tez

2013-10-22 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-3540: --- Summary: e2e test for tez Key: PIG-3540 URL: https://issues.apache.org/jira/browse/PIG-3540 Project: Pig Issue Type: Sub-task Components: tez Rep

[jira] [Created] (PIG-3539) Pig should be able to submit multiple DAG

2013-10-22 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-3539: --- Summary: Pig should be able to submit multiple DAG Key: PIG-3539 URL: https://issues.apache.org/jira/browse/PIG-3539 Project: Pig Issue Type: Sub-task Compon

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Alan Gates
I don't think we should change our use of stable. Our usage is in line with the Hadoop usage of the term in their releases. To the best of our knowledge as Apache developers it is stable. It passes all of the tests we have. We have no criteria for deciding stability beyond this. Alan. On O

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Daniel Dai
Yes, we can revisit. The question is how to determine the stability? 0.11.1 is released for a while and should be considered stable, but actually it contains problem raised just recently. After we release 0.12.1, how soon should we declare it a stable release? Thanks, Daniel On Tue, Oct 22, 2013

[jira] [Updated] (PIG-3325) Adding a tuple to a bag is slow

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3325: Resolution: Fixed Status: Resolved (was: Patch Available) > Adding a tuple to a bag is slow

[jira] [Commented] (PIG-3325) Adding a tuple to a bag is slow

2013-10-22 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802378#comment-13802378 ] Aniket Mokashi commented on PIG-3325: - Also, committed to 0.12. > Adding a tuple to a b

Re: svn commit: r1531441 - in /pig/trunk: CHANGES.txt src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java test/org/apache/pig/test/TestJobSubmission.java

2013-10-22 Thread Aniket Mokashi
I don't see PIG-3512 in CHANGES.txt, is that expected? On Fri, Oct 11, 2013 at 2:54 PM, wrote: > Author: daijy > Date: Fri Oct 11 21:54:23 2013 > New Revision: 1531441 > > URL: http://svn.apache.org/r1531441 > Log: > PIG-3512: Reducer estimater is broken by PIG-3497 > > Modified: > pig/trun

[jira] [Updated] (PIG-3531) Add TezMiniCluster for unit tests

2013-10-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3531: --- Attachment: PIG-3531-3.patch Incorporated Rohini's comments in a new patch. > Add TezMiniCluster for u

Re: Review Request 14801: PIG-3531 Add TezMiniCluster for unit tests

2013-10-22 Thread Cheolsoo Park
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14801/ --- (Updated Oct. 22, 2013, 10:04 p.m.) Review request for pig, Daniel Dai, Mark Wa

[jira] [Assigned] (PIG-3538) Make limit work

2013-10-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park reassigned PIG-3538: -- Assignee: Alex Bain Assigning to Alex. > Make limit work > --- > > K

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Koji Noguchi
Thanks Daniel, Olga! Keeping 3 versions would be nice. As for 'stable', can we revisit the definition? If it's *always* pointing to the latest release, I don't see the need for having this link(dir). Is it adding any value? Koji On Oct 22, 2013, at 1:43 PM, Daniel Dai wrote: > That's to

Re: Review Request 14801: PIG-3531 Add TezMiniCluster for unit tests

2013-10-22 Thread Cheolsoo Park
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14801/ --- (Updated Oct. 22, 2013, 9:21 p.m.) Review request for pig, Daniel Dai, Mark Wag

[jira] [Created] (PIG-3538) Make limit work

2013-10-22 Thread Alex Bain (JIRA)
Alex Bain created PIG-3538: -- Summary: Make limit work Key: PIG-3538 URL: https://issues.apache.org/jira/browse/PIG-3538 Project: Pig Issue Type: Sub-task Components: tez Affects Versio

[jira] [Updated] (PIG-3533) Make register work

2013-10-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3533: Attachment: PIG-3533-1.patch Will add unit test later > Make register work > -- > >

[jira] [Updated] (PIG-3532) Empty plan fails to run

2013-10-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3532: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Patch commi

[jira] [Created] (PIG-3537) Tweak build to support HBase with Hadoop 23 profile

2013-10-22 Thread Jarek Jarcec Cecho (JIRA)
Jarek Jarcec Cecho created PIG-3537: --- Summary: Tweak build to support HBase with Hadoop 23 profile Key: PIG-3537 URL: https://issues.apache.org/jira/browse/PIG-3537 Project: Pig Issue Type:

[jira] [Updated] (PIG-3529) Upgrade HBase dependency from 0.95-SNAPSHOT to 0.96

2013-10-22 Thread Jarek Jarcec Cecho (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Jarcec Cecho updated PIG-3529: Attachment: PIG-3529.patch The issue was in following exception: {code} Caused by: java.lan

[jira] [Updated] (PIG-3529) Upgrade HBase dependency from 0.95-SNAPSHOT to 0.96

2013-10-22 Thread Jarek Jarcec Cecho (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Jarcec Cecho updated PIG-3529: Status: Patch Available (was: Open) > Upgrade HBase dependency from 0.95-SNAPSHOT to 0.96 >

[jira] [Commented] (PIG-2629) Wrong Usage of Scalar which is null causes high namenode operation

2013-10-22 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802107#comment-13802107 ] Rohini Palaniswamy commented on PIG-2629: - We hit this again. Need to also fix ReadS

[jira] [Assigned] (PIG-2629) Wrong Usage of Scalar which is null causes high namenode operation

2013-10-22 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy reassigned PIG-2629: --- Assignee: Rohini Palaniswamy > Wrong Usage of Scalar which is null causes high namenode

[jira] [Commented] (PIG-3529) Upgrade HBase dependency from 0.95-SNAPSHOT to 0.96

2013-10-22 Thread Jarek Jarcec Cecho (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802066#comment-13802066 ] Jarek Jarcec Cecho commented on PIG-3529: - Just upgrading the version is not working

Re: Review Request 14801: PIG-3531 Add TezMiniCluster for unit tests

2013-10-22 Thread Rohini Palaniswamy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14801/#review27313 --- Would be good to avoid mentioning TezMiniCluster in test code. What

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Daniel Dai
That's totally make sense. Let's keep both download/documentation for 3 versions. Thanks, Daniel On Tue, Oct 22, 2013 at 10:20 AM, Olga Natkovich wrote: > Couple of suggestions: > > (1) I think we are trying to go for a more frequent release model and in > that case it would make sense to keep

Attach bag for each tuple and pass to UDF

2013-10-22 Thread Serega Sheypak
Hi, I have two relations: relation *rows* (>10GB) relation *tinyDictionary* (<1MB) I want to take each tuple from *rows* and attach *tinyDictionary *to it. And then pass it to python UDF: result = FOREACH someRelation GENERATE udf.my_python_udf(single_row_from_* Rows*, whole*TinyDictionary*); Ho

[jira] [Created] (PIG-3536) Make distinct work

2013-10-22 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3536: -- Summary: Make distinct work Key: PIG-3536 URL: https://issues.apache.org/jira/browse/PIG-3536 Project: Pig Issue Type: Sub-task Components: tez Aff

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Olga Natkovich
Couple of suggestions: (1) I think we are trying to go for a more frequent release model and in that case it would make sense to keep perhaps 3 releases. Based on our experience at Yahoo, Pig 10 is the really stable release. We recently found a couple of critical bugs in 11 for which we posted

Re: How do we determine 'stable' pig version?

2013-10-22 Thread Daniel Dai
Hi, Koji, Here is the criteria I use: (i) How do we determine how many releases to show on the front download page? We usually keep two most recent releases on the front page according to https://cwiki.apache.org/confluence/display/PIG/HowToRelease. (ii) How do we determine which release is consid

How do we determine 'stable' pig version?

2013-10-22 Thread Koji Noguchi
When I went to the pig release download page (through http://www.apache.org/dyn/closer.cgi/pig), I only saw 0.11.1 and 0.12 available. I later learned that there is an 'archive' link(http://archive.apache.org/dist/pig/) that list other versions (0.8 to 0.10). Two questions. (i) How do we d