[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824375#comment-13824375
 ] 

Hudson commented on PIG-3285:
-

SUCCESS: Integrated in HBase-TRUNK #4681 (See 
[https://builds.apache.org/job/HBase-TRUNK/4681/])
HBASE-9165 [mapreduce] Modularize building dependency jars

 - Separate adding HBase and dependencies from adding other job
   dependencies, and expose it as a separate method that other
   projects can use (for PIG-3285).
 - Explicitly add hbase-server to the list of dependencies we ship
   with the job, for users who extend the classes we provide (see
   HBASE-9112).
 - Add integration test for addDependencyJars.
 - Code reuse for TestTableMapReduce. (ndimiduk: rev 1542341)
* 
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestTableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/IdentityTableMapper.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java


> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Assigned] (PIG-3579) pig.script's deserialized version does not maintain line numbers

2013-11-15 Thread Aniket Mokashi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi reassigned PIG-3579:
---

Assignee: Jiaji Grace Zhang

> pig.script's deserialized version does not maintain line numbers
> 
>
> Key: PIG-3579
> URL: https://issues.apache.org/jira/browse/PIG-3579
> Project: Pig
>  Issue Type: Bug
>Reporter: Aniket Mokashi
>Assignee: Jiaji Grace Zhang
>
> If pig.script is decoded with base64, it loses line numbers because the 
> buffered reader that adds the lines, removes '\n's.
> {code}
> ScriptState.java#setScript
> protected void setScript(BufferedReader reader) {
> StringBuilder sb = new StringBuilder();
> try {
> String line = reader.readLine();
> while (line != null) {
> if (line.length() > 0) {
> sb.append(line).append("\n");
> }
> line = reader.readLine();
> }
> } catch (IOException e) {
> LOG.warn("unable to parse the script", e);
> }
> setScript(sb.toString());
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Moved] (PIG-3579) pig.script's deserialized version does not maintain line numbers

2013-11-15 Thread Aniket Mokashi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi moved MAPREDUCE-5629 to PIG-3579:


Key: PIG-3579  (was: MAPREDUCE-5629)
Project: Pig  (was: Hadoop Map/Reduce)

> pig.script's deserialized version does not maintain line numbers
> 
>
> Key: PIG-3579
> URL: https://issues.apache.org/jira/browse/PIG-3579
> Project: Pig
>  Issue Type: Bug
>Reporter: Aniket Mokashi
>
> If pig.script is decoded with base64, it loses line numbers because the 
> buffered reader that adds the lines, removes '\n's.
> {code}
> ScriptState.java#setScript
> protected void setScript(BufferedReader reader) {
> StringBuilder sb = new StringBuilder();
> try {
> String line = reader.readLine();
> while (line != null) {
> if (line.length() > 0) {
> sb.append(line).append("\n");
> }
> line = reader.readLine();
> }
> } catch (IOException e) {
> LOG.warn("unable to parse the script", e);
> }
> setScript(sb.toString());
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] Subscription: PIG patch available

2013-11-15 Thread jira 
Issue Subscription
Filter: PIG patch available (7 issues)

Subscriber: pigdaily

Key Summary
PIG-3572Fix all unit test for during build pig with Hadoop 2.X on Windows.
https://issues.apache.org/jira/browse/PIG-3572
PIG-3567LogicalPlanPrinter throws OOM for large scripts
https://issues.apache.org/jira/browse/PIG-3567
PIG-3453Implement a Storm backend to Pig
https://issues.apache.org/jira/browse/PIG-3453
PIG-3441Allow Pig to use default resources from Configuration objects
https://issues.apache.org/jira/browse/PIG-3441
PIG-3347Store invocation brings side effect
https://issues.apache.org/jira/browse/PIG-3347
PIG-3285Jobs using HBaseStorage fail to ship dependency jars
https://issues.apache.org/jira/browse/PIG-3285
PIG-2629Wrong Usage of Scalar which is null causes high namenode operation 
https://issues.apache.org/jira/browse/PIG-2629

You may edit this subscription at:
https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&filterId=12322384

[jira] [Updated] (PIG-3577) TestGrunt fails on trunk

2013-11-15 Thread Mark Wagner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated PIG-3577:
-

Attachment: TEST-org.apache.pig.test.TestGrunt.txt

> TestGrunt fails on trunk
> 
>
> Key: PIG-3577
> URL: https://issues.apache.org/jira/browse/PIG-3577
> Project: Pig
>  Issue Type: Bug
>Reporter: Mark Wagner
>Priority: Minor
> Attachments: TEST-org.apache.pig.test.TestGrunt.txt
>
>
> TestGrunt fails on trunk. It looks benign to me (a leftover file and a 
> parsing error), but it should be fixed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (PIG-3578) TestInvokerSpeed fails on trunk

2013-11-15 Thread Mark Wagner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated PIG-3578:
-

Attachment: TEST-org.apache.pig.test.TestInvokerSpeed.txt

> TestInvokerSpeed fails on trunk
> ---
>
> Key: PIG-3578
> URL: https://issues.apache.org/jira/browse/PIG-3578
> Project: Pig
>  Issue Type: Bug
>Reporter: Mark Wagner
> Attachments: TEST-org.apache.pig.test.TestInvokerSpeed.txt
>
>
> TestInvokerSpeed is failing on trunk. The allowed slowdown ratio is being 
> exceeded.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (PIG-3578) TestInvokerSpeed fails on trunk

2013-11-15 Thread Mark Wagner (JIRA) 
Mark Wagner created PIG-3578:


 Summary: TestInvokerSpeed fails on trunk
 Key: PIG-3578
 URL: https://issues.apache.org/jira/browse/PIG-3578
 Project: Pig
  Issue Type: Bug
Reporter: Mark Wagner


TestInvokerSpeed is failing on trunk. The allowed slowdown ratio is being 
exceeded.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (PIG-3577) TestGrunt fails on trunk

2013-11-15 Thread Mark Wagner (JIRA) 
Mark Wagner created PIG-3577:


 Summary: TestGrunt fails on trunk
 Key: PIG-3577
 URL: https://issues.apache.org/jira/browse/PIG-3577
 Project: Pig
  Issue Type: Bug
Reporter: Mark Wagner
Priority: Minor


TestGrunt fails on trunk. It looks benign to me (a leftover file and a parsing 
error), but it should be fixed.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (PIG-3568) Define the semantics of POStatus.STATUS_NULL

2013-11-15 Thread Cheolsoo Park (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cheolsoo Park updated PIG-3568:
---

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Committed.

> Define the semantics of POStatus.STATUS_NULL
> 
>
> Key: PIG-3568
> URL: https://issues.apache.org/jira/browse/PIG-3568
> Project: Pig
>  Issue Type: Bug
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Fix For: 0.13.0
>
> Attachments: PIG-3568.1.patch, PIG-3568.2.patch
>
>
> The meaning of POStatus.STATUS_NULL is not well documented and there are 
> conflicting view points on its interpretation. The two interpretations are:
> 1. POStatus.STATUS_NULL indicates that the pulled output IS null. This is 
> mostly found in expression operators, particularly comparison operators.
> 2. POStatus.STATUS_NULL indicates that the pull did not produce any output. 
> This is backed up by its usage in POPackage (not JoinPackage) for flattening 
> an empty bag, and PigGenericMapBase where pulls on the operator pipeline that 
> result in STATUS_NULL are discarded.
> I propose that 2 should be the official definition going forward. The first 
> meaning is easily indicated by (null, STATUS_OK) and all the relational 
> operators already seem to follow 2. I'd like to hear others' opinions as well 
> though.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3568) Define the semantics of POStatus.STATUS_NULL

2013-11-15 Thread Cheolsoo Park (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824262#comment-13824262
 ] 

Cheolsoo Park commented on PIG-3568:


+1. Will commit to trunk and merge down to tez branch.

> Define the semantics of POStatus.STATUS_NULL
> 
>
> Key: PIG-3568
> URL: https://issues.apache.org/jira/browse/PIG-3568
> Project: Pig
>  Issue Type: Bug
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Attachments: PIG-3568.1.patch, PIG-3568.2.patch
>
>
> The meaning of POStatus.STATUS_NULL is not well documented and there are 
> conflicting view points on its interpretation. The two interpretations are:
> 1. POStatus.STATUS_NULL indicates that the pulled output IS null. This is 
> mostly found in expression operators, particularly comparison operators.
> 2. POStatus.STATUS_NULL indicates that the pull did not produce any output. 
> This is backed up by its usage in POPackage (not JoinPackage) for flattening 
> an empty bag, and PigGenericMapBase where pulls on the operator pipeline that 
> result in STATUS_NULL are discarded.
> I propose that 2 should be the official definition going forward. The first 
> meaning is easily indicated by (null, STATUS_OK) and all the relational 
> operators already seem to follow 2. I'd like to hear others' opinions as well 
> though.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824195#comment-13824195
 ] 

Nick Dimiduk commented on PIG-3285:
---

More compatibility is better. I think that can be done with a little reflection 
magic. That logic will be pretty hairy as it spans at least 4 HBase releases. 
I'm not familiar with Pig internals so I'll leave a more sophisticated patch up 
to a maintainer (unless someone can illustrate how I can test my patches).

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Jarek Jarcec Cecho (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824186#comment-13824186
 ] 

Jarek Jarcec Cecho commented on PIG-3285:
-

I really do like idea behind this patch, thank you for driving the effort on 
both sides  [~ndimiduk]! Do you think that we can leave the older code laying 
around for couple of Pig releases and switch between those two conditionally 
based on the HBase version? I'm concerned a bit about backward compatibility 
here as this patch will make pig not working on a HBase releases where it is 
currently working.

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Jarek Jarcec Cecho (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824211#comment-13824211
 ] 

Jarek Jarcec Cecho commented on PIG-3285:
-

Yeah I agree, more compatibility is definitely better. Considering that the 
class {{TableMapReduceUtil}} have "always" been there, it might become fairly 
straightforward. Do you think that following snippet might work?

{code}
Method m = TableMapReduceUtil.class.getMethod("addHBaseDependencyJars", 
Configuration.class);
if(m != null) {
  m.invoke(null, job.getConfiguration());
} else {
  // Old code
}
{code}

And then the basic sanity testing would be with changing the HBase version on 
the command line, for example:

{code}
ant clean test -Dtestcase=TestHBaseStorage -Dhbase.version=0.94.1
ant clean test -Dtestcase=TestHBaseStorage -Dhbase.version=0.94.14
{code}

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (PIG-3568) Define the semantics of POStatus.STATUS_NULL

2013-11-15 Thread Mark Wagner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Wagner updated PIG-3568:
-

Attachment: PIG-3568.2.patch

Updated in response to Daniel's comments on RB.

> Define the semantics of POStatus.STATUS_NULL
> 
>
> Key: PIG-3568
> URL: https://issues.apache.org/jira/browse/PIG-3568
> Project: Pig
>  Issue Type: Bug
>Reporter: Mark Wagner
>Assignee: Mark Wagner
> Attachments: PIG-3568.1.patch, PIG-3568.2.patch
>
>
> The meaning of POStatus.STATUS_NULL is not well documented and there are 
> conflicting view points on its interpretation. The two interpretations are:
> 1. POStatus.STATUS_NULL indicates that the pulled output IS null. This is 
> mostly found in expression operators, particularly comparison operators.
> 2. POStatus.STATUS_NULL indicates that the pull did not produce any output. 
> This is backed up by its usage in POPackage (not JoinPackage) for flattening 
> an empty bag, and PigGenericMapBase where pulls on the operator pipeline that 
> result in STATUS_NULL are discarded.
> I propose that 2 should be the official definition going forward. The first 
> meaning is easily indicated by (null, STATUS_OK) and all the relational 
> operators already seem to follow 2. I'd like to hear others' opinions as well 
> though.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: Review Request 15524: PIG-3568: Define semantics of STATUS_NULL

2013-11-15 Thread Mark Wagner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15524/
---

(Updated Nov. 15, 2013, 11:22 p.m.)


Review request for pig, Cheolsoo Park, Daniel Dai, and Rohini Palaniswamy.


Bugs: PIG-3568
https://issues.apache.org/jira/browse/PIG-3568


Repository: pig-git


Description
---

This patch defines STATUS_NULL to mean that no result was produced and changes 
all conflicts to conform with this interpretation.


Diffs (updated)
-

  src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/POStatus.java 
6b7fa97 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java
 083702b 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/EqualToExpr.java
 dfe43b4 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/GTOrEqualToExpr.java
 64e180a 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/GreaterThanExpr.java
 77c2dfa 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/LTOrEqualToExpr.java
 2d7c888 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/LessThanExpr.java
 15ad635 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/NotEqualToExpr.java
 076bc98 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POAnd.java
 b9761c4 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POOr.java
 e80a37d 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POProject.java
 9d070de 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/PORelationToExprProject.java
 56a53e8 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/plans/PhysicalPlan.java
 c33e3ad 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java
 bb93273 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POFilter.java
 d6f4a4f 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POLocalRearrange.java
 bbf11c4 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java
 ffa211f 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPreCombinerLocalRearrange.java
 49c74bc 
  
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java
 3f4f914 
  test/org/apache/pig/test/TestEqualTo.java d827caa 
  test/org/apache/pig/test/TestGTOrEqual.java bb23eb4 
  test/org/apache/pig/test/TestGreaterThan.java 6422e27 
  test/org/apache/pig/test/TestLTOrEqual.java f9310d8 
  test/org/apache/pig/test/TestLessThan.java 49b044f 
  test/org/apache/pig/test/TestNotEqualTo.java cf1ffee 

Diff: https://reviews.apache.org/r/15524/diff/


Testing
---

'ant test-commit' on this patch, and 'ant test' before merging PIG-3570. There 
were a couple failures from 'ant test', but they're unrelated


Thanks,

Mark Wagner

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824214#comment-13824214
 ] 

Nick Dimiduk commented on PIG-3285:
---

I suspect that would work, but I've been bitten enough by the nuances of 
indirect dependency resolution at runtime that I'll withhold judgement until 
running a job on a real cluster ;)

You'll also need to add permutations for 0.96.0 and 0.96.1. At least with this 
single API, we should be able to iterate on our side without repercussions for 
Pig.

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-15 Thread Aniket Mokashi (JIRA) 
Aniket Mokashi created PIG-3576:
---

 Summary: NPE due to PIG-3549 when job never gets submitted
 Key: PIG-3576
 URL: https://issues.apache.org/jira/browse/PIG-3576
 Project: Pig
  Issue Type: Bug
Reporter: Aniket Mokashi
Priority: Trivial


NPE can happen if job never gets submitted due to errors like - input path does 
not exists. Following line (toString method) throws npe-
{code}
msg.append("JobID: " + j.getAssignedJobID().toString() + " Reason: " + 
j.getMessage());
{code}

Apparently, a.toString can throw NPE, but String.valueOf(a) handles null 
well(writes "null" for null values).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated PIG-3285:
--

Fix Version/s: (was: 0.11.1)
   0.13.0
   Status: Patch Available  (was: Open)

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated PIG-3285:
--

Attachment: 0001-PIG-3285-Add-HBase-dependency-jars.patch

Here's an updated patch. It would be nice to test and confirm end-to-end. 
Testing in my local checkout results in an NPE; I assume I'm doing something 
wrong.

{noformat}
$ ant clean && ant && ant -Dtestcase=TestHBaseStorage test
...
Testcase: org.apache.pig.test.TestHBaseStorage took 0 sec
Caused an ERROR
null not an instance of org.apache.hadoop.hbase.MiniHBaseCluster
java.lang.RuntimeException: null not an instance of 
org.apache.hadoop.hbase.MiniHBaseCluster
at 
org.apache.hadoop.hbase.HBaseTestingUtility.getMiniHBaseCluster(HBaseTestingUtility.java:701)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.getHBaseCluster(HBaseTestingUtility.java:1591)
at 
org.apache.pig.test.TestHBaseStorage.oneTimeTearDown(TestHBaseStorage.java:92)
{noformat}

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.11.1
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824159#comment-13824159
 ] 

Nick Dimiduk commented on PIG-3285:
---

Backport to HBase 0.94.14 is complete.

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.11.1
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (PIG-3548) Allow pig to load multiple paths specified in a filenames.txt

2013-11-15 Thread Alan Gates (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824148#comment-13824148
 ] 

Alan Gates commented on PIG-3548:
-

Could you store the parameters in a file rather than specify them on the 
command line?  See http://pig.apache.org/docs/r0.12.0/cont.html#Parameter-Sub 
for details.

> Allow pig to load multiple paths specified in a filenames.txt
> -
>
> Key: PIG-3548
> URL: https://issues.apache.org/jira/browse/PIG-3548
> Project: Pig
>  Issue Type: Improvement
>Reporter: Madhavi Nadig
>
> I have a list of paths stored in a filenames.txt. I would like to load them 
> all using a single LOAD command. The paths don't conform to one or more 
> regexes, so they have to specified individually.
> So far I've used the -param option with pig to specify them. But it results 
> in an extremely long commandline and I'm afraid I wont be able to scale my 
> script.
> shell : pig -param read_paths= something.pig
> something.pig : requests = LOAD '$read_paths' USING PigStorage(',');



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: Review Request 15565: PIG-3567 LogicalPlanPrinter throws OOM for large scripts

2013-11-15 Thread Aniket Mokashi 


> On Nov. 15, 2013, 9:53 p.m., Daniel Dai wrote:
> > Didn't go through line by line, but general approach looks good. You can 
> > add a unit test to check the memory footprint.

We had few scripts failing due to OOM because of this issue. With this change, 
they work now. I will add a test to show plan print before the change and after 
a change is same.


- Aniket


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15565/#review28990
---


On Nov. 15, 2013, 5:25 a.m., Aniket Mokashi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15565/
> ---
> 
> (Updated Nov. 15, 2013, 5:25 a.m.)
> 
> 
> Review request for pig, Cheolsoo Park, Daniel Dai, Julien Le Dem, and Rohini 
> Palaniswamy.
> 
> 
> Bugs: PIG-3567
> https://issues.apache.org/jira/browse/PIG-3567
> 
> 
> Repository: pig
> 
> 
> Description
> ---
> 
> Changed LPPrinter recursion to directly write to stream so that memory 
> footprint is reduced.
> 
> 
> Diffs
> -
> 
>   trunk/src/org/apache/pig/newplan/logical/optimizer/LogicalPlanPrinter.java 
> 1538708 
> 
> Diff: https://reviews.apache.org/r/15565/diff/
> 
> 
> Testing
> ---
> 
> Diff'ed explain plan of few existing scripts. No diffs found. I will try to 
> submit a testcase to show that LP print of before after is same (test need 
> not be committed).
> 
> 
> Thanks,
> 
> Aniket Mokashi
> 
>

Re: Review Request 15565: PIG-3567 LogicalPlanPrinter throws OOM for large scripts

2013-11-15 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15565/#review28990
---

Ship it!


Didn't go through line by line, but general approach looks good. You can add a 
unit test to check the memory footprint. 

- Daniel Dai


On Nov. 15, 2013, 5:25 a.m., Aniket Mokashi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15565/
> ---
> 
> (Updated Nov. 15, 2013, 5:25 a.m.)
> 
> 
> Review request for pig, Cheolsoo Park, Daniel Dai, Julien Le Dem, and Rohini 
> Palaniswamy.
> 
> 
> Bugs: PIG-3567
> https://issues.apache.org/jira/browse/PIG-3567
> 
> 
> Repository: pig
> 
> 
> Description
> ---
> 
> Changed LPPrinter recursion to directly write to stream so that memory 
> footprint is reduced.
> 
> 
> Diffs
> -
> 
>   trunk/src/org/apache/pig/newplan/logical/optimizer/LogicalPlanPrinter.java 
> 1538708 
> 
> Diff: https://reviews.apache.org/r/15565/diff/
> 
> 
> Testing
> ---
> 
> Diff'ed explain plan of few existing scripts. No diffs found. I will try to 
> submit a testcase to show that LP print of before after is same (test need 
> not be committed).
> 
> 
> Thanks,
> 
> Aniket Mokashi
> 
>

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824071#comment-13824071
 ] 

Hudson commented on PIG-3285:
-

SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #838 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/838/])
HBASE-9165 [mapreduce] Modularize building dependency jars

 - Separate adding HBase and dependencies from adding other job
   dependencies, and expose it as a separate method that other
   projects can use (for PIG-3285).
 - Explicitly add hbase-server to the list of dependencies we ship
   with the job, for users who extend the classes we provide (see
   HBASE-9112).
 - Add integration test for addDependencyJars.
 - Code reuse for TestTableMapReduce. (ndimiduk: rev 1542341)
* 
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestTableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/IdentityTableMapper.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java


> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.11.1
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: Review Request 15524: PIG-3568: Define semantics of STATUS_NULL

2013-11-15 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15524/#review28982
---



src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java


Seems this is an optimization, shall we retain it?



src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java


See previous comment



src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java


See previous comment


Another place to get rid of STATUS_NULL is POProject:313.

Also in PORelationToExprProject (the actual code is in 
POProject.consumeInputBag), we shall make sure we ignore STATUS_NULL.

- Daniel Dai


On Nov. 14, 2013, 6:31 p.m., Mark Wagner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15524/
> ---
> 
> (Updated Nov. 14, 2013, 6:31 p.m.)
> 
> 
> Review request for pig, Cheolsoo Park, Daniel Dai, and Rohini Palaniswamy.
> 
> 
> Bugs: PIG-3568
> https://issues.apache.org/jira/browse/PIG-3568
> 
> 
> Repository: pig-git
> 
> 
> Description
> ---
> 
> This patch defines STATUS_NULL to mean that no result was produced and 
> changes all conflicts to conform with this interpretation.
> 
> 
> Diffs
> -
> 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/POStatus.java 
> 6b7fa97 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java
>  083702b 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/EqualToExpr.java
>  dfe43b4 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/GTOrEqualToExpr.java
>  64e180a 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/GreaterThanExpr.java
>  77c2dfa 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/LTOrEqualToExpr.java
>  2d7c888 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/LessThanExpr.java
>  15ad635 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/NotEqualToExpr.java
>  076bc98 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POAnd.java
>  b9761c4 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POOr.java
>  e80a37d 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/plans/PhysicalPlan.java
>  c33e3ad 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java
>  bb93273 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POFilter.java
>  d6f4a4f 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POLocalRearrange.java
>  bbf11c4 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java
>  ffa211f 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPreCombinerLocalRearrange.java
>  49c74bc 
>   
> src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java
>  3f4f914 
>   test/org/apache/pig/test/TestEqualTo.java d827caa 
>   test/org/apache/pig/test/TestGTOrEqual.java bb23eb4 
>   test/org/apache/pig/test/TestGreaterThan.java 6422e27 
>   test/org/apache/pig/test/TestLTOrEqual.java f9310d8 
>   test/org/apache/pig/test/TestLessThan.java 49b044f 
>   test/org/apache/pig/test/TestNotEqualTo.java cf1ffee 
> 
> Diff: https://reviews.apache.org/r/15524/diff/
> 
> 
> Testing
> ---
> 
> 'ant test-commit' on this patch, and 'ant test' before merging PIG-3570. 
> There were a couple failures from 'ant test', but they're unrelated
> 
> 
> Thanks,
> 
> Mark Wagner
> 
>

[jira] [Commented] (PIG-3285) Jobs using HBaseStorage fail to ship dependency jars

2013-11-15 Thread Nick Dimiduk (JIRA) 

[ 
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823910#comment-13823910
 ] 

Nick Dimiduk commented on PIG-3285:
---

FYI HBASE-9165 is committed to 0.96 and trunk. It will be available with HBase 
0.96.1. I hope to also backport it to 0.94, available on 0.94.14 and later. The 
new method is 
[TableMapReduceUtil#addHBaseDependencyJars(Configuration)|https://github.com/apache/hbase/commit/8d844e61a55c93bf23cf9427e02f9814b66d68a8#diff-7d445ba2b99d6f251da23153a1cc82f4R560].
 I'm now using it in a new hbase bin script command {{mapredcp}}, as of 
HBASE-8438. Its output is as follows:

{noformat}
$ ./bin/hbase mapredcp | tr ':' '\n'
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/netty-3.6.6.Final.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-hadoop-compat-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/protobuf-java-2.5.0.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/guava-12.0.1.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/htrace-core-2.01.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-protocol-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-client-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/zookeeper-3.4.5.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-server-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-common-0.97.0-SNAPSHOT.jar
{noformat}

> Jobs using HBaseStorage fail to ship dependency jars
> 
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
>  Issue Type: Bug
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.11.1
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch, 
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user 
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies. 
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running 
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
>   at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.(HbaseObjectWritable.java:266)
>   at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
>   at 
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
>   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
>   at $Proxy7.getProtocolVersion(Unknown Source)
>   at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
>   at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (PIG-3575) Provide information about logical operators in jobconfiguration

2013-11-15 Thread Sudhanshu Arora (JIRA) 
Sudhanshu Arora created PIG-3575:


 Summary: Provide information about logical operators in 
jobconfiguration
 Key: PIG-3575
 URL: https://issues.apache.org/jira/browse/PIG-3575
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.12.0
Reporter: Sudhanshu Arora


Today there is no way to gather information about pig logical operations 
through job configuration file. It will be beneficial to provide that 
information so that process that are monitoring(/intercepting) mapreduce jobs 
can provide details about the Pig operations at logical level.




--
This message was sent by Atlassian JIRA
(v6.1#6144)