Build failed in Jenkins: Pig-trunk-commit #814

[Apache Jenkins Server]

See 

Changes:

[daijy] PIG-2044: Patten match bug in org.apache.pig.newplan.optimizer.Rule

--
[...truncated 5553 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

javadoc-jar:
  [jar] Building jar: 

[jira] [Updated] (PIG-2044) Patten match bug in org.apache.pig.newplan.optimizer.Rule

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2044:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Test-patch:
 [exec] +1 overall.  
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 3 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.

Patch commit to trunk. Thanks Koji!

> Patten match bug in org.apache.pig.newplan.optimizer.Rule
> -
>
> Key: PIG-2044
> URL: https://issues.apache.org/jira/browse/PIG-2044
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Koji Noguchi
> Fix For: 0.10
>
> Attachments: PIG-2044-00.patch
>
>
> Koji find that we have a bug org.apache.pig.newplan.optimizer.Rule. The 
> "break" in line 179 seems to be wrong. This multiple branch matching is not 
> used in Pig, but could be a problem for the future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2044) Patten match bug in org.apache.pig.newplan.optimizer.Rule

[Koji Noguchi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Noguchi updated PIG-2044:
--

Status: Patch Available  (was: Open)

> Patten match bug in org.apache.pig.newplan.optimizer.Rule
> -
>
> Key: PIG-2044
> URL: https://issues.apache.org/jira/browse/PIG-2044
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Koji Noguchi
> Fix For: 0.10
>
> Attachments: PIG-2044-00.patch
>
>
> Koji find that we have a bug org.apache.pig.newplan.optimizer.Rule. The 
> "break" in line 179 seems to be wrong. This multiple branch matching is not 
> used in Pig, but could be a problem for the future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2076) update documentation, help command with correct default value of pig.cachedbag.memusage

[Richard Ding (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033394#comment-13033394
 ] 

Richard Ding commented on PIG-2076:
---

+1

> update documentation, help command with correct default value of 
> pig.cachedbag.memusage
> ---
>
> Key: PIG-2076
> URL: https://issues.apache.org/jira/browse/PIG-2076
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2076.1.patch
>
>
> The default value of pig.cachedbag.memusage was changed to 0.2 in pig 0.8, as 
> part of changes in PIG-1447 .
> But the help command and documentation shows older default value of 0.1 .

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2044) Patten match bug in org.apache.pig.newplan.optimizer.Rule

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033393#comment-13033393
 ] 

Daniel Dai commented on PIG-2044:
-

+1, will commit if tests pass.

> Patten match bug in org.apache.pig.newplan.optimizer.Rule
> -
>
> Key: PIG-2044
> URL: https://issues.apache.org/jira/browse/PIG-2044
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Koji Noguchi
> Fix For: 0.10
>
> Attachments: PIG-2044-00.patch
>
>
> Koji find that we have a bug org.apache.pig.newplan.optimizer.Rule. The 
> "break" in line 179 seems to be wrong. This multiple branch matching is not 
> used in Pig, but could be a problem for the future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #813

[Apache Jenkins Server]

See 

Changes:

[daijy] PIG-2075: Bring back TestNewPlanPushUpFilter

--
[...truncated 5561 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

javadoc-jar:
  [jar] Building jar: 

[jira] [Updated] (PIG-2076) update documentation, help command with correct default value of pig.cachedbag.memusage

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2076:
---

Attachment: PIG-2076.1.patch

> update documentation, help command with correct default value of 
> pig.cachedbag.memusage
> ---
>
> Key: PIG-2076
> URL: https://issues.apache.org/jira/browse/PIG-2076
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2076.1.patch
>
>
> The default value of pig.cachedbag.memusage was changed to 0.2 in pig 0.8, as 
> part of changes in PIG-1447 .
> But the help command and documentation shows older default value of 0.1 .

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2076) update documentation, help command with correct default value of pig.cachedbag.memusage

[Thejas M Nair (JIRA)]

update documentation, help command with correct default value of 
pig.cachedbag.memusage
---

 Key: PIG-2076
 URL: https://issues.apache.org/jira/browse/PIG-2076
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Fix For: 0.9.0


The default value of pig.cachedbag.memusage was changed to 0.2 in pig 0.8, as 
part of changes in PIG-1447 .
But the help command and documentation shows older default value of 0.1 .


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2069) LoadFunc jar does not ship to backend in MultiQuery case

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033386#comment-13033386
 ] 

Daniel Dai commented on PIG-2069:
-

+1 if test pass.

> LoadFunc jar does not ship to backend in MultiQuery case
> 
>
> Key: PIG-2069
> URL: https://issues.apache.org/jira/browse/PIG-2069
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-2069.patch
>
>
> Pig is able to automatically figure out the jar containing the LoadFunc and 
> ship them to backend. However, the following script didn't:
> {code}
> A = load '1.txt' using SomeLoadFunc();
> B = filter A by $0==0;
> C = filter A by $1==1;
> D = join B by $0, C by $0;
> dump D;
> {code}
> The reason is this query is a multiquery (A is reused and thus create an 
> implicit split). When we merge multiquery into one job, we didn't merge udfs 
> list properly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2075) Bring back TestNewPlanPushUpFilter

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-2075.
-

Resolution: Fixed

New test case committed to both trunk and 0.9 branch.

> Bring back TestNewPlanPushUpFilter
> --
>
> Key: PIG-2075
> URL: https://issues.apache.org/jira/browse/PIG-2075
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: TestNewPlanPushUpFilter.java
>
>
> When I commit PIG-1575, forget the file TestNewPlanPushUpFilter. Bring it 
> back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2075) Bring back TestNewPlanPushUpFilter

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2075:


Attachment: TestNewPlanPushUpFilter.java

> Bring back TestNewPlanPushUpFilter
> --
>
> Key: PIG-2075
> URL: https://issues.apache.org/jira/browse/PIG-2075
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: TestNewPlanPushUpFilter.java
>
>
> When I commit PIG-1575, forget the file TestNewPlanPushUpFilter. Bring it 
> back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2054) Need to clarify globbing on command line vs in load statement

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel resolved PIG-2054.
--

  Resolution: Fixed
Hadoop Flags: [Reviewed]

Documentation updated. 
Fix will be included in the GA patch for PIG-1772.
Thanks for review Olga!


> Need to clarify globbing on command line vs in load statement
> -
>
> Key: PIG-2054
> URL: https://issues.apache.org/jira/browse/PIG-2054
> Project: Pig
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Olga Natkovich
>Assignee: Corinne Chandel
> Fix For: 0.9.0
>
>
> We had several user reports saying that "globbing in Pig and Hadoop are not 
> the same". They based this assertion on the fact that some patterns work from 
> hadoop command line but would not work in Pig load statement.
> Pig uses Hadoop globbing so the functionality is identical; however, when you 
> run on command line, shell can be doing some of the substitution giving 
> impression that things are different.
> Example:
> hadoop fs -ls 
> /mydata/20110423{00,01,02,03,04,05,06,07,08,09,{10..23}}00/*/part* - this 
> works
> LOAD '/mydata/20110423{00,01,02,03,04,05,06,07,08,09,{10..23}}00/*/part*' - 
> this does not
> We should add a note to the description of globbing 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2075) Bring back TestNewPlanPushUpFilter

[Daniel Dai (JIRA)]

Bring back TestNewPlanPushUpFilter
--

 Key: PIG-2075
 URL: https://issues.apache.org/jira/browse/PIG-2075
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Daniel Dai
Assignee: Daniel Dai
 Fix For: 0.9.0


When I commit PIG-1575, forget the file TestNewPlanPushUpFilter. Bring it back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2073) update union documentation - numeric types will not be converted to chararray

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel resolved PIG-2073.
--

  Resolution: Fixed
Hadoop Flags: [Reviewed]

Documentation updated. 
Fix will be included in the GA patch for PIG-1772.
Thanks for review Thejas!


> update union documentation - numeric types will  not be converted to chararray
> --
>
> Key: PIG-2073
> URL: https://issues.apache.org/jira/browse/PIG-2073
> Project: Pig
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Corinne Chandel
> Fix For: 0.9.0
>
>
> As a result of changes in PIG-2071, the documentation needs to be updated -
> {noformat}
>  Union columns of compatible type will produce an "escalate" type. The 
> priority is chararray > double > float > long > int > bytearray, 
> tuple|bag|map > bytearray:
> {noformat}
> should be changed to remove chararray in the numeric type sequence. ie - 
> {noformat}
>  Union columns of compatible type will produce an "escalate" type. The 
> priority is double > float > long > int > bytearray, tuple|bag|map|chararray 
> > bytearray:
> {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2068) documentation of union onschema restrictions need to to be updated

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel resolved PIG-2068.
--

Resolution: Fixed

Fixed.

> documentation of union onschema restrictions need to to be updated
> --
>
> Key: PIG-2068
> URL: https://issues.apache.org/jira/browse/PIG-2068
> Project: Pig
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Corinne Chandel
> Fix For: 0.9.0
>
>
> The following requirement mentioned under union-onschema section is no longer 
> applicable -
> {code}
> The data type for columns with same name in different input schemas should be 
> compatible:
> Numeric types are compatible, and if column having same name in different 
> input schemas have different numeric types, an implicit conversion will 
> happen.
> Bytearray type is considered compatible with all other types, a cast will 
> be added to convert to other type.
> Bags or tuples having different inner schema are considered incompatible.
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2068) documentation of union onschema restrictions need to to be updated

[Corinne Chandel (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033375#comment-13033375
 ] 

Corinne Chandel commented on PIG-2068:
--

Documentation updated. 
Fix will be included in the GA patch for PIG-1772.
Thanks for review Thejas!

> documentation of union onschema restrictions need to to be updated
> --
>
> Key: PIG-2068
> URL: https://issues.apache.org/jira/browse/PIG-2068
> Project: Pig
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Corinne Chandel
> Fix For: 0.9.0
>
>
> The following requirement mentioned under union-onschema section is no longer 
> applicable -
> {code}
> The data type for columns with same name in different input schemas should be 
> compatible:
> Numeric types are compatible, and if column having same name in different 
> input schemas have different numeric types, an implicit conversion will 
> happen.
> Bytearray type is considered compatible with all other types, a cast will 
> be added to convert to other type.
> Bags or tuples having different inner schema are considered incompatible.
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2069) LoadFunc jar does not ship to backend in MultiQuery case

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-2069:
--

Attachment: PIG-2069.patch

This happens when the original MapReduce DAG (before optimization) contains a 
diamond node.

User can workaround this by explicitly registering the LoadFunc jar in the 
script.

The attached patch provides a fix. It's verified with manual test.

> LoadFunc jar does not ship to backend in MultiQuery case
> 
>
> Key: PIG-2069
> URL: https://issues.apache.org/jira/browse/PIG-2069
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-2069.patch
>
>
> Pig is able to automatically figure out the jar containing the LoadFunc and 
> ship them to backend. However, the following script didn't:
> {code}
> A = load '1.txt' using SomeLoadFunc();
> B = filter A by $0==0;
> C = filter A by $1==1;
> D = join B by $0, C by $0;
> dump D;
> {code}
> The reason is this query is a multiquery (A is reused and thus create an 
> implicit split). When we merge multiquery into one job, we didn't merge udfs 
> list properly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #812

[Apache Jenkins Server]

See 

Changes:

[daijy] PIG-2067: FilterLogicExpressionSimplifier removed some branches in some 
cases

--
[...truncated 5552 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

javadoc-jar:
  [jar] Building jar: 

[jira] [Commented] (PIG-1827) When passing a parameter to Pig, if the value contains $ it has to be escaped for no apparent reason

[Julien Le Dem (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033285#comment-13033285
 ] 

Julien Le Dem commented on PIG-1827:


I created PIG-2074 to discuss the PreprocessorContext

> When passing a parameter to Pig, if the value contains $ it has to be escaped 
> for no apparent reason
> 
>
> Key: PIG-1827
> URL: https://issues.apache.org/jira/browse/PIG-1827
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Julien Le Dem
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1827-1.patch, PIG-1827_2.patch, PIG-1827_3.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2074) When passing a parameter to Pig, if the value contains $ it has to be escaped because of a bug in PrecprocessorContext

[Julien Le Dem (JIRA)]

When passing a parameter to Pig, if the value contains $ it has to be escaped 
because of a bug in PrecprocessorContext
--

 Key: PIG-2074
 URL: https://issues.apache.org/jira/browse/PIG-2074
 Project: Pig
  Issue Type: Bug
  Components: impl
Reporter: Julien Le Dem
Priority: Minor


This was raised while looking at PIG-1827

There seems to be a bug in PreprocessorContext:
http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/tools/parameters/PreprocessorContext.java?view=markup

{code}
235 //String litVal = Matcher.quoteReplacement(val);
236 replaced_line = replaced_line.replaceFirst("\\$"+key, val);
{code}

the replacement (2nd) parameter of replaceFirst is not a plain string, it can 
contain references to the matched pattern like "$0" so $ in val must be escaped.
Does someone know why line 235 is commented out ?


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2072) NPE when udf has project-star argument and input schema is null

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033271#comment-13033271
 ] 

Daniel Dai commented on PIG-2072:
-

+1

> NPE when udf has project-star argument and input schema is null
> ---
>
> Key: PIG-2072
> URL: https://issues.apache.org/jira/browse/PIG-2072
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2072.1.patch
>
>
> {code}
> grunt> l = load 'x' ; 
> grunt> f = foreach l generate CONCAT(*);
> 2011-05-13 12:04:48,355 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
> 2999: Unexpected internal error. null
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2067) FilterLogicExpressionSimplifier removed some branches in some cases

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-2067.
-

  Resolution: Fixed
Hadoop Flags: [Reviewed]

Patch committed to 0.8, 0.9, trunk.

> FilterLogicExpressionSimplifier removed some branches in some cases
> ---
>
> Key: PIG-2067
> URL: https://issues.apache.org/jira/browse/PIG-2067
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.9.0, 0.8.1
>
> Attachments: PIG-2067-1-0.8.patch, PIG-2067-1.patch
>
>
> The following script produce wrong result:
> {code}
> A = load 'a.dat' as (cookie);
> B = load 'b.dat' as (cookie);
> C = cogroup A by cookie, B by cookie;
> E = filter C by COUNT(B)>0 AND COUNT(A)>0;
> explain E;
> {code}
> a.dat:
> 1   1
> 2   2
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> b.dat:
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> 8   8
> Expected output:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> We get:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> (8,{},{(8)})

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2073) update union documentation - numeric types will not be converted to chararray

[Thejas M Nair (JIRA)]

update union documentation - numeric types will  not be converted to chararray
--

 Key: PIG-2073
 URL: https://issues.apache.org/jira/browse/PIG-2073
 Project: Pig
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.9.0
Reporter: Thejas M Nair
Assignee: Corinne Chandel
 Fix For: 0.9.0


As a result of changes in PIG-2071, the documentation needs to be updated -
{noformat}
 Union columns of compatible type will produce an "escalate" type. The priority 
is chararray > double > float > long > int > bytearray, tuple|bag|map > 
bytearray:
{noformat}

should be changed to remove chararray in the numeric type sequence. ie - 
{noformat}
 Union columns of compatible type will produce an "escalate" type. The priority 
is double > float > long > int > bytearray, tuple|bag|map|chararray > bytearray:
{noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2072) NPE when udf has project-star argument and input schema is null

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2072:
---

Attachment: PIG-2072.1.patch

> NPE when udf has project-star argument and input schema is null
> ---
>
> Key: PIG-2072
> URL: https://issues.apache.org/jira/browse/PIG-2072
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2072.1.patch
>
>
> {code}
> grunt> l = load 'x' ; 
> grunt> f = foreach l generate CONCAT(*);
> 2011-05-13 12:04:48,355 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
> 2999: Unexpected internal error. null
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2072) NPE when udf has project-star argument and input schema is null

[Thejas M Nair (JIRA)]

NPE when udf has project-star argument and input schema is null
---

 Key: PIG-2072
 URL: https://issues.apache.org/jira/browse/PIG-2072
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Fix For: 0.9.0


{code}
grunt> l = load 'x' ; 
grunt> f = foreach l generate CONCAT(*);
2011-05-13 12:04:48,355 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
2999: Unexpected internal error. null
{code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #811

[Apache Jenkins Server]

See 

Changes:

[xuefu] PIG-2059: PIG doesn't validate incomplete query in batch mode even if 
-c option is given

[xuefu] PIG-2062: Script silently ended

--
[...truncated 5560 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

[jira] [Updated] (PIG-2071) casting numeric type to chararray during schema merge for union is inconsistent with other schema merge cases

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2071:
---

Affects Version/s: 0.9.0
Fix Version/s: 0.9.0

> casting numeric type to chararray during schema merge for union is 
> inconsistent with other schema merge cases
> -
>
> Key: PIG-2071
> URL: https://issues.apache.org/jira/browse/PIG-2071
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
>
> In PIG-1536, if a column in output of union has a source column one input of 
> type charraray and in other it is of a numeric type, the column of numeric 
> type will be cast to chararray. But in all other cases, chararray and numeric 
> types are considered incompatible, and as a result such implicit casting is 
> not done.
> To be consistent with other cases, union also should consider chararray and 
> numeric types to be incompatible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2059) PIG doesn't validate incomplete query in batch mode even if -c option is given

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved PIG-2059.
--

Resolution: Fixed

patch PIG-2059-3.patch is committed to both trunk and 0.9.

> PIG doesn't validate incomplete query in batch mode even if -c option is given
> --
>
> Key: PIG-2059
> URL: https://issues.apache.org/jira/browse/PIG-2059
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2059-2.patch, PIG-2059-3.patch, PIG-2059.patch
>
>
> Given the following in a file to Pig, pig doesn't report any error, even if 
> -c option is given:
> A = load 'x' as (u, v);
> B = foreach A generate $3;
> It's questionable whether to validate the query in batch mode as it doesn't 
> contain any store/dump statement. However, if -c option is given, validation 
> should be nevertheless performed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2059) PIG doesn't validate incomplete query in batch mode even if -c option is given

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033212#comment-13033212
 ] 

Xuefu Zhang commented on PIG-2059:
--

For patch PIG-2059-3.patch, unit test passed. Test-patch run:

 [exec] +1 overall.
 [exec]
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec]
 [exec] +1 tests included.  The patch appears to include 24 new or 
modified tests.
 [exec]
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec]
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec]
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec]
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.


> PIG doesn't validate incomplete query in batch mode even if -c option is given
> --
>
> Key: PIG-2059
> URL: https://issues.apache.org/jira/browse/PIG-2059
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2059-2.patch, PIG-2059-3.patch, PIG-2059.patch
>
>
> Given the following in a file to Pig, pig doesn't report any error, even if 
> -c option is given:
> A = load 'x' as (u, v);
> B = foreach A generate $3;
> It's questionable whether to validate the query in batch mode as it doesn't 
> contain any store/dump statement. However, if -c option is given, validation 
> should be nevertheless performed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2067) FilterLogicExpressionSimplifier removed some branches in some cases

[jirapos...@reviews.apache.org (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033172#comment-13033172
 ] 

jirapos...@reviews.apache.org commented on PIG-2067:



---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/743/
---

(Updated 2011-05-13 18:13:38.714903)


Review request for pig and Richard Ding.


Summary
---

See PIG-2067


This addresses bug PIG-2067.
https://issues.apache.org/jira/browse/PIG-2067


Diffs
-

  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/LogicalExpressionSimplifier.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFilterSimplification.java
 1102169 

Diff: https://reviews.apache.org/r/743/diff


Testing (updated)
---

Test-patch:
 [exec] -1 overall.  
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 3 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec] 
 [exec] -1 release audit.  The applied patch generated 562 release 
audit warnings (more than the trunk's current 561 warnings).
Ignore release audit warning since no new file is added.

Unit test:
all pass

End-to-end test:
all pass


Thanks,

Daniel



> FilterLogicExpressionSimplifier removed some branches in some cases
> ---
>
> Key: PIG-2067
> URL: https://issues.apache.org/jira/browse/PIG-2067
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.8.1, 0.9.0
>
> Attachments: PIG-2067-1-0.8.patch, PIG-2067-1.patch
>
>
> The following script produce wrong result:
> {code}
> A = load 'a.dat' as (cookie);
> B = load 'b.dat' as (cookie);
> C = cogroup A by cookie, B by cookie;
> E = filter C by COUNT(B)>0 AND COUNT(A)>0;
> explain E;
> {code}
> a.dat:
> 1   1
> 2   2
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> b.dat:
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> 8   8
> Expected output:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> We get:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> (8,{},{(8)})

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2062) Script silently ended

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved PIG-2062.
--

Resolution: Fixed

Patch is committed to both trunk and 0.9.

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: FilterLogicExpressionSimplifier removed some branches in some cases

[Daniel Dai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/743/
---

(Updated 2011-05-13 18:13:38.714903)


Review request for pig and Richard Ding.


Summary
---

See PIG-2067


This addresses bug PIG-2067.
https://issues.apache.org/jira/browse/PIG-2067


Diffs
-

  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/LogicalExpressionSimplifier.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFilterSimplification.java
 1102169 

Diff: https://reviews.apache.org/r/743/diff


Testing (updated)
---

Test-patch:
 [exec] -1 overall.  
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 3 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec] 
 [exec] -1 release audit.  The applied patch generated 562 release 
audit warnings (more than the trunk's current 561 warnings).
Ignore release audit warning since no new file is added.

Unit test:
all pass

End-to-end test:
all pass


Thanks,

Daniel

[jira] [Created] (PIG-2071) casting numeric type to chararray during schema merge for union is inconsistent with other schema merge cases

[Thejas M Nair (JIRA)]

casting numeric type to chararray during schema merge for union is inconsistent 
with other schema merge cases
-

 Key: PIG-2071
 URL: https://issues.apache.org/jira/browse/PIG-2071
 Project: Pig
  Issue Type: Bug
Reporter: Thejas M Nair
Assignee: Thejas M Nair


In PIG-1536, if a column in output of union has a source column one input of 
type charraray and in other it is of a numeric type, the column of numeric type 
will be cast to chararray. But in all other cases, chararray and numeric types 
are considered incompatible, and as a result such implicit casting is not done.
To be consistent with other cases, union also should consider chararray and 
numeric types to be incompatible.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2062) Script silently ended

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033165#comment-13033165
 ] 

Xuefu Zhang commented on PIG-2062:
--

Unit test passed. Test-patch run:

 [exec] +1 overall.
 [exec]
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec]
 [exec] +1 tests included.  The patch appears to include 12 new or 
modified tests.
 [exec]
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec]
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec]
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec]
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.


> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2070) "Unknown" appears in error message for an error case

[Xuefu Zhang (JIRA)]

"Unknown" appears in error message for an error case


 Key: PIG-2070
 URL: https://issues.apache.org/jira/browse/PIG-2070
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Xuefu Zhang
Assignee: Thejas M Nair
 Fix For: 0.9.0


For the following query:

a = load '1.txt' as (a0:int, a1:int);
b = load '2.txt' as (a0:int, a1:chararray);
c = cogroup a by (a0,a1), b by (a0,a1);

Pig gives the following message, which includes "unknown" word. 

2011-05-13 11:01:18,682 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
1051:
 Cannot cast to Unknown

The error message should be more meaningful.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #810

[Apache Jenkins Server]

See 

Changes:

[rding] PIG-1827: When passing a parameter to Pig, if the value contains $ it 
has to be escaped for no apparent reason

--
[...truncated 5560 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

javadoc-jar:
  

[jira] [Commented] (PIG-2062) Script silently ended

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033161#comment-13033161
 ] 

Xuefu Zhang commented on PIG-2062:
--

Well, while not perfect, but I think it's good enough. If you really want to 
say something like "'all' is a reserved keyword', then you will need to 
customize the error message for each keyword, which I don't think worthwhile. 
We had such discussions before. 

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2067) FilterLogicExpressionSimplifier removed some branches in some cases

[Richard Ding (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033151#comment-13033151
 ] 

Richard Ding commented on PIG-2067:
---

+1

> FilterLogicExpressionSimplifier removed some branches in some cases
> ---
>
> Key: PIG-2067
> URL: https://issues.apache.org/jira/browse/PIG-2067
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.8.1, 0.9.0
>
> Attachments: PIG-2067-1-0.8.patch, PIG-2067-1.patch
>
>
> The following script produce wrong result:
> {code}
> A = load 'a.dat' as (cookie);
> B = load 'b.dat' as (cookie);
> C = cogroup A by cookie, B by cookie;
> E = filter C by COUNT(B)>0 AND COUNT(A)>0;
> explain E;
> {code}
> a.dat:
> 1   1
> 2   2
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> b.dat:
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> 8   8
> Expected output:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> We get:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> (8,{},{(8)})

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2062) Script silently ended

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033149#comment-13033149
 ] 

Daniel Dai commented on PIG-2062:
-

Now it throws exception. The error message is:
ERROR 1200:   mismatched input 'all' expecting 
EOF
Seems still not obvious. 

Can you give some hint of how to improve error message?

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1946) HBaseStorage constructor syntax is error prone

[Bill Graham (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033148#comment-13033148
 ] 

Bill Graham commented on PIG-1946:
--

The column descriptors take anything I can throw at them:
{code}
hbase(main):001:0>  create 't1', {NAME => 'f1', VERSIONS => 5}  
   
0 row(s) in 0.6400 seconds

hbase(main):002:0> put 't1', 'r1', 'f1:!@#$%)(:+_-=\][{}|;:"><,./?`~', 'value'  
   
0 row(s) in 0.0660 seconds
{code}

I'm also able to create column families with both '/' and '\' in them. Any 
suggestions for a valid encoding scheme?

> HBaseStorage constructor syntax is error prone
> --
>
> Key: PIG-1946
> URL: https://issues.apache.org/jira/browse/PIG-1946
> Project: Pig
>  Issue Type: Improvement
>Reporter: Bill Graham
>Assignee: Bill Graham
> Fix For: 0.10
>
> Attachments: PIG-1946_1.patch
>
>
> Using {{HBaseStorage}} like so seems like a reasonable thing to do, but it 
> will yield unexpected results:
> {code}
> STORE result INTO 'hbase://foo' USING
>  org.apache.pig.backend.hadoop.hbase.HBaseStorage(
>  'info:first_name, info:last_name');
> {code}
> The problem us that a column named {{info:first_name,}} will be created, with 
> the trailing comma included. I've had numerous developers get tripped up on 
> this issue since everywhere else in Pig variables are separated by commas, so 
> I propose we fix it.
> I propose we trim leading/trailing commas from column names, but I'm open to 
> other ideas.
> Also should we accept column names that are comman-delimited without spaces?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2059) PIG doesn't validate incomplete query in batch mode even if -c option is given

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated PIG-2059:
-

Attachment: PIG-2059-3.patch

Same code change, but modified additional test cases to make them pass.

> PIG doesn't validate incomplete query in batch mode even if -c option is given
> --
>
> Key: PIG-2059
> URL: https://issues.apache.org/jira/browse/PIG-2059
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Xuefu Zhang
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2059-2.patch, PIG-2059-3.patch, PIG-2059.patch
>
>
> Given the following in a file to Pig, pig doesn't report any error, even if 
> -c option is given:
> A = load 'x' as (u, v);
> B = foreach A generate $3;
> It's questionable whether to validate the query in batch mode as it doesn't 
> contain any store/dump statement. However, if -c option is given, validation 
> should be nevertheless performed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2062) Script silently ended

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033145#comment-13033145
 ] 

Thejas M Nair commented on PIG-2062:


+1

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1827) When passing a parameter to Pig, if the value contains $ it has to be escaped for no apparent reason

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-1827:
--

  Resolution: Fixed
Hadoop Flags: [Reviewed]
  Status: Resolved  (was: Patch Available)

Patch committed to trunk and 0.9 branch.

> When passing a parameter to Pig, if the value contains $ it has to be escaped 
> for no apparent reason
> 
>
> Key: PIG-1827
> URL: https://issues.apache.org/jira/browse/PIG-1827
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Julien Le Dem
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1827-1.patch, PIG-1827_2.patch, PIG-1827_3.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-1819) For implicit binding, Jython embedded Pig should skip any variable/value that contains $.

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding resolved PIG-1819.
---

Resolution: Fixed

This is fixed per PIG-1827.

> For implicit binding, Jython embedded Pig should skip any variable/value that 
> contains $. 
> --
>
> Key: PIG-1819
> URL: https://issues.apache.org/jira/browse/PIG-1819
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1819.patch, PIG-1819_1.patch, PIG-1819_2.patch
>
>
> We use the Pig parameter substitution for the bindings so variable/value that 
> contains $ cannot be used.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2062) Script silently ended

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033136#comment-13033136
 ] 

Xuefu Zhang commented on PIG-2062:
--

Related to PIG-2003, which is fixed, but it appears that we need a more general 
fix.

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2062) Script silently ended

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated PIG-2062:
-

Attachment: PIG-2062.patch

> Script silently ended
> -
>
> Key: PIG-2062
> URL: https://issues.apache.org/jira/browse/PIG-2062
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-2062.patch
>
>
> The following script ended silently without execution.
> {code}
> a = load '1.txt' as (a0, a1);
> b = load '2.txt' as (b0, b1);
> all = join a by a0, b by b0;
> store all into '';
> {code}
> If change the alias "all", it will run. We need to throw exception saying 
> "all" is a keyword.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2067) FilterLogicExpressionSimplifier removed some branches in some cases

[jirapos...@reviews.apache.org (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033127#comment-13033127
 ] 

jirapos...@reviews.apache.org commented on PIG-2067:



---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/743/
---

Review request for pig and Richard Ding.


Summary
---

See PIG-2067


This addresses bug PIG-2067.
https://issues.apache.org/jira/browse/PIG-2067


Diffs
-

  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/LogicalExpressionSimplifier.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFilterSimplification.java
 1102169 

Diff: https://reviews.apache.org/r/743/diff


Testing
---

Unit test:
all pass

End-to-end test:
all pass


Thanks,

Daniel



> FilterLogicExpressionSimplifier removed some branches in some cases
> ---
>
> Key: PIG-2067
> URL: https://issues.apache.org/jira/browse/PIG-2067
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.8.1, 0.9.0
>
> Attachments: PIG-2067-1-0.8.patch, PIG-2067-1.patch
>
>
> The following script produce wrong result:
> {code}
> A = load 'a.dat' as (cookie);
> B = load 'b.dat' as (cookie);
> C = cogroup A by cookie, B by cookie;
> E = filter C by COUNT(B)>0 AND COUNT(A)>0;
> explain E;
> {code}
> a.dat:
> 1   1
> 2   2
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> b.dat:
> 3   3
> 4   4
> 5   5
> 6   6
> 7   7
> 8   8
> Expected output:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> We get:
> (3,{(3)},{(3)})
> (4,{(4)},{(4)})
> (5,{(5)},{(5)})
> (6,{(6)},{(6)})
> (7,{(7)},{(7)})
> (8,{},{(8)})

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: FilterLogicExpressionSimplifier removed some branches in some cases

[Daniel Dai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/743/
---

Review request for pig and Richard Ding.


Summary
---

See PIG-2067


This addresses bug PIG-2067.
https://issues.apache.org/jira/browse/PIG-2067


Diffs
-

  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/LogicalExpressionSimplifier.java
 1102169 
  
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFilterSimplification.java
 1102169 

Diff: https://reviews.apache.org/r/743/diff


Testing
---

Unit test:
all pass

End-to-end test:
all pass


Thanks,

Daniel

[jira] [Assigned] (PIG-2069) LoadFunc jar does not ship to backend in MultiQuery case

[Olga Natkovich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olga Natkovich reassigned PIG-2069:
---

Assignee: Richard Ding

> LoadFunc jar does not ship to backend in MultiQuery case
> 
>
> Key: PIG-2069
> URL: https://issues.apache.org/jira/browse/PIG-2069
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.9.0
>
>
> Pig is able to automatically figure out the jar containing the LoadFunc and 
> ship them to backend. However, the following script didn't:
> {code}
> A = load '1.txt' using SomeLoadFunc();
> B = filter A by $0==0;
> C = filter A by $1==1;
> D = join B by $0, C by $0;
> dump D;
> {code}
> The reason is this query is a multiquery (A is reused and thus create an 
> implicit split). When we merge multiquery into one job, we didn't merge udfs 
> list properly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #809

[Apache Jenkins Server]

See 

Changes:

[thejas] PIG-1938: support project-range as udf argument - fixing test case

--
[...truncated 5568 lines...]
  [javadoc] Loading source files for package 
org.apache.pig.backend.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.executionengine...
  [javadoc] Loading source files for package org.apache.pig.backend.hadoop...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.datastorage...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.regex...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.executionengine.util...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.hbase...
  [javadoc] Loading source files for package 
org.apache.pig.backend.hadoop.streaming...
  [javadoc] Loading source files for package org.apache.pig.builtin...
  [javadoc] Loading source files for package org.apache.pig.classification...
  [javadoc] Loading source files for package org.apache.pig.data...
  [javadoc] Loading source files for package org.apache.pig.impl...
  [javadoc] Loading source files for package org.apache.pig.impl.builtin...
  [javadoc] Loading source files for package org.apache.pig.impl.io...
  [javadoc] Loading source files for package org.apache.pig.impl.logicalLayer...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.schema...
  [javadoc] Loading source files for package 
org.apache.pig.impl.logicalLayer.validators...
  [javadoc] Loading source files for package org.apache.pig.impl.plan...
  [javadoc] Loading source files for package 
org.apache.pig.impl.plan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.impl.streaming...
  [javadoc] Loading source files for package org.apache.pig.impl.util...
  [javadoc] Loading source files for package org.apache.pig.newplan...
  [javadoc] Loading source files for package org.apache.pig.newplan.logical...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.expression...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.optimizer...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.relational...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.rules...
  [javadoc] Loading source files for package 
org.apache.pig.newplan.logical.visitor...
  [javadoc] Loading source files for package org.apache.pig.newplan.optimizer...
  [javadoc] Loading source files for package org.apache.pig.parser...
  [javadoc] Loading source files for package org.apache.pig.pen...
  [javadoc] Loading source files for package org.apache.pig.pen.util...
  [javadoc] Loading source files for package org.apache.pig.scripting...
  [javadoc] Loading source files for package org.apache.pig.scripting.js...
  [javadoc] Loading source files for package org.apache.pig.scripting.jython...
  [javadoc] Loading source files for package org.apache.pig.tools...
  [javadoc] Loading source files for package org.apache.pig.tools.cmdline...
  [javadoc] Loading source files for package org.apache.pig.tools.grunt...
  [javadoc] Loading source files for package org.apache.pig.tools.parameters...
  [javadoc] Loading source files for package org.apache.pig.tools.pigstats...
  [javadoc] Loading source files for package org.apache.pig.tools.streams...
  [javadoc] Loading source files for package org.apache.pig.tools.timer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_23
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...

javadoc-jar:
  [jar] Building jar: 

[jira] [Commented] (PIG-2069) LoadFunc jar does not ship to backend in MultiQuery case

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032875#comment-13032875
 ] 

Daniel Dai commented on PIG-2069:
-

This is a general affecting all those versions.

> LoadFunc jar does not ship to backend in MultiQuery case
> 
>
> Key: PIG-2069
> URL: https://issues.apache.org/jira/browse/PIG-2069
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Daniel Dai
> Fix For: 0.9.0
>
>
> Pig is able to automatically figure out the jar containing the LoadFunc and 
> ship them to backend. However, the following script didn't:
> {code}
> A = load '1.txt' using SomeLoadFunc();
> B = filter A by $0==0;
> C = filter A by $1==1;
> D = join B by $0, C by $0;
> dump D;
> {code}
> The reason is this query is a multiquery (A is reused and thus create an 
> implicit split). When we merge multiquery into one job, we didn't merge udfs 
> list properly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1827) When passing a parameter to Pig, if the value contains $ it has to be escaped for no apparent reason

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032870#comment-13032870
 ] 

Daniel Dai commented on PIG-1827:
-

+1

> When passing a parameter to Pig, if the value contains $ it has to be escaped 
> for no apparent reason
> 
>
> Key: PIG-1827
> URL: https://issues.apache.org/jira/browse/PIG-1827
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Julien Le Dem
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1827-1.patch, PIG-1827_2.patch, PIG-1827_3.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira