[jira] Commented: (PIG-1405) Need to move many standard functions from piggybank into Pig

2010-06-17 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880049#action_12880049
 ] 

Hadoop QA commented on PIG-1405:


-1 overall.  Here are the results of testing the latest attachment 
  
http://issues.apache.org/jira/secure/attachment/12447381/StandardUDFtoPig3.patch
  against trunk revision 955701.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 5 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated 1 warning messages.

-1 javac.  The applied patch generated 146 javac compiler warnings (more 
than the trunk's current 138 warnings).

-1 findbugs.  The patch appears to introduce 2 new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

-1 contrib tests.  The patch failed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/330/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/330/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/330/console

This message is automatically generated.

> Need to move many standard functions from piggybank into Pig
> 
>
> Key: PIG-1405
> URL: https://issues.apache.org/jira/browse/PIG-1405
> Project: Pig
>  Issue Type: Improvement
>Reporter: Alan Gates
>Assignee: Aniket Mokashi
> Fix For: 0.8.0
>
> Attachments: StandardUDFtoPig.patch, StandardUDFtoPig3.patch
>
>
> There are currently a number of functions in Piggybank that represent 
> features commonly supported by languages and database engines.  We need to 
> decide which of these Pig should support as built in functions and put them 
> in org.apache.pig.builtin.  This will also mean adding unit tests and 
> javadocs for some UDFs.  The existing classes will be left in Piggybank for 
> some time for backward compatibility.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1221) Filter equality does not work for tuples

2010-06-17 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880035#action_12880035
 ] 

Jeff Zhang commented on PIG-1221:
-

I checked the test log, the errors from contrib tests do not related with this 
patch.


> Filter equality does not work for tuples
> 
>
> Key: PIG-1221
> URL: https://issues.apache.org/jira/browse/PIG-1221
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
> Environment: Windows and Linux. Java 1.6 hadoop 0.20.1
>Reporter: Neil Blue
>Assignee: Jeff Zhang
> Fix For: 0.8.0
>
> Attachments: PIG_1221.patch
>
>
> From the documentation I understand that it should be possible to  filter a 
> relation based on the equality of tuples. 
> http://wiki.apache.org/pig/PigTypesFunctionalSpec , 
> http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#deref:
>  However with this data file
> -- indext.txt:
> (1,one) (1,ONE)
> (2,two) (22, twentytwo)
> (3,three)   (3,three)
> I run this pig script:
> A = LOAD 'indext.txt' AS (t1:(a:int, b:chararray), t2:(a:int, b:chararray)); 
> B = FILTER A BY t1==t2; DUMP B;
> Expecting the output:
> ((3,three),(3,three))
> However there is an error:
> 2010-02-03 09:05:20,523 [main] ERROR org.apache.pig.tools.grunt.Grunt 
> - ERROR 2067: EqualToExpr does not know how to handle type: tuple
> > Pig Stack Trace
> > ---
> > ERROR 2067: EqualToExpr does not know how to handle type: tuple
> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:  
> > Unable to
> > open iterator for alias B
> >at org.apache.pig.PigServer.openIterator(PigServer.java:475)
> >at
> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java: 
> > 532)
> >at
> > org
> > .apache
> > .pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.
> > java:190)
> >at
> > org
> > .apache
> > .pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166
> > )
> >at
> > org
> > .apache
> > .pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:142
> > )
> >at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> >at org.apache.pig.Main.main(Main.java:397)
> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> > 1002:
> > Unable to store alias B
> >at org.apache.pig.PigServer.store(PigServer.java:530)
> >at org.apache.pig.PigServer.openIterator(PigServer.java:458)
> >... 6 more
> > Caused by: org.apache.pig.backend.executionengine.ExecException:  
> > ERROR 2067:
> > EqualToExpr does not know how to handle type: tuple
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
> > ors.EqualToExpr.getNext(EqualToExpr.java:108)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
> > ors.POFilter.getNext(POFilter.java:148)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
> > .processInput(PhysicalOperator.java:231)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.physicalLayer.counters.POCounte
> > r.getNext(POCounter.java:71)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
> > .processInput(PhysicalOperator.java:231)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
> > ors.POStore.getNext(POStore.java:117)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.LocalPigLauncher.runPipeline(Lo
> > calPigLauncher.java:146)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.LocalPigLauncher.launchPig(Loca
> > lPigLauncher.java:109)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.LocalExecutionEngine.execute(Lo
> > calExecutionEngine.java:165)
> Thanks
> Neil

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1453) [zebra] Intermittent failure for TestOrderPreserveUnionHDFS

2010-06-17 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880025#action_12880025
 ] 

Hadoop QA commented on PIG-1453:


-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12447373/PIG-1453.patch
  against trunk revision 955701.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 36 new or modified tests.

-1 javadoc.  The javadoc tool appears to have generated 1 warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

-1 contrib tests.  The patch failed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/341/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/341/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/341/console

This message is automatically generated.

> [zebra] Intermittent failure for TestOrderPreserveUnionHDFS
> ---
>
> Key: PIG-1453
> URL: https://issues.apache.org/jira/browse/PIG-1453
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Yan Zhou
> Fix For: 0.8.0
>
> Attachments: PIG-1453.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Created: (PIG-1458) aggregate files for replicated join

2010-06-17 Thread Olga Natkovich (JIRA)
aggregate files for replicated join
---

 Key: PIG-1458
 URL: https://issues.apache.org/jira/browse/PIG-1458
 Project: Pig
  Issue Type: Improvement
Reporter: Olga Natkovich


We have noticed that if the smaller data in replicated join has many files, 
this puts  unneeded burden on the name node. pre-aggregating the files can 
improve the situation

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Resolved: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Richard Ding (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding resolved PIG-1456.
---

Hadoop Flags: [Reviewed]
  Resolution: Fixed

> TestMultiQuery takes a long time to run
> ---
>
> Key: PIG-1456
> URL: https://issues.apache.org/jira/browse/PIG-1456
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1456.patch
>
>
> We see couple of times TestMultiQuery run out of time. Currently unit test 
> timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
> usually. It is very likely TestMultiQuery run out of time. We need to either 
> refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Richard Ding (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-1456:
--

Status: Open  (was: Patch Available)

> TestMultiQuery takes a long time to run
> ---
>
> Key: PIG-1456
> URL: https://issues.apache.org/jira/browse/PIG-1456
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1456.patch
>
>
> We see couple of times TestMultiQuery run out of time. Currently unit test 
> timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
> usually. It is very likely TestMultiQuery run out of time. We need to either 
> refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Eli Collins (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879940#action_12879940
 ] 

Eli Collins commented on PIG-1450:
--

I confirmed this patch fixed the test failures. Thanks Daniel.


> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: PIG-1450.patch, 
> TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879928#action_12879928
 ] 

Daniel Dai commented on PIG-1456:
-

+1

> TestMultiQuery takes a long time to run
> ---
>
> Key: PIG-1456
> URL: https://issues.apache.org/jira/browse/PIG-1456
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1456.patch
>
>
> We see couple of times TestMultiQuery run out of time. Currently unit test 
> timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
> usually. It is very likely TestMultiQuery run out of time. We need to either 
> refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Resolved: (PIG-1457) Pig will run complete zebra test even we give -Dtestcase=xxx

2010-06-17 Thread Daniel Dai (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-1457.
-

Hadoop Flags: [Reviewed]
  Resolution: Fixed

Patch committed.

> Pig will run complete zebra test even we give -Dtestcase=xxx
> 
>
> Key: PIG-1457
> URL: https://issues.apache.org/jira/browse/PIG-1457
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1457.patch
>
>
> After [PIG-1302|https://issues.apache.org/jira/browse/PIG-1302], even we want 
> to run an individual test using -Dtestcase=, pig will still invoke 
> complete zebra test. We shall pass -Dtestcase to zebra pigtest to suppress 
> running unwantted test.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Richard Ding (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-1456:
--

Attachment: PIG-1456.patch

This patch puts  multi-query tests into three files.

> TestMultiQuery takes a long time to run
> ---
>
> Key: PIG-1456
> URL: https://issues.apache.org/jira/browse/PIG-1456
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1456.patch
>
>
> We see couple of times TestMultiQuery run out of time. Currently unit test 
> timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
> usually. It is very likely TestMultiQuery run out of time. We need to either 
> refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Richard Ding (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-1456:
--

Status: Patch Available  (was: Open)

> TestMultiQuery takes a long time to run
> ---
>
> Key: PIG-1456
> URL: https://issues.apache.org/jira/browse/PIG-1456
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1456.patch
>
>
> We see couple of times TestMultiQuery run out of time. Currently unit test 
> timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
> usually. It is very likely TestMultiQuery run out of time. We need to either 
> refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1457) Pig will run complete zebra test even we give -Dtestcase=xxx

2010-06-17 Thread Pradeep Kamath (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879922#action_12879922
 ] 

Pradeep Kamath commented on PIG-1457:
-

+1

> Pig will run complete zebra test even we give -Dtestcase=xxx
> 
>
> Key: PIG-1457
> URL: https://issues.apache.org/jira/browse/PIG-1457
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1457.patch
>
>
> After [PIG-1302|https://issues.apache.org/jira/browse/PIG-1302], even we want 
> to run an individual test using -Dtestcase=, pig will still invoke 
> complete zebra test. We shall pass -Dtestcase to zebra pigtest to suppress 
> running unwantted test.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1295) Binary comparator for secondary sort

2010-06-17 Thread Gianmarco De Francisci Morales (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-1295:


Attachment: PIG-1295_0.4.patch

I modified the main in order to include your code snippet. Thanks very much for 
the suggestion!

I reduced the number of tuples in order to avoid OutOfMemory exceptions on my 
machine (only 1 GiB of RAM). I see the same numbers you reported: 6.5 times 
improve in speed only for the compare() method.

I took care of the tabs issue.

I will add support for ByteArrays and CharArrays next, together with fallback 
behaviour for complex datatypes.

> Binary comparator for secondary sort
> 
>
> Key: PIG-1295
> URL: https://issues.apache.org/jira/browse/PIG-1295
> Project: Pig
>  Issue Type: Improvement
>  Components: impl
>Affects Versions: 0.7.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: PIG-1295_0.1.patch, PIG-1295_0.2.patch, 
> PIG-1295_0.3.patch, PIG-1295_0.4.patch
>
>
> When hadoop framework doing the sorting, it will try to use binary version of 
> comparator if available. The benefit of binary comparator is we do not need 
> to instantiate the object before we compare. We see a ~30% speedup after we 
> switch to binary comparator. Currently, Pig use binary comparator in 
> following case:
> 1. When semantics of order doesn't matter. For example, in distinct, we need 
> to do a sort in order to filter out duplicate values; however, we do not care 
> how comparator sort keys. Groupby also share this character. In this case, we 
> rely on hadoop's default binary comparator
> 2. Semantics of order matter, but the key is of simple type. In this case, we 
> have implementation for simple types, such as integer, long, float, 
> chararray, databytearray, string
> However, if the key is a tuple and the sort semantics matters, we do not have 
> a binary comparator implementation. This especially matters when we switch to 
> use secondary sort. In secondary sort, we convert the inner sort of nested 
> foreach into the secondary key and rely on hadoop to sorting on both main key 
> and secondary key. The sorting key will become a two items tuple. Since the 
> secondary key the sorting key of the nested foreach, so the sorting semantics 
> matters. It turns out we do not have binary comparator once we use secondary 
> sort, and we see a significant slow down.
> Binary comparator for tuple should be doable once we understand the binary 
> structure of the serialized tuple. We can focus on most common use cases 
> first, which is "group by" followed by a nested sort. In this case, we will 
> use secondary sort. Semantics of the first key does not matter but semantics 
> of secondary key matters. We need to identify the boundary of main key and 
> secondary key in the binary tuple buffer without instantiate tuple itself. 
> Then if the first key equals, we use a binary comparator to compare secondary 
> key. Secondary key can also be a complex data type, but for the first step, 
> we focus on simple secondary key, which is the most common use case.
> We mark this issue to be a candidate project for "Google summer of code 2010" 
> program. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1405) Need to move many standard functions from piggybank into Pig

2010-06-17 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879919#action_12879919
 ] 

Daniel Dai commented on PIG-1405:
-

+1, will commit once hudson pass.

> Need to move many standard functions from piggybank into Pig
> 
>
> Key: PIG-1405
> URL: https://issues.apache.org/jira/browse/PIG-1405
> Project: Pig
>  Issue Type: Improvement
>Reporter: Alan Gates
>Assignee: Aniket Mokashi
> Fix For: 0.8.0
>
> Attachments: StandardUDFtoPig.patch, StandardUDFtoPig3.patch
>
>
> There are currently a number of functions in Piggybank that represent 
> features commonly supported by languages and database engines.  We need to 
> decide which of these Pig should support as built in functions and put them 
> in org.apache.pig.builtin.  This will also mean adding unit tests and 
> javadocs for some UDFs.  The existing classes will be left in Piggybank for 
> some time for backward compatibility.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1405) Need to move many standard functions from piggybank into Pig

2010-06-17 Thread Aniket Mokashi (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi updated PIG-1405:


Attachment: StandardUDFtoPig3.patch

Added test cases for all the supported functions in TestBuiltin.java
Test Cases added--
Math functions are tested using reflection with java.lang.Math class.
String functions are tested with a sample string.
Stats and misc functions are tested with sample input.

> Need to move many standard functions from piggybank into Pig
> 
>
> Key: PIG-1405
> URL: https://issues.apache.org/jira/browse/PIG-1405
> Project: Pig
>  Issue Type: Improvement
>Reporter: Alan Gates
>Assignee: Aniket Mokashi
> Fix For: 0.8.0
>
> Attachments: StandardUDFtoPig.patch, StandardUDFtoPig3.patch
>
>
> There are currently a number of functions in Piggybank that represent 
> features commonly supported by languages and database engines.  We need to 
> decide which of these Pig should support as built in functions and put them 
> in org.apache.pig.builtin.  This will also mean adding unit tests and 
> javadocs for some UDFs.  The existing classes will be left in Piggybank for 
> some time for backward compatibility.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1405) Need to move many standard functions from piggybank into Pig

2010-06-17 Thread Aniket Mokashi (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aniket Mokashi updated PIG-1405:


Status: Patch Available  (was: Open)

> Need to move many standard functions from piggybank into Pig
> 
>
> Key: PIG-1405
> URL: https://issues.apache.org/jira/browse/PIG-1405
> Project: Pig
>  Issue Type: Improvement
>Reporter: Alan Gates
>Assignee: Aniket Mokashi
> Fix For: 0.8.0
>
> Attachments: StandardUDFtoPig.patch, StandardUDFtoPig3.patch
>
>
> There are currently a number of functions in Piggybank that represent 
> features commonly supported by languages and database engines.  We need to 
> decide which of these Pig should support as built in functions and put them 
> in org.apache.pig.builtin.  This will also mean adding unit tests and 
> javadocs for some UDFs.  The existing classes will be left in Piggybank for 
> some time for backward compatibility.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1457) Pig will run complete zebra test even we give -Dtestcase=xxx

2010-06-17 Thread Daniel Dai (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-1457:


Attachment: PIG-1457.patch

> Pig will run complete zebra test even we give -Dtestcase=xxx
> 
>
> Key: PIG-1457
> URL: https://issues.apache.org/jira/browse/PIG-1457
> Project: Pig
>  Issue Type: Test
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1457.patch
>
>
> After [PIG-1302|https://issues.apache.org/jira/browse/PIG-1302], even we want 
> to run an individual test using -Dtestcase=, pig will still invoke 
> complete zebra test. We shall pass -Dtestcase to zebra pigtest to suppress 
> running unwantted test.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1453) [zebra] Intermittent failure for TestOrderPreserveUnionHDFS

2010-06-17 Thread Yan Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Zhou updated PIG-1453:
--

Status: Patch Available  (was: Open)

> [zebra] Intermittent failure for TestOrderPreserveUnionHDFS
> ---
>
> Key: PIG-1453
> URL: https://issues.apache.org/jira/browse/PIG-1453
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Yan Zhou
> Fix For: 0.8.0
>
> Attachments: PIG-1453.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1453) [zebra] Intermittent failure for TestOrderPreserveUnionHDFS

2010-06-17 Thread Yan Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Zhou updated PIG-1453:
--

Attachment: PIG-1453.patch

> [zebra] Intermittent failure for TestOrderPreserveUnionHDFS
> ---
>
> Key: PIG-1453
> URL: https://issues.apache.org/jira/browse/PIG-1453
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
>Reporter: Daniel Dai
>Assignee: Yan Zhou
> Fix For: 0.8.0
>
> Attachments: PIG-1453.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Resolved: (PIG-1455) [zebra] "test-unit" is needed as an ant target to unit test Zebra

2010-06-17 Thread Yan Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Zhou resolved PIG-1455.
---

Resolution: Fixed

Committed to the trunk and 0.6, 0.7 branches.

> [zebra] "test-unit"  is needed as an ant target to unit test Zebra
> --
>
> Key: PIG-1455
> URL: https://issues.apache.org/jira/browse/PIG-1455
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.6.0, 0.7.0, 0.8.0
>Reporter: Yan Zhou
>Assignee: Yan Zhou
>Priority: Minor
> Fix For: site, 0.8.0, 0.7.0, 0.6.0
>
> Attachments: PIG-1455.patch
>
>
> No "test-unit" ant target is in Zebra which is needed for  CI.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1428) Make a StatusReporter singleton available for incrementing counters

2010-06-17 Thread Dmitriy V. Ryaboy (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy V. Ryaboy updated PIG-1428:
---

Status: Resolved  (was: Patch Available)
Resolution: Fixed
  Tags: pig-0.7.1

Committed to trunk.
We may want to consider this for a 0.7.1, if such a thing comes about, as in a 
sense it's addressing a regression.

I tagged this issue with "pig-0.7.1" so we can find it later if we decide a 
dot-release is warranted.

> Make a StatusReporter singleton available for incrementing counters
> ---
>
> Key: PIG-1428
> URL: https://issues.apache.org/jira/browse/PIG-1428
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Ashutosh Chauhan
>Assignee: Dmitriy V. Ryaboy
> Fix For: 0.8.0
>
> Attachments: PIG-1428.patch, PIG-1428.patch, PIG-1428.patch
>
>
> Without this getter method, its not possible to get counters, report progress 
> etc. from UDFs. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Resolved: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Daniel Dai (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-1450.
-

Hadoop Flags: [Reviewed]
  Resolution: Fixed

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: PIG-1450.patch, 
> TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Richard Ding (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879874#action_12879874
 ] 

Richard Ding commented on PIG-1450:
---

+1 for commit. This should fix this OOM problem.

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: PIG-1450.patch, 
> TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Created: (PIG-1457) Pig will run complete zebra test even we give -Dtestcase=xxx

2010-06-17 Thread Daniel Dai (JIRA)
Pig will run complete zebra test even we give -Dtestcase=xxx


 Key: PIG-1457
 URL: https://issues.apache.org/jira/browse/PIG-1457
 Project: Pig
  Issue Type: Test
  Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
 Fix For: 0.8.0


After [PIG-1302|https://issues.apache.org/jira/browse/PIG-1302], even we want 
to run an individual test using -Dtestcase=, pig will still invoke complete 
zebra test. We shall pass -Dtestcase to zebra pigtest to suppress running 
unwantted test.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Daniel Dai (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-1450:


Attachment: PIG-1450.patch

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: PIG-1450.patch, 
> TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Created: (PIG-1456) TestMultiQuery takes a long time to run

2010-06-17 Thread Daniel Dai (JIRA)
TestMultiQuery takes a long time to run
---

 Key: PIG-1456
 URL: https://issues.apache.org/jira/browse/PIG-1456
 Project: Pig
  Issue Type: Test
  Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Richard Ding
 Fix For: 0.8.0


We see couple of times TestMultiQuery run out of time. Currently unit test 
timeout setting is 45 min, TestMultiQuery will take more than 40 min to run 
usually. It is very likely TestMultiQuery run out of time. We need to either 
refractory the test or break it into two.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1455) [zebra] "test-unit" is needed as an ant target to unit test Zebra

2010-06-17 Thread Olga Natkovich (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879852#action_12879852
 ] 

Olga Natkovich commented on PIG-1455:
-

It would be nice to re-factor the common code from different unit targets. You 
can see how this done in pig's top level build.xml.

I am fine if this is done afterwards in a separate ticket. So +1 for this patch

> [zebra] "test-unit"  is needed as an ant target to unit test Zebra
> --
>
> Key: PIG-1455
> URL: https://issues.apache.org/jira/browse/PIG-1455
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.6.0, 0.7.0, 0.8.0
>Reporter: Yan Zhou
>Assignee: Yan Zhou
>Priority: Minor
> Fix For: site, 0.6.0, 0.7.0, 0.8.0
>
> Attachments: PIG-1455.patch
>
>
> No "test-unit" ant target is in Zebra which is needed for  CI.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Daniel Dai (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879846#action_12879846
 ] 

Daniel Dai commented on PIG-1450:
-

Yes, I also see this error couple of times. Sounds like we should increase the 
heap space.

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879833#action_12879833
 ] 

Jeff Zhang commented on PIG-1450:
-

I believe so, hadoop's local mode need much more memory. One solution fo this 
problem is to increase the heap space for test case by setting the Xmx argument

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1450) TestAlgebraicEvalLocal failures due to OOM

2010-06-17 Thread Eli Collins (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879830#action_12879830
 ] 

Eli Collins commented on PIG-1450:
--

Could this be due to the switch to Hadoop's local mode?

> TestAlgebraicEvalLocal failures due to OOM
> --
>
> Key: PIG-1450
> URL: https://issues.apache.org/jira/browse/PIG-1450
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Eli Collins
> Fix For: 0.8.0
>
> Attachments: TEST-org.apache.pig.test.TestAlgebraicEvalLocal.txt
>
>
> 6 test cases in TestAlgebraicEvalLocal fail on trunk and release 0.7 across a 
> number of different machines.
> Example failure:
> Unable to open iterator for alias myid
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
> open iterator for alias myid
> at org.apache.pig.PigServer.openIterator(PigServer.java:521)
> at 
> org.apache.pig.test.TestAlgebraicEvalLocal.testGroupUniqueColumnCount(TestAlgebraicEvalLocal.java:236)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:515)
> Probably due to OOMs in the log:
> 10/06/14 19:38:43 WARN mapred.LocalJobRunner: job_local_0002
> java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
> at 
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1455) [zebra] "test-unit" is needed as an ant target to unit test Zebra

2010-06-17 Thread Yan Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Zhou updated PIG-1455:
--

Attachment: PIG-1455.patch

> [zebra] "test-unit"  is needed as an ant target to unit test Zebra
> --
>
> Key: PIG-1455
> URL: https://issues.apache.org/jira/browse/PIG-1455
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.6.0, 0.7.0, 0.8.0
>Reporter: Yan Zhou
>Assignee: Yan Zhou
>Priority: Minor
> Fix For: site, 0.6.0, 0.7.0, 0.8.0
>
> Attachments: PIG-1455.patch
>
>
> No "test-unit" ant target is in Zebra which is needed for  CI.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (PIG-1455) [zebra] "test-unit" is needed as an ant target to unit test Zebra

2010-06-17 Thread Yan Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/PIG-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Zhou updated PIG-1455:
--

Attachment: (was: PIG-1451.patch)

> [zebra] "test-unit"  is needed as an ant target to unit test Zebra
> --
>
> Key: PIG-1455
> URL: https://issues.apache.org/jira/browse/PIG-1455
> Project: Pig
>  Issue Type: Test
>Affects Versions: 0.6.0, 0.7.0, 0.8.0
>Reporter: Yan Zhou
>Assignee: Yan Zhou
>Priority: Minor
> Fix For: site, 0.6.0, 0.7.0, 0.8.0
>
>
> No "test-unit" ant target is in Zebra which is needed for  CI.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (PIG-1221) Filter equality does not work for tuples

2010-06-17 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PIG-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879765#action_12879765
 ] 

Hadoop QA commented on PIG-1221:


-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12447317/PIG_1221.patch
  against trunk revision 955028.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 6 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

-1 contrib tests.  The patch failed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/340/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/340/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/340/console

This message is automatically generated.

> Filter equality does not work for tuples
> 
>
> Key: PIG-1221
> URL: https://issues.apache.org/jira/browse/PIG-1221
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0
> Environment: Windows and Linux. Java 1.6 hadoop 0.20.1
>Reporter: Neil Blue
>Assignee: Jeff Zhang
> Fix For: 0.8.0
>
> Attachments: PIG_1221.patch
>
>
> From the documentation I understand that it should be possible to  filter a 
> relation based on the equality of tuples. 
> http://wiki.apache.org/pig/PigTypesFunctionalSpec , 
> http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#deref:
>  However with this data file
> -- indext.txt:
> (1,one) (1,ONE)
> (2,two) (22, twentytwo)
> (3,three)   (3,three)
> I run this pig script:
> A = LOAD 'indext.txt' AS (t1:(a:int, b:chararray), t2:(a:int, b:chararray)); 
> B = FILTER A BY t1==t2; DUMP B;
> Expecting the output:
> ((3,three),(3,three))
> However there is an error:
> 2010-02-03 09:05:20,523 [main] ERROR org.apache.pig.tools.grunt.Grunt 
> - ERROR 2067: EqualToExpr does not know how to handle type: tuple
> > Pig Stack Trace
> > ---
> > ERROR 2067: EqualToExpr does not know how to handle type: tuple
> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:  
> > Unable to
> > open iterator for alias B
> >at org.apache.pig.PigServer.openIterator(PigServer.java:475)
> >at
> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java: 
> > 532)
> >at
> > org
> > .apache
> > .pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.
> > java:190)
> >at
> > org
> > .apache
> > .pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166
> > )
> >at
> > org
> > .apache
> > .pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:142
> > )
> >at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> >at org.apache.pig.Main.main(Main.java:397)
> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> > 1002:
> > Unable to store alias B
> >at org.apache.pig.PigServer.store(PigServer.java:530)
> >at org.apache.pig.PigServer.openIterator(PigServer.java:458)
> >... 6 more
> > Caused by: org.apache.pig.backend.executionengine.ExecException:  
> > ERROR 2067:
> > EqualToExpr does not know how to handle type: tuple
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
> > ors.EqualToExpr.getNext(EqualToExpr.java:108)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
> > ors.POFilter.getNext(POFilter.java:148)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
> > .processInput(PhysicalOperator.java:231)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.physicalLayer.counters.POCounte
> > r.getNext(POCounter.java:71)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
> > .processInput(PhysicalOperator.java:231)
> >at
> > org
> > .apache
> > .pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
> > ors.POStore.getNext(POStore.java:117)
> >at
> > org
> > .apache
> > .pig.backend.local.executionengine.LocalPigLauncher.runPipeline(Lo
> > calPigLauncher.java:146)
> >at
> > org
> > .apache
>