[jira] Commented: (PIG-1022) optimizer pushes filter before the foreach that generates column used by filter

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768120#action_12768120
 ] 

Hadoop QA commented on PIG-1022:


-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12422754/PIG-1022-1.patch
  against trunk revision 827829.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/104/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/104/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/104/console

This message is automatically generated.

> optimizer pushes filter before the foreach that generates column used by 
> filter
> ---
>
> Key: PIG-1022
> URL: https://issues.apache.org/jira/browse/PIG-1022
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Thejas M Nair
>Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1022-1.patch
>
>
> grunt> l = load 'students.txt' using PigStorage() as (name:chararray, 
> gender:chararray, age:chararray, score:chararray);
> grunt> f = foreach l generate name, gender, age,score, '200'  as 
> gid:chararray;
> grunt> g = group f by (name, gid);
> grunt> f2 = foreach g generate group.name as name: chararray, group.gid as 
> gid: chararray;
> grunt> filt = filter f2 by gid == '200';
> grunt> explain filt;
> In the plan generated filt is pushed up after the load and before the first 
> foreach, even though the filter is on gid which is generated in first foreach.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1028) FINDBUGS: DM_NUMBER_CTOR: Method invokes inefficient Number constructor; use static valueOf instead

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768229#action_12768229
 ] 

Hadoop QA commented on PIG-1028:


-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12422755/PIG-1028.patch
  against trunk revision 827829.

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no tests are needed for this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/105/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/105/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/105/console

This message is automatically generated.

> FINDBUGS: DM_NUMBER_CTOR: Method invokes inefficient Number constructor; use 
> static valueOf instead
> ---
>
> Key: PIG-1028
> URL: https://issues.apache.org/jira/browse/PIG-1028
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
>Assignee: Olga Natkovich
> Attachments: PIG-1028.patch
>
>
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.getStatistics() 
> invokes inefficient new Long(long) constructor; use Long.valueOf(long) instead
> BxMethod org.apache.pig.backend.hadoop.datastorage.HDataStorage.init() 
> invokes inefficient new Short(short) constructor; use Short.valueOf(short) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HPath.getConfiguration() invokes 
> inefficient new Long(long) constructor; use Long.valueOf(long) instead
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HPath.getConfiguration() invokes 
> inefficient new Short(short) constructor; use Short.valueOf(short) instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.addShiftedKeyInfoIndex(int,
>  POPackage) invokes inefficient new Integer(int) constructor; use 
> Integer.valueOf(int) instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.POPackageAnnotator$LoRearrangeDiscoverer.visitLocalRearrange(POLocalRearrange)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Mod.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Mod.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Multiply.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Multiply.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use In

Build failed in Hudson: Pig-trunk #596

[Apache Hudson Server]

See 

Changes:

[olga] PIG-1012: FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable 
instance field in
serializable class (olgan)

--
[...truncated 2545 lines...]

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:

ivy-buildJar:
[ivy:resolve] :: resolving dependencies :: 
org.apache.pig#Pig;2009-10-21_13-30-15
[ivy:resolve]   confs: [buildJar]
[ivy:resolve]   found com.jcraft#jsch;0.1.38 in maven2
[ivy:resolve]   found jline#jline;0.9.94 in maven2
[ivy:resolve]   found net.java.dev.javacc#javacc;4.2 in maven2
[ivy:resolve]   found junit#junit;4.5 in default
[ivy:resolve] :: resolution report :: resolve 53ms :: artifacts dl 4ms
-
|  |modules||   artifacts   |
|   conf   | number| search|dwnlded|evicted|| number|dwnlded|
-
| buildJar |   4   |   0   |   0   |   0   ||   4   |   0   |
-
[ivy:retrieve] :: retrieving :: org.apache.pig#Pig
[ivy:retrieve]  confs: [buildJar]
[ivy:retrieve]  1 artifacts copied, 3 already retrieved (288kB/4ms)

buildJar:
 [echo] svnString 828008
  [jar] Building jar: 

 [copy] Copying 1 file to 


jarWithOutSvn:

findbugs:
[mkdir] Created dir: 

 [findbugs] Executing findbugs from ant task
 [findbugs] Running FindBugs...
 [findbugs] The following classes needed for analysis were missing:
 [findbugs]   com.jcraft.jsch.SocketFactory
 [findbugs]   com.jcraft.jsch.Logger
 [findbugs]   jline.Completor
 [findbugs]   com.jcraft.jsch.Session
 [findbugs]   com.jcraft.jsch.HostKeyRepository
 [findbugs]   com.jcraft.jsch.JSch
 [findbugs]   com.jcraft.jsch.UserInfo
 [findbugs]   jline.ConsoleReaderInputStream
 [findbugs]   com.jcraft.jsch.HostKey
 [findbugs]   jline.ConsoleReader
 [findbugs]   com.jcraft.jsch.ChannelExec
 [findbugs]   jline.History
 [findbugs]   com.jcraft.jsch.ChannelDirectTCPIP
 [findbugs]   com.jcraft.jsch.JSchException
 [findbugs]   com.jcraft.jsch.Channel
 [findbugs] Warnings generated: 299
 [findbugs] Missing classes: 16
 [findbugs] Calculating exit code...
 [findbugs] Setting 'missing class' flag (2)
 [findbugs] Setting 'bugs found' flag (1)
 [findbugs] Exit code set to: 3
 [findbugs] Java Result: 3
 [findbugs] Classes needed for analysis were missing
 [findbugs] Output saved to 

 [xslt] Processing 

 to 

 [xslt] Loading stylesheet 
/homes/gkesavan/tools/findbugs/latest/src/xsl/default.xsl

BUILD SUCCESSFUL
Total time: 2 minutes 47 seconds
+ mv build/pig-2009-10-21_13-30-15.tar.gz 

+ mv build/test/findbugs 

+ mv build/docs/api 

+ /homes/hudson/tools/ant/apache-ant-1.7.0/bin/ant clean
Buildfile: build.xml

clean:
   [delete] Deleting directory 

   [delete] Deleting directory 

   [delete] Deleting directory 

   [delete] Deleting directory 


BUILD SUCCESSFUL
Total time: 0 seconds
+ /homes/hudson/tools/ant/apache-ant-1.7.0/bin/ant 
-Dtest.junit.output.format=xml -Dtest.output=yes 
-Dcheckstyle.home=/homes/hudson/tools/checkstyle/latest -Drun.clover=true 
-Dclover.home=/homes/hudson/tools/clover/clover-ant-2.3.2 clover test 
generate-clover-reports
Buildfile: build.xml

clover.setup:
[mkdir] Created dir: 

[clover-setup] Clover Version 2.3.2, built on July 15 2008 (build-732)
[clover-setup] Loaded from: 
/homes/hudson/tools/clover/clover-ant-2.3.2/lib/clover.jar
[clover-setup] Clover: Open Source License registered to Apache Software 
Foundation.
[clover-setup] Clover is enabled with initstring 
'

[jira] Commented: (PIG-1027) Number of bytes written are always zero in local mode

[Jeff Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768233#action_12768233
 ] 

Jeff Zhang commented on PIG-1027:
-

I look the test report, It seems the failed test case is not related the patch.

Could anyone help look into it ?



> Number of bytes written are always zero in local mode
> -
>
> Key: PIG-1027
> URL: https://issues.apache.org/jira/browse/PIG-1027
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.6.0
>Reporter: Ashutosh Chauhan
>Assignee: Jeff Zhang
>Priority: Minor
> Attachments: Pig_1027.Patch
>
>
> Consider this very simple script containing few records
> {code}
> a = load 'foo';
> store a into 'out';
> {code}
> Following message gets printed on grunt shell:
> [main] INFO  org.apache.pig.backend.local.executionengine.LocalPigLauncher - 
> Records written : 39
> [main] INFO  org.apache.pig.backend.local.executionengine.LocalPigLauncher - 
> Bytes written : 0
> File has 39 records which is correctly reported. But number of bytes is 
> always reported as zero, no matter what.  I am observing this on latest 
> trunk, not sure if this existed on previous/current releases.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1027) Number of bytes written are always zero in local mode

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768237#action_12768237
 ] 

Alan Gates commented on PIG-1027:
-

There's something wrong with the test runs on Hudson.  We are looking into it.  
We'll get to running your patch through the tests manually, but it will take a 
bit as there's several in the queue and we have to run all the unit tests 
(about 3 hours) on each.

> Number of bytes written are always zero in local mode
> -
>
> Key: PIG-1027
> URL: https://issues.apache.org/jira/browse/PIG-1027
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.6.0
>Reporter: Ashutosh Chauhan
>Assignee: Jeff Zhang
>Priority: Minor
> Attachments: Pig_1027.Patch
>
>
> Consider this very simple script containing few records
> {code}
> a = load 'foo';
> store a into 'out';
> {code}
> Following message gets printed on grunt shell:
> [main] INFO  org.apache.pig.backend.local.executionengine.LocalPigLauncher - 
> Records written : 39
> [main] INFO  org.apache.pig.backend.local.executionengine.LocalPigLauncher - 
> Bytes written : 0
> File has 39 records which is correctly reported. But number of bytes is 
> always reported as zero, no matter what.  I am observing this on latest 
> trunk, not sure if this existed on previous/current releases.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-944) Zebra schema is taken from Pig through TableStorer's construct

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-944:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Patch checked in.

> Zebra schema is taken from Pig through TableStorer's construct
> --
>
> Key: PIG-944
> URL: https://issues.apache.org/jira/browse/PIG-944
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.4.0
>Reporter: Yan Zhou
>Assignee: Yan Zhou
> Fix For: 0.6.0
>
> Attachments: SchemaConversion.patch, SchemaConversion.patch
>
>
> It should be from StoreConfig in TableOutputFormat.checkOutputSpecs method 
> because the information is dynamic in Pig's execution engine and should not 
> be taking a static argument to the constructor.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1034) Pig does not support ORDER ... BY group alias

[David Ciemiewicz (JIRA)]

Pig does not support ORDER ... BY group alias
-

 Key: PIG-1034
 URL: https://issues.apache.org/jira/browse/PIG-1034
 Project: Pig
  Issue Type: Bug
Reporter: David Ciemiewicz


GROUP ... ALL and GROUP ... BY produce an alias "group".

Pig produces a syntax error if you attempt to ORDER ... BY group.

This does seem like a perfectly reasonable thing to do.

The workaround is to create an alias for group using an AS clause.  But I think 
this workaround should be unnecessary.

Here's sample code which elicits the syntax error:

{code}
A = load 'one.txt' using PigStorage as (one: int);

B = group A all;

C = foreach B generate
group,
COUNT(A) as count;

D = order C by group parallel 1; -- group is one of the aliases in C, why does 
this throw a syntax error?

dump D;
{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-790) Error message should indicate in which line number in the Pig script the error occured (debugging BinCond)

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768314#action_12768314
 ] 

Daniel Dai commented on PIG-790:


core tests pass manually

> Error message should indicate in which line number in the Pig script the 
> error occured (debugging BinCond)
> --
>
> Key: PIG-790
> URL: https://issues.apache.org/jira/browse/PIG-790
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.4.0
>Reporter: Viraj Bhat
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: error_rerport.pig, myerrordata.txt, PIG-790-1.patch, 
> pig_1240972895275.log
>
>
> I have a simple Pig script which loads integer data and does a Bincond, where 
> it compares, (col1 eq ''). There is an error message that is generated in 
> this case, but it does not specify the line number in the script. 
> {code}
> MYDATA = load '/user/viraj/myerrordata.txt' using PigStorage() as (col1:int, 
> col2:int);
> MYDATA_PROJECT = FOREACH MYDATA GENERATE ((col1 eq '') ? 1 : 0) as newcol1,
>  ((col1 neq '') ? col1 - col2 : 
> 16)
> as time_diff;
> dump MYDATA_PROJECT;
> {code}
> ==
> 2009-04-29 02:33:07,182 [main] INFO  
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting 
> to hadoop file system at: hdfs://localhost:9000
> 2009-04-29 02:33:08,584 [main] INFO  
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting 
> to map-reduce job tracker at: localhost:9001
> 2009-04-29 02:33:08,836 [main] INFO  org.apache.pig.PigServer - Create a new 
> graph.
> 2009-04-29 02:33:10,040 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
> 1039: Incompatible types in EqualTo Operator left hand side:int right hand 
> side:chararray
> Details at logfile: /home/viraj/pig-svn/trunk/pig_1240972386081.log
> ==
> It would be good if the error message has a line number and a copy of the 
> line in the script which is causing the problem.
> Attaching data, script and log file. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-996) [zebra] Zebra build script does not have findbugs and clover targets.

[Chao Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Wang updated PIG-996:
--

Status: Patch Available  (was: Open)

> [zebra] Zebra build script does not have findbugs and clover targets.
> -
>
> Key: PIG-996
> URL: https://issues.apache.org/jira/browse/PIG-996
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.4.0
>Reporter: Chao Wang
>Assignee: Chao Wang
> Fix For: 0.6.0
>
> Attachments: patch_build, patch_build
>
>
> Zebra build script does not have findbugs and clover targets, leading hudson 
> build process to fail on Zebra.
> This jira is to fix this by adding these two targets.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1022) optimizer pushes filter before the foreach that generates column used by filter

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768312#action_12768312
 ] 

Daniel Dai commented on PIG-1022:
-

core tests pass manually

> optimizer pushes filter before the foreach that generates column used by 
> filter
> ---
>
> Key: PIG-1022
> URL: https://issues.apache.org/jira/browse/PIG-1022
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Thejas M Nair
>Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1022-1.patch
>
>
> grunt> l = load 'students.txt' using PigStorage() as (name:chararray, 
> gender:chararray, age:chararray, score:chararray);
> grunt> f = foreach l generate name, gender, age,score, '200'  as 
> gid:chararray;
> grunt> g = group f by (name, gid);
> grunt> f2 = foreach g generate group.name as name: chararray, group.gid as 
> gid: chararray;
> grunt> filt = filter f2 by gid == '200';
> grunt> explain filt;
> In the plan generated filt is pushed up after the load and before the first 
> foreach, even though the filter is on gid which is generated in first foreach.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-976) Multi-query optimization throws ClassCastException

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-976:
-

Attachment: PIG-976_1.patch

This patch fixed a unit test error.

> Multi-query optimization throws ClassCastException
> --
>
> Key: PIG-976
> URL: https://issues.apache.org/jira/browse/PIG-976
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Ankur
>Assignee: Richard Ding
> Fix For: 0.6.0
>
> Attachments: PIG-976.patch, PIG-976.patch, PIG-976.patch, 
> PIG-976.patch, PIG-976.patch, PIG-976_1.patch
>
>
> Multi-query optimization fails to merge 2 branches when 1 is a result of 
> Group By ALL and another is a result of Group By field1 where field 1 is of 
> type long. Here is the script that fails with multi-query on.
> data = LOAD 'test' USING PigStorage('\t') AS (a:long, b:double, c:double); 
> A = GROUP data ALL;
> B = FOREACH A GENERATE SUM(data.b) AS sum1, SUM(data.c) AS sum2;
> C = FOREACH B GENERATE (sum1/sum2) AS rate; 
> STORE C INTO 'result1';
> D = GROUP data BY a; 
> E = FOREACH D GENERATE group AS a, SUM(data.b), SUM(data.c);
> STORE E into 'result2';
>  
> Here is the exception from the logs
> java.lang.ClassCastException: org.apache.pig.data.DefaultTuple cannot be cast 
> to org.apache.pig.data.DataBag
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:399)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:180)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.processInput(POUserFunc.java:145)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:197)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.runPipeline(PODemux.java:264)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.getNext(PODemux.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:196)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:174)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:63)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:906)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:786)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
>   at 
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2206)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768353#action_12768353
 ] 

Ashutosh Chauhan commented on PIG-1012:
---

We just looked at POFRJoin, this might be happening at other places as well.

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines non-transient non-seria

[jira] Reopened: (PIG-976) Multi-query optimization throws ClassCastException

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding reopened PIG-976:
--


> Multi-query optimization throws ClassCastException
> --
>
> Key: PIG-976
> URL: https://issues.apache.org/jira/browse/PIG-976
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Ankur
>Assignee: Richard Ding
> Fix For: 0.6.0
>
> Attachments: PIG-976.patch, PIG-976.patch, PIG-976.patch, 
> PIG-976.patch, PIG-976.patch, PIG-976_1.patch
>
>
> Multi-query optimization fails to merge 2 branches when 1 is a result of 
> Group By ALL and another is a result of Group By field1 where field 1 is of 
> type long. Here is the script that fails with multi-query on.
> data = LOAD 'test' USING PigStorage('\t') AS (a:long, b:double, c:double); 
> A = GROUP data ALL;
> B = FOREACH A GENERATE SUM(data.b) AS sum1, SUM(data.c) AS sum2;
> C = FOREACH B GENERATE (sum1/sum2) AS rate; 
> STORE C INTO 'result1';
> D = GROUP data BY a; 
> E = FOREACH D GENERATE group AS a, SUM(data.b), SUM(data.c);
> STORE E into 'result2';
>  
> Here is the exception from the logs
> java.lang.ClassCastException: org.apache.pig.data.DefaultTuple cannot be cast 
> to org.apache.pig.data.DataBag
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:399)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:180)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.processInput(POUserFunc.java:145)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:197)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.runPipeline(PODemux.java:264)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.getNext(PODemux.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:196)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:174)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:63)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:906)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:786)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
>   at 
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2206)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Reopened: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan reopened PIG-1012:
---


> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux

[jira] Commented: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768352#action_12768352
 ] 

Ashutosh Chauhan commented on PIG-1012:
---

Marking "log" in POFRJoin transient causes FRJoin to fail. Because at the 
backend it can't be deserialized and log.debug is used while building 
hashtables resulting in NPE. Either it shouldn't be marked transient or it 
should be instantiated in "readObject()" method.  

Stack Trace:

Pig Stack Trace
---
ERROR 2999: Unexpected internal error. null

java.lang.NullPointerException
at 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.setUpHashMap(POFRJoin.java:293)
at 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.getNext(POFRJoin.java:197)
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249)
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240)
at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)

Thanks to Tejal to pointing this out.

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> o

[jira] Updated: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-1012:


Attachment: PIG-1012-2.patch

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012-2.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.phys

[jira] Updated: (PIG-1025) Should be able to set job priority through Pig Latin

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-1025:


Attachment: TEST-org.apache.pig.test.TestFRJoin.txt

Kevin,

I'm still seeing failures in TestFRJoin that look related.  I've attached the 
logs from my test run.

> Should be able to set job priority through Pig Latin
> 
>
> Key: PIG-1025
> URL: https://issues.apache.org/jira/browse/PIG-1025
> Project: Pig
>  Issue Type: New Feature
>  Components: grunt
>Affects Versions: 0.4.0
>Reporter: Kevin Weil
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1025.patch, PIG-1025_2.patch, 
> TEST-org.apache.pig.test.TestFRJoin.txt
>
>
> Currently users can set the job name through Pig Latin by saying
> set job.name 'my job name'
> The ability to set the priority would also be nice, and the patch should be 
> small.  The goal is to be able to say
> set job.priority 'high'
> and throw a JobCreationException in the JobControlCompiler if the priority is 
> not one of the allowed string values from the o.a.h.mapred.JobPriority enum: 
> very_low, low, normal, high, very_high.   Case insensitivity makes this a 
> little nicer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Olga Natkovich (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768364#action_12768364
 ] 

Olga Natkovich commented on PIG-1012:
-

+1; I don't think we need to rerun tests for this change. Thanks. Daniel for 
figuring this out!

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012-2.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage

[jira] Commented: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Santhosh Srinivasan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768368#action_12768368
 ] 

Santhosh Srinivasan commented on PIG-1012:
--

I just looked at the first patch. It was setting generate to true in 
TestMRCompiler.java It should be set to false in order to run the test case 
correctly.

+++ test/org/apache/pig/test/TestMRCompiler.java

-private boolean generate = false;
+private boolean generate = true;

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012-2.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressi

[jira] Commented: (PIG-1025) Should be able to set job priority through Pig Latin

[Dmitriy V. Ryaboy (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768370#action_12768370
 ] 

Dmitriy V. Ryaboy commented on PIG-1025:


Sounds like Ashutosh and Tejal found the cause of TestFRJoin failures, it's in 
PIG-1012 and unrelated to this patch.

-D

> Should be able to set job priority through Pig Latin
> 
>
> Key: PIG-1025
> URL: https://issues.apache.org/jira/browse/PIG-1025
> Project: Pig
>  Issue Type: New Feature
>  Components: grunt
>Affects Versions: 0.4.0
>Reporter: Kevin Weil
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1025.patch, PIG-1025_2.patch, 
> TEST-org.apache.pig.test.TestFRJoin.txt
>
>
> Currently users can set the job name through Pig Latin by saying
> set job.name 'my job name'
> The ability to set the priority would also be nice, and the patch should be 
> small.  The goal is to be able to say
> set job.priority 'high'
> and throw a JobCreationException in the JobControlCompiler if the priority is 
> not one of the allowed string values from the o.a.h.mapred.JobPriority enum: 
> very_low, low, normal, high, very_high.   Case insensitivity makes this a 
> little nicer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Olga Natkovich (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768427#action_12768427
 ] 

Olga Natkovich commented on PIG-1012:
-

good catch! Daniel, could you swap the value in your patch

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012-2.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines non-transient non-serializa

[jira] Commented: (PIG-1028) FINDBUGS: DM_NUMBER_CTOR: Method invokes inefficient Number constructor; use static valueOf instead

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768429#action_12768429
 ] 

Daniel Dai commented on PIG-1028:
-

+1, target findbugs warnings suppressed.

> FINDBUGS: DM_NUMBER_CTOR: Method invokes inefficient Number constructor; use 
> static valueOf instead
> ---
>
> Key: PIG-1028
> URL: https://issues.apache.org/jira/browse/PIG-1028
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
>Assignee: Olga Natkovich
> Attachments: PIG-1028.patch
>
>
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.getStatistics() 
> invokes inefficient new Long(long) constructor; use Long.valueOf(long) instead
> BxMethod org.apache.pig.backend.hadoop.datastorage.HDataStorage.init() 
> invokes inefficient new Short(short) constructor; use Short.valueOf(short) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HPath.getConfiguration() invokes 
> inefficient new Long(long) constructor; use Long.valueOf(long) instead
> BxMethod 
> org.apache.pig.backend.hadoop.datastorage.HPath.getConfiguration() invokes 
> inefficient new Short(short) constructor; use Short.valueOf(short) instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.addShiftedKeyInfoIndex(int,
>  POPackage) invokes inefficient new Integer(int) constructor; use 
> Integer.valueOf(int) instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.POPackageAnnotator$LoRearrangeDiscoverer.visitLocalRearrange(POLocalRearrange)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Mod.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Mod.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Multiply.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Multiply.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Integer)
>  invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Long)
>  invokes inefficient new Long(long) constructor; use Long.valueOf(long) 
> instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.setIndex(int,
>  boolean) invokes inefficient new Byte(byte) constructor; use 
> Byte.valueOf(byte) instead
> BxMethod 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrangeForIllustrate.constructLROutput(List,
>  Tuple) invokes inefficient new Byte(byte) constructor; use 
> Byte.valueOf(byte) instead
> BxMethod org.apache.pig.builtin.ARITY.exec(Tuple) invokes inefficient new 
> Integer(int) constructor; use Integer.valu

[jira] Updated: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-1012:


Attachment: PIG-1012-3.patch

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Attachments: PIG-1012-2.patch, PIG-1012-3.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.ex

[jira] Commented: (PIG-1025) Should be able to set job priority through Pig Latin

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768435#action_12768435
 ] 

Alan Gates commented on PIG-1025:
-

Alright, I'll wait for Daniel to get that patch checked in, then I'll update 
and rerun the tests.

> Should be able to set job priority through Pig Latin
> 
>
> Key: PIG-1025
> URL: https://issues.apache.org/jira/browse/PIG-1025
> Project: Pig
>  Issue Type: New Feature
>  Components: grunt
>Affects Versions: 0.4.0
>Reporter: Kevin Weil
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1025.patch, PIG-1025_2.patch, 
> TEST-org.apache.pig.test.TestFRJoin.txt
>
>
> Currently users can set the job name through Pig Latin by saying
> set job.name 'my job name'
> The ability to set the priority would also be nice, and the patch should be 
> small.  The goal is to be able to say
> set job.priority 'high'
> and throw a JobCreationException in the JobControlCompiler if the priority is 
> not one of the allowed string values from the o.a.h.mapred.JobPriority enum: 
> very_low, low, normal, high, very_high.   Case insensitivity makes this a 
> little nicer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-747) Logical to Physical Plan Translation fails when temporary alias are created within foreach

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768444#action_12768444
 ] 

Hadoop QA commented on PIG-747:
---

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12422758/PIG-747-1.patch
  against trunk revision 827829.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/106/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/106/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/106/console

This message is automatically generated.

> Logical to Physical Plan Translation fails when temporary alias are created 
> within foreach
> --
>
> Key: PIG-747
> URL: https://issues.apache.org/jira/browse/PIG-747
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.4.0
>Reporter: Viraj Bhat
>Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: physicalplan.txt, physicalplanprob.pig, PIG-747-1.patch
>
>
> Consider a the pig script which calculates a new column F inside the foreach 
> as:
> {code}
> A = load 'physicalplan.txt' as (col1,col2,col3);
> B = foreach A {
>D = col1/col2;
>E = col3/col2;
>F = E - (D*D);
>generate
>F as newcol;
> };
> dump B;
> {code}
> This gives the following error:
> ===
> Caused by: 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogicalToPhysicalTranslatorException:
>  ERROR 2015: Invalid physical operators in the physical plan
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:377)
> at 
> org.apache.pig.impl.logicalLayer.LOMultiply.visit(LOMultiply.java:63)
> at 
> org.apache.pig.impl.logicalLayer.LOMultiply.visit(LOMultiply.java:29)
> at 
> org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:908)
> at 
> org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:122)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:41)
> at 
> org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:246)
> ... 10 more
> Caused by: org.apache.pig.impl.plan.PlanException: ERROR 0: Attempt to give 
> operator of type 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Divide
>  multiple outputs.  This operator does not support multiple outputs.
> at 
> org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:158)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:89)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:373)
> ... 19 more
> ===

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-976) Multi-query optimization throws ClassCastException

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768445#action_12768445
 ] 

Daniel Dai commented on PIG-976:


+1, fix is good.

> Multi-query optimization throws ClassCastException
> --
>
> Key: PIG-976
> URL: https://issues.apache.org/jira/browse/PIG-976
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Ankur
>Assignee: Richard Ding
> Fix For: 0.6.0
>
> Attachments: PIG-976.patch, PIG-976.patch, PIG-976.patch, 
> PIG-976.patch, PIG-976.patch, PIG-976_1.patch
>
>
> Multi-query optimization fails to merge 2 branches when 1 is a result of 
> Group By ALL and another is a result of Group By field1 where field 1 is of 
> type long. Here is the script that fails with multi-query on.
> data = LOAD 'test' USING PigStorage('\t') AS (a:long, b:double, c:double); 
> A = GROUP data ALL;
> B = FOREACH A GENERATE SUM(data.b) AS sum1, SUM(data.c) AS sum2;
> C = FOREACH B GENERATE (sum1/sum2) AS rate; 
> STORE C INTO 'result1';
> D = GROUP data BY a; 
> E = FOREACH D GENERATE group AS a, SUM(data.b), SUM(data.c);
> STORE E into 'result2';
>  
> Here is the exception from the logs
> java.lang.ClassCastException: org.apache.pig.data.DefaultTuple cannot be cast 
> to org.apache.pig.data.DataBag
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:399)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:180)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.processInput(POUserFunc.java:145)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:197)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.runPipeline(PODemux.java:264)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.getNext(PODemux.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:196)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:174)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:63)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:906)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:786)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
>   at 
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2206)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (PIG-976) Multi-query optimization throws ClassCastException

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-976.


Resolution: Fixed

Patch committed.

> Multi-query optimization throws ClassCastException
> --
>
> Key: PIG-976
> URL: https://issues.apache.org/jira/browse/PIG-976
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Ankur
>Assignee: Richard Ding
> Fix For: 0.6.0
>
> Attachments: PIG-976.patch, PIG-976.patch, PIG-976.patch, 
> PIG-976.patch, PIG-976.patch, PIG-976_1.patch
>
>
> Multi-query optimization fails to merge 2 branches when 1 is a result of 
> Group By ALL and another is a result of Group By field1 where field 1 is of 
> type long. Here is the script that fails with multi-query on.
> data = LOAD 'test' USING PigStorage('\t') AS (a:long, b:double, c:double); 
> A = GROUP data ALL;
> B = FOREACH A GENERATE SUM(data.b) AS sum1, SUM(data.c) AS sum2;
> C = FOREACH B GENERATE (sum1/sum2) AS rate; 
> STORE C INTO 'result1';
> D = GROUP data BY a; 
> E = FOREACH D GENERATE group AS a, SUM(data.b), SUM(data.c);
> STORE E into 'result2';
>  
> Here is the exception from the logs
> java.lang.ClassCastException: org.apache.pig.data.DefaultTuple cannot be cast 
> to org.apache.pig.data.DataBag
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:399)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:180)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.processInput(POUserFunc.java:145)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:197)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.runPipeline(PODemux.java:264)
>   at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.getNext(PODemux.java:254)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:196)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:174)
>   at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:63)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:906)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:786)
>   at 
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
>   at 
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2206)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (PIG-1012) FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in serializable class

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-1012.
-

   Resolution: Fixed
Fix Version/s: 0.6.0
 Hadoop Flags: [Reviewed]

Patch committed.

> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in 
> serializable class
> ---
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
>  Issue Type: Bug
>Reporter: Olga Natkovich
> Fix For: 0.6.0
>
> Attachments: PIG-1012-2.patch, PIG-1012-3.patch, PIG-1012.patch
>
>
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field is
> SeClass org.apache.pig.backend.executionengine.PigSlice defines 
> non-transient non-serializable instance field loader
> Sejava.util.zip.GZIPInputStream stored into non-transient field 
> PigSlice.is
> Seorg.apache.pig.backend.datastorage.SeekableInputStream stored into 
> non-transient field PigSlice.is
> Seorg.apache.tools.bzip2r.CBZip2InputStream stored into non-transient 
> field PigSlice.is
> Seorg.apache.pig.builtin.PigStorage stored into non-transient field 
> PigSlice.loader
> Seorg.apache.pig.backend.hadoop.DoubleWritable$Comparator implements 
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
>  implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
>  implements Comparator but not Serializable
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper 
> defines non-transient non-serializable instance field nig
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
>  defines non-transient non-serializable instance field bagIterator
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>  defines non-transient non-serializable instance field log
> SeClass 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
>  defines n

[jira] Commented: (PIG-1025) Should be able to set job priority through Pig Latin

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768456#action_12768456
 ] 

Hadoop QA commented on PIG-1025:


-1 overall.  Here are the results of testing the latest attachment 
  
http://issues.apache.org/jira/secure/attachment/12422834/TEST-org.apache.pig.test.TestFRJoin.txt
  against trunk revision 828213.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 415 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/107/console

This message is automatically generated.

> Should be able to set job priority through Pig Latin
> 
>
> Key: PIG-1025
> URL: https://issues.apache.org/jira/browse/PIG-1025
> Project: Pig
>  Issue Type: New Feature
>  Components: grunt
>Affects Versions: 0.4.0
>Reporter: Kevin Weil
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1025.patch, PIG-1025_2.patch, 
> TEST-org.apache.pig.test.TestFRJoin.txt
>
>
> Currently users can set the job name through Pig Latin by saying
> set job.name 'my job name'
> The ability to set the priority would also be nice, and the patch should be 
> small.  The goal is to be able to say
> set job.priority 'high'
> and throw a JobCreationException in the JobControlCompiler if the priority is 
> not one of the allowed string values from the o.a.h.mapred.JobPriority enum: 
> very_low, low, normal, high, very_high.   Case insensitivity makes this a 
> little nicer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1035) support for skewed outer join

[Olga Natkovich (JIRA)]

support for skewed outer join
-

 Key: PIG-1035
 URL: https://issues.apache.org/jira/browse/PIG-1035
 Project: Pig
  Issue Type: New Feature
Reporter: Olga Natkovich
Assignee: Sriranjan Manjunath


Similarly to skewed inner join, skewed outer join will help to scale in the 
presense of join keys that don't fit into memory

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1036) Fragment-replicate left outer join

[Olga Natkovich (JIRA)]

Fragment-replicate left outer join
--

 Key: PIG-1036
 URL: https://issues.apache.org/jira/browse/PIG-1036
 Project: Pig
  Issue Type: New Feature
Reporter: Olga Natkovich




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1037) better memory layout and spill for sorted and distinct bags

[Olga Natkovich (JIRA)]

better memory layout and spill for sorted and distinct bags
---

 Key: PIG-1037
 URL: https://issues.apache.org/jira/browse/PIG-1037
 Project: Pig
  Issue Type: Improvement
Reporter: Olga Natkovich
Assignee: Ying He




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1038) stream nested distinct for in case of accumulate interface

[Olga Natkovich (JIRA)]

stream nested distinct for in case of accumulate interface
--

 Key: PIG-1038
 URL: https://issues.apache.org/jira/browse/PIG-1038
 Project: Pig
  Issue Type: Improvement
Reporter: Olga Natkovich
Assignee: Daniel Dai


Since the data coming to the reducer is sorted on group+distinct, we don't need 
to see all distinct values at once

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (PIG-1039) Pig 0.5 Doc Updates

[Corinne Chandel (JIRA)]

Pig 0.5 Doc Updates
---

 Key: PIG-1039
 URL: https://issues.apache.org/jira/browse/PIG-1039
 Project: Pig
  Issue Type: Task
  Components: documentation
Affects Versions: 0.5.0
Reporter: Corinne Chandel
Priority: Blocker
 Fix For: 0.5.0


Pig 0.5 doc updates (to be applied to Trunk and branch-0.5)

1. updates to tutorial
2. updates to pig latin reference manual
3. updated doc tab to 0.5.0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1039) Pig 0.5 Doc Updates

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel updated PIG-1039:
-

Attachment: trunk.patch

Patch for TRUNK
Apply to: 
http://svn.apache.org/repos/asf/hadoop/pig/trunk


> Pig 0.5 Doc Updates
> ---
>
> Key: PIG-1039
> URL: https://issues.apache.org/jira/browse/PIG-1039
> Project: Pig
>  Issue Type: Task
>  Components: documentation
>Affects Versions: 0.5.0
>Reporter: Corinne Chandel
>Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: branch-0.5.patch, trunk.patch
>
>
> Pig 0.5 doc updates (to be applied to Trunk and branch-0.5)
> 1. updates to tutorial
> 2. updates to pig latin reference manual
> 3. updated doc tab to 0.5.0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1039) Pig 0.5 Doc Updates

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel updated PIG-1039:
-

Attachment: branch-0.5.patch

Patch for branch-0.5
Apply to:
http://svn.apache.org/repos/asf/hadoop/pig/branches/branch-0.5


> Pig 0.5 Doc Updates
> ---
>
> Key: PIG-1039
> URL: https://issues.apache.org/jira/browse/PIG-1039
> Project: Pig
>  Issue Type: Task
>  Components: documentation
>Affects Versions: 0.5.0
>Reporter: Corinne Chandel
>Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: branch-0.5.patch, trunk.patch
>
>
> Pig 0.5 doc updates (to be applied to Trunk and branch-0.5)
> 1. updates to tutorial
> 2. updates to pig latin reference manual
> 3. updated doc tab to 0.5.0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1039) Pig 0.5 Doc Updates

[Corinne Chandel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Corinne Chandel updated PIG-1039:
-

Status: Patch Available  (was: Open)

Apply 2 patches (1) trunk (2) branch-0.5

Note: No new test code required; changes to documentation only.

> Pig 0.5 Doc Updates
> ---
>
> Key: PIG-1039
> URL: https://issues.apache.org/jira/browse/PIG-1039
> Project: Pig
>  Issue Type: Task
>  Components: documentation
>Affects Versions: 0.5.0
>Reporter: Corinne Chandel
>Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: branch-0.5.patch, trunk.patch
>
>
> Pig 0.5 doc updates (to be applied to Trunk and branch-0.5)
> 1. updates to tutorial
> 2. updates to pig latin reference manual
> 3. updated doc tab to 0.5.0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (PIG-1037) better memory layout and spill for sorted and distinct bags

[Ying He (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ying He updated PIG-1037:
-

Attachment: PIG-1037.patch

first cut of patch for initial testing purpose. regression tests are not done 
yet. It may contain bugs.

> better memory layout and spill for sorted and distinct bags
> ---
>
> Key: PIG-1037
> URL: https://issues.apache.org/jira/browse/PIG-1037
> Project: Pig
>  Issue Type: Improvement
>Reporter: Olga Natkovich
>Assignee: Ying He
> Attachments: PIG-1037.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (PIG-990) Provide a way to pin LogicalOperator Options

[Dmitriy V. Ryaboy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy V. Ryaboy reassigned PIG-990:
-

Assignee: Dmitriy V. Ryaboy

> Provide a way to pin LogicalOperator Options
> 
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Dmitriy V. Ryaboy
>Assignee: Dmitriy V. Ryaboy
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety 
> of join types (regular, fr, skewed, merge), which can be set by the user or 
> chosen by a hypothetical optimizer.  If a user selects a join type, pig 
> philoophy guides us to always respect the user's choice and not explore 
> alternatives.  Therefore, we need a way to "pin" options.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1033) javac warnings: deprecated hadoop APIs

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768528#action_12768528
 ] 

Daniel Dai commented on PIG-1033:
-

We have options:
1. Change build.xml to disable deprecation warnings. BTW, hadoop does not check 
for deprecation when compiling
2. Using @SuppressWarnings("deprecation") to suppress specific blocks. The 
problem is we cannot suppress import block, all import warnings still there
3. Using @SuppressWarnings("deprecation") and do not import deprecate class. 
Use full qualified class name for deprecate class when using it. The downside 
is patch is relatively lengthy.

> javac warnings: deprecated hadoop APIs
> --
>
> Key: PIG-1033
> URL: https://issues.apache.org/jira/browse/PIG-1033
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.4.0
>Reporter: Daniel Dai
> Fix For: 0.6.0
>
>
> Suppress javac warnings related to deprecated hadoop APIs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-996) [zebra] Zebra build script does not have findbugs and clover targets.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768549#action_12768549
 ] 

Hadoop QA commented on PIG-996:
---

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12422619/patch_build
  against trunk revision 828213.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 4 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/108/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/108/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/108/console

This message is automatically generated.

> [zebra] Zebra build script does not have findbugs and clover targets.
> -
>
> Key: PIG-996
> URL: https://issues.apache.org/jira/browse/PIG-996
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.4.0
>Reporter: Chao Wang
>Assignee: Chao Wang
> Fix For: 0.6.0
>
> Attachments: patch_build, patch_build
>
>
> Zebra build script does not have findbugs and clover targets, leading hudson 
> build process to fail on Zebra.
> This jira is to fix this by adding these two targets.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Pig-trunk #597

[Apache Hudson Server]

See 

Changes:

[daijy] Fix PIG-976, PIG-1012 unit test failures

[olga] PIG-1028: FINDBUGS: DM_NUMBER_CTOR: Method invokes inefficient Number
constructor; use static valueOf instead (olgan)

[gates] PIG-944  Change schema to be taken from StoreConfig instead of 
TableStorer's constructor.

--
[...truncated 2558 lines...]

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:

ivy-buildJar:
[ivy:resolve] :: resolving dependencies :: 
org.apache.pig#Pig;2009-10-22_03-08-24
[ivy:resolve]   confs: [buildJar]
[ivy:resolve]   found com.jcraft#jsch;0.1.38 in maven2
[ivy:resolve]   found jline#jline;0.9.94 in maven2
[ivy:resolve]   found net.java.dev.javacc#javacc;4.2 in maven2
[ivy:resolve]   found junit#junit;4.5 in default
[ivy:resolve] :: resolution report :: resolve 61ms :: artifacts dl 4ms
-
|  |modules||   artifacts   |
|   conf   | number| search|dwnlded|evicted|| number|dwnlded|
-
| buildJar |   4   |   0   |   0   |   0   ||   4   |   0   |
-
[ivy:retrieve] :: retrieving :: org.apache.pig#Pig
[ivy:retrieve]  confs: [buildJar]
[ivy:retrieve]  1 artifacts copied, 3 already retrieved (288kB/5ms)

buildJar:
 [echo] svnString 828285
  [jar] Building jar: 

 [copy] Copying 1 file to 


jarWithOutSvn:

findbugs:
[mkdir] Created dir: 

 [findbugs] Executing findbugs from ant task
 [findbugs] Running FindBugs...
 [findbugs] The following classes needed for analysis were missing:
 [findbugs]   com.jcraft.jsch.SocketFactory
 [findbugs]   com.jcraft.jsch.Logger
 [findbugs]   jline.Completor
 [findbugs]   com.jcraft.jsch.Session
 [findbugs]   com.jcraft.jsch.HostKeyRepository
 [findbugs]   com.jcraft.jsch.JSch
 [findbugs]   com.jcraft.jsch.UserInfo
 [findbugs]   jline.ConsoleReaderInputStream
 [findbugs]   com.jcraft.jsch.HostKey
 [findbugs]   jline.ConsoleReader
 [findbugs]   com.jcraft.jsch.ChannelExec
 [findbugs]   jline.History
 [findbugs]   com.jcraft.jsch.ChannelDirectTCPIP
 [findbugs]   com.jcraft.jsch.JSchException
 [findbugs]   com.jcraft.jsch.Channel
 [findbugs] Warnings generated: 229
 [findbugs] Missing classes: 16
 [findbugs] Calculating exit code...
 [findbugs] Setting 'missing class' flag (2)
 [findbugs] Setting 'bugs found' flag (1)
 [findbugs] Exit code set to: 3
 [findbugs] Java Result: 3
 [findbugs] Classes needed for analysis were missing
 [findbugs] Output saved to 

 [xslt] Processing 

 to 

 [xslt] Loading stylesheet 
/homes/gkesavan/tools/findbugs/latest/src/xsl/default.xsl

BUILD SUCCESSFUL
Total time: 2 minutes 47 seconds
+ mv build/pig-2009-10-22_03-08-24.tar.gz 

+ mv build/test/findbugs 

+ mv build/docs/api 

+ /homes/hudson/tools/ant/apache-ant-1.7.0/bin/ant clean
Buildfile: build.xml

clean:
   [delete] Deleting directory 

   [delete] Deleting directory 

   [delete] Deleting directory 

   [delete] Deleting directory 


BUILD SUCCESSFUL
Total time: 0 seconds
+ /homes/hudson/tools/ant/apache-ant-1.7.0/bin/ant 
-Dtest.junit.output.format=xml -Dtest.output=yes 
-Dcheckstyle.home=/homes/hudson/tools/checkstyle/latest -Drun.clover=true 
-Dclover.home=/homes/hudson/tools/clover/clover-ant-2.3.2 clover test 
generate-clover-reports
Buildfile: build.xml

clover.setup:
[mkdir] Created dir: 

[clover-setup] Clover Version 2.3.2, built on July 15 2008 (build-732)
[clover-setup] Loaded from: 
/homes/hudson/tools/clover/clover-ant-2.3.2/lib/clover.jar
[clover-setup] Clover: Open Source License registered to Apache Software 

[jira] Commented: (PIG-996) [zebra] Zebra build script does not have findbugs and clover targets.

[Giridharan Kesavan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768567#action_12768567
 ] 

Giridharan Kesavan commented on PIG-996:


How do we want this findbugs and checkstsyle to be run?
with this patch if someone wants to execute findbugs they cant run it from the 
top level pig project..
 
Can we not use subant task to run finbugs on zebra as well when findbugs at the 
root level(pig tunk) is called?






> [zebra] Zebra build script does not have findbugs and clover targets.
> -
>
> Key: PIG-996
> URL: https://issues.apache.org/jira/browse/PIG-996
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.4.0
>Reporter: Chao Wang
>Assignee: Chao Wang
> Fix For: 0.6.0
>
> Attachments: patch_build, patch_build
>
>
> Zebra build script does not have findbugs and clover targets, leading hudson 
> build process to fail on Zebra.
> This jira is to fix this by adding these two targets.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.