[jira] [Commented] (PIG-2579) Support for multiple input schemas in AvroStorage

[Santhosh Srinivasan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475507#comment-13475507
 ] 

Santhosh Srinivasan commented on PIG-2579:
--

My apologies on missing out on adding these files. I have committed both of 
them to trunk and branch-0.11. Cheolsoo, thanks for pointing it out.

> Support for multiple input schemas in AvroStorage
> -
>
> Key: PIG-2579
> URL: https://issues.apache.org/jira/browse/PIG-2579
> Project: Pig
>  Issue Type: New Feature
>  Components: piggybank
>Affects Versions: 0.9.2, 0.11
>Reporter: Stan Rosenberg
>Assignee: Cheolsoo Park
>Priority: Minor
> Fix For: 0.11
>
> Attachments: avro_storage_union_schema.patch, 
> avro_storage_union_schema_test.tar.gz, PIG-2579-2-avro_test_files.tar.gz, 
> PIG-2579-2.patch, PIG-2579-3.patch, PIG-2579-4.patch, PIG-2579-5.patch, 
> PIG-2579-6.patch
>
>
> This is a barebones patch for AvroStorage which enables support of multiple 
> input schemas.  The assumption is that the input consists of avro files 
> having different schemas that can be unioned, e.g., flat records.  
> A simple illustrative example is attached 
> (avro_storage_union_schema_test.tar.gz): run create_avro1.pig, followed by 
> create_avro2.pig, followed by read_avro.pig.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2910) Add function to read schema from outout of Schema.toString()

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2910:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

+1 Patch committed to trunk.
Eli, Thanks for the patch!

> Add function to read schema from outout of Schema.toString()
> 
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
>  Issue Type: Improvement
>  Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Eli Reisman
>  Labels: newbie
> Fix For: 0.11
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch, 
> PIG-2910-4.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At 
> the moment this requires writing your own toString() method that 
> Utils.getSchemaFromString() can read. Making a readable schema for the 
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the 
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1, 
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2910) Add function to read schema from outout of Schema.toString()

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2910:
---

Fix Version/s: (was: 0.10.1)
   Status: Patch Available  (was: Open)

> Add function to read schema from outout of Schema.toString()
> 
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
>  Issue Type: Improvement
>  Components: impl, parser
>Affects Versions: 0.10.0, 0.9.2, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Eli Reisman
>  Labels: newbie
> Fix For: 0.11
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch, 
> PIG-2910-4.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At 
> the moment this requires writing your own toString() method that 
> Utils.getSchemaFromString() can read. Making a readable schema for the 
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the 
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1, 
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2910) Add function to read schema from outout of Schema.toString()

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2910:
---

Summary: Add function to read schema from outout of Schema.toString()  
(was: Make toString() methods on Schema and FieldSchema be readable by 
Utils.getSchemaFromString())

> Add function to read schema from outout of Schema.toString()
> 
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
>  Issue Type: Improvement
>  Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Eli Reisman
>  Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch, 
> PIG-2910-4.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At 
> the moment this requires writing your own toString() method that 
> Utils.getSchemaFromString() can read. Making a readable schema for the 
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the 
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1, 
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2910:
---

Issue Type: Improvement  (was: Bug)

> Make toString() methods on Schema and FieldSchema be readable by 
> Utils.getSchemaFromString()
> 
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
>  Issue Type: Improvement
>  Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Eli Reisman
>  Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch, 
> PIG-2910-4.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At 
> the moment this requires writing your own toString() method that 
> Utils.getSchemaFromString() can read. Making a readable schema for the 
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the 
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1, 
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2910) Make toString() methods on Schema and FieldSchema be readable by Utils.getSchemaFromString()

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2910:
---

Assignee: Eli Reisman  (was: Thejas M Nair)

> Make toString() methods on Schema and FieldSchema be readable by 
> Utils.getSchemaFromString()
> 
>
> Key: PIG-2910
> URL: https://issues.apache.org/jira/browse/PIG-2910
> Project: Pig
>  Issue Type: Bug
>  Components: impl, parser
>Affects Versions: 0.9.2, 0.10.0, 0.11, 0.10.1
>Reporter: Russell Jurney
>Assignee: Eli Reisman
>  Labels: newbie
> Fix For: 0.11, 0.10.1
>
> Attachments: PIG-2910-1.patch, PIG-2910-2.patch, PIG-2910-3.patch, 
> PIG-2910-4.patch
>
>
> I want to toString() schemas and send them to the backend via UDFContext. At 
> the moment this requires writing your own toString() method that 
> Utils.getSchemaFromString() can read. Making a readable schema for the 
> backend would be an improvement.
> I spoke with Thejas, who believes this is a bug. The workaround for the 
> moment is, for example:
> String schemaString = inputSchema.toString().substring(1, 
> inputSchema.toString().length() - 1);
> // Set the input schema for processing
> UDFContext context = UDFContext.getUDFContext();
> Properties udfProp = context.getUDFProperties(this.getClass());
> udfProp.setProperty("horton.json.udf.schema", schemaString);
> ...
> schema = Utils.getSchemaFromString(strSchema);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2579) Support for multiple input schemas in AvroStorage

[Cheolsoo Park (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475488#comment-13475488
 ] 

Cheolsoo Park commented on PIG-2579:


@Santhosh,
I think that you omitted two files:
contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testMultipleSchemas1.avro
contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testMultipleSchemas2.avro

TestAvroStorage is failing due to missing files. Can you please commit them to 
trunk and brach-0.11?

Thanks!

> Support for multiple input schemas in AvroStorage
> -
>
> Key: PIG-2579
> URL: https://issues.apache.org/jira/browse/PIG-2579
> Project: Pig
>  Issue Type: New Feature
>  Components: piggybank
>Affects Versions: 0.9.2, 0.11
>Reporter: Stan Rosenberg
>Assignee: Cheolsoo Park
>Priority: Minor
> Fix For: 0.11
>
> Attachments: avro_storage_union_schema.patch, 
> avro_storage_union_schema_test.tar.gz, PIG-2579-2-avro_test_files.tar.gz, 
> PIG-2579-2.patch, PIG-2579-3.patch, PIG-2579-4.patch, PIG-2579-5.patch, 
> PIG-2579-6.patch
>
>
> This is a barebones patch for AvroStorage which enables support of multiple 
> input schemas.  The assumption is that the input consists of avro files 
> having different schemas that can be unioned, e.g., flat records.  
> A simple illustrative example is attached 
> (avro_storage_union_schema_test.tar.gz): run create_avro1.pig, followed by 
> create_avro2.pig, followed by read_avro.pig.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Subscription: PIG patch available

[jira]

Issue Subscription
Filter: PIG patch available (38 issues)

Subscriber: pigdaily

Key Summary
PIG-2963Illustrate command and POPackageLite
https://issues.apache.org/jira/browse/PIG-2963
PIG-2960Increase the timeout for unit test
https://issues.apache.org/jira/browse/PIG-2960
PIG-2959Add a pig.cmd for Pig to run under Windows
https://issues.apache.org/jira/browse/PIG-2959
PIG-2958Pig tests do not appear to have a logger attached
https://issues.apache.org/jira/browse/PIG-2958
PIG-2957TetsScriptUDF fail due to volume prefix in jar
https://issues.apache.org/jira/browse/PIG-2957
PIG-2956Invalid cache specification for some streaming statement
https://issues.apache.org/jira/browse/PIG-2956
PIG-2955 Fix bunch of Pig e2e tests on Windows 
https://issues.apache.org/jira/browse/PIG-2955
PIG-2954 TestParamSubPreproc still depends on "bash" to run 
https://issues.apache.org/jira/browse/PIG-2954
PIG-2953"which" utility does not exist on Windows
https://issues.apache.org/jira/browse/PIG-2953
PIG-2943DevTests, Refactor Windows checks to use new Util.WINDOWS method 
for code health
https://issues.apache.org/jira/browse/PIG-2943
PIG-2942DevTests, TestLoad has a false failure on Windows
https://issues.apache.org/jira/browse/PIG-2942
PIG-2940HBaseStorage store fails in secure cluster
https://issues.apache.org/jira/browse/PIG-2940
PIG-2931$ signs in the replacement string make parameter substitution fail
https://issues.apache.org/jira/browse/PIG-2931
PIG-2928Fix e2e test failures in trunk: FilterBoolean_23/24
https://issues.apache.org/jira/browse/PIG-2928
PIG-2925Extremely long JobConf values should not be added to Streaming 
environment
https://issues.apache.org/jira/browse/PIG-2925
PIG-2908Fix unit tests to work with jdk7
https://issues.apache.org/jira/browse/PIG-2908
PIG-2904Scripting UDFs should allow DEFINE statements to pass parameters to 
the UDF's constructor
https://issues.apache.org/jira/browse/PIG-2904
PIG-2898Parallel execution of e2e tests
https://issues.apache.org/jira/browse/PIG-2898
PIG-2881Add SUBTRACT eval function
https://issues.apache.org/jira/browse/PIG-2881
PIG-2873Converting bin/pig shell script to python
https://issues.apache.org/jira/browse/PIG-2873
PIG-2834MultiStorage requires unused constructor argument
https://issues.apache.org/jira/browse/PIG-2834
PIG-2824Pushing checking number of fields into LoadFunc
https://issues.apache.org/jira/browse/PIG-2824
PIG-2801grunt "sh" command should invoke the shell implicitly instead of 
calling exec directly with the command tokens
https://issues.apache.org/jira/browse/PIG-2801
PIG-2798pig streaming tests assume interpreters are auto-resolved
https://issues.apache.org/jira/browse/PIG-2798
PIG-2796Local temporary paths are not always valid HDFS path names.
https://issues.apache.org/jira/browse/PIG-2796
PIG-2795Fix test cases that generate pig scripts with "load " + pathStr to 
encode "\" in the path
https://issues.apache.org/jira/browse/PIG-2795
PIG-2794Pig test: add utils to simplify testing on Windows
https://issues.apache.org/jira/browse/PIG-2794
PIG-2778Add 'matches' operator to predicate pushdown
https://issues.apache.org/jira/browse/PIG-2778
PIG-2661Pig uses an extra job for loading data in Pigmix L9
https://issues.apache.org/jira/browse/PIG-2661
PIG-2657Print warning if using wrong jython version
https://issues.apache.org/jira/browse/PIG-2657
PIG-2495Using merge JOIN from a HBaseStorage produces an error
https://issues.apache.org/jira/browse/PIG-2495
PIG-2417Streaming UDFs -  allow users to easily write UDFs in scripting 
languages with no JVM implementation.
https://issues.apache.org/jira/browse/PIG-2417
PIG-2405svn tags/release-0.9.1: some unit test case failed with open JDK
https://issues.apache.org/jira/browse/PIG-2405
PIG-2362Rework Ant build.xml to use macrodef instead of antcall
https://issues.apache.org/jira/browse/PIG-2362
PIG-2312NPE when relation and column share the same name and used in Nested 
Foreach 
https://issues.apache.org/jira/browse/PIG-2312
PIG-1942script UDF (jython) should utilize the intended output schema to 
more directly convert Py objects to Pig objects
https://issues.apache.org/jira/browse/PIG-1942
PIG-1283COUNT on null bag causes failure
https://issues.apache.org/jira/browse/PIG-1283
PIG-1237Piggybank MutliStorage - specify field to write in output
https://issues.apache.org/jira/browse/PIG-1237

You may edit this 

Re: Pig 0.11

[Dmitriy Ryaboy]

Thanks Olga and welcome back! 
I know there's some process for linking jiras to releases, but I'm not sure 
what that is. If you could explain and maybe cover a portion of that work, 
that'd be super helpful. And reviews, of course. 

On Oct 12, 2012, at 2:06 PM, Olga Natkovich  wrote:

> Dmitry, I would be happy to help with the release process. Want to get back 
> into this now that I am back at work. Let me know what you would like me to 
> do.
>  
> Olga
> 
> 
> 
> 
> From: Dmitriy Ryaboy 
> To: dev@pig.apache.org 
> Cc: billgra...@gmail.com 
> Sent: Thursday, October 11, 2012 2:44 PM
> Subject: Re: Pig 0.11
> 
> Ok I will branch 0.11 tomorrow morning unless someone objects.
> From then on, committers should be careful to commit bug fixes to both
> 0.11 branch and trunk; minor polish can go into the branch, but whole
> new features should not (we can discuss on the list if something is in
> the gray area).
> 
> D
> 
> On Thu, Oct 11, 2012 at 2:16 PM, Gianmarco De Francisci Morales
>  wrote:
>> I added it as a dependency as it has already its own Jira.
>> I hope it is OK.
>> 
>> Cheers,
>> --
>> Gianmarco
>> 
>> 
>> 
>> On Wed, Oct 10, 2012 at 11:23 PM, Bill Graham  wrote:
>> 
>>> +1 for me.
>>> 
>>> There's https://issues.apache.org/jira/browse/PIG-2756 which tracks a few
>>> documentation issues that should block Pig 0.11, but they can also be done
>>> on the trunk and merged to the branch. Gianmarco, you can add a rank
>>> subtask there to serve as a reminder.
>>> 
>>> 
>>> On Wed, Oct 10, 2012 at 11:03 PM, Gianmarco De Francisci Morales <
>>> g...@apache.org> wrote:
>>> 
 We are missing some documentation on the RANK but I guess we could add
>>> that
 to the branch and trunk in parallel.
 All the patches I was keeping an eye on are in.
 
 So +1 for me.
 --
 Gianmarco
 
 
 
 On Wed, Oct 10, 2012 at 5:31 PM, Jonathan Coveney  wrote:
 
> I think all of the major patches are in, no? Now it's just bug testing?
> Just wanted to touch base on where we are at with this.
> 
 
>>> 
>>> 
>>> 
>>> --
>>> *Note that I'm no longer using my Yahoo! email address. Please email me at
>>> billgra...@gmail.com going forward.*

[jira] [Commented] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query

[Gianmarco De Francisci Morales (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475423#comment-13475423
 ] 

Gianmarco De Francisci Morales commented on PIG-2970:
-

Haven't had time to look at the patch, but I guess it is related to PIG-2119.
I thought we had solved it though.

> Nested foreach getting incorrect schema when having unrelated inner query
> -
>
> Key: PIG-2970
> URL: https://issues.apache.org/jira/browse/PIG-2970
> Project: Pig
>  Issue Type: Bug
>  Components: parser
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Minor
> Attachments: pig-2970-trunk-v01.txt
>
>
> While looking at PIG-2968, hit a weird error message.
> {noformat}
> $ cat -n test/foreach2.pig
>  1  daily = load 'nyse' as (exchange, symbol);
>  2  grpd = group daily by exchange;
>  3  unique = foreach grpd {
>  4  sym = daily.symbol;
>  5  uniq_sym = distinct sym;
>  6  --ignoring uniq_sym result
>  7  generate group, daily;
>  8  };
>  9  describe unique;
> 10  zzz = foreach unique generate group;
> 11  explain zzz;
> % pig -x local -t ColumnMapKeyPrune test/foreach2.pig
> ...
> unique: {symbol: bytearray}
> 2012-10-12 16:55:44,226 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
> 1025: 
>  Invalid field projection. 
> Projected field [group] does not exist in schema: symbol:bytearray.
> ...
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query

[Koji Noguchi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Noguchi updated PIG-2970:
--

Attachment: pig-2970-trunk-v01.txt

Parser assumes foreach's innerPlan has a single generate op as sink.  When this 
fails, it is returning an incorrect schema.  Attaching a patch that moves all 
the unrelated ops to LOGenerate.

My understanding of pig is weak.  Appreciate if someone can take a careful 
look. Thanks.

> Nested foreach getting incorrect schema when having unrelated inner query
> -
>
> Key: PIG-2970
> URL: https://issues.apache.org/jira/browse/PIG-2970
> Project: Pig
>  Issue Type: Bug
>  Components: parser
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Minor
> Attachments: pig-2970-trunk-v01.txt
>
>
> While looking at PIG-2968, hit a weird error message.
> {noformat}
> $ cat -n test/foreach2.pig
>  1  daily = load 'nyse' as (exchange, symbol);
>  2  grpd = group daily by exchange;
>  3  unique = foreach grpd {
>  4  sym = daily.symbol;
>  5  uniq_sym = distinct sym;
>  6  --ignoring uniq_sym result
>  7  generate group, daily;
>  8  };
>  9  describe unique;
> 10  zzz = foreach unique generate group;
> 11  explain zzz;
> % pig -x local -t ColumnMapKeyPrune test/foreach2.pig
> ...
> unique: {symbol: bytearray}
> 2012-10-12 16:55:44,226 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
> 1025: 
>  Invalid field projection. 
> Projected field [group] does not exist in schema: symbol:bytearray.
> ...
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk #1334

[Apache Jenkins Server]

See 

Changes:

[dvryaboy] Preparing for 0.12.0 development

[dvryaboy] Preparing for release 0.11.0

[jcoveney] PIG-2965: RANDOM should allow seed initialization for ease of 
testing (jcoveney)

--
[...truncated 6948 lines...]
 [findbugs]   org.apache.hadoop.io.file.tfile.TFile$Reader$Scanner$Entry
 [findbugs]   org.apache.hadoop.fs.FSDataInputStream
 [findbugs]   org.python.core.PyObject
 [findbugs]   jline.History
 [findbugs]   org.jruby.embed.internal.LocalContextProvider
 [findbugs]   org.apache.hadoop.io.BooleanWritable
 [findbugs]   org.apache.log4j.Logger
 [findbugs]   org.apache.hadoop.hbase.filter.FamilyFilter
 [findbugs]   groovy.lang.Tuple
 [findbugs]   org.antlr.runtime.IntStream
 [findbugs]   org.apache.hadoop.util.ReflectionUtils
 [findbugs]   org.apache.hadoop.fs.ContentSummary
 [findbugs]   org.jruby.runtime.builtin.IRubyObject
 [findbugs]   org.jruby.RubyInteger
 [findbugs]   org.python.core.PyTuple
 [findbugs]   org.mortbay.log.Log
 [findbugs]   org.apache.hadoop.conf.Configuration
 [findbugs]   com.google.common.base.Joiner
 [findbugs]   org.apache.hadoop.mapreduce.lib.input.FileSplit
 [findbugs]   org.apache.hadoop.mapred.Counters$Counter
 [findbugs]   com.jcraft.jsch.Channel
 [findbugs]   org.apache.hadoop.mapred.JobPriority
 [findbugs]   org.apache.commons.cli.Options
 [findbugs]   org.apache.hadoop.mapred.JobID
 [findbugs]   org.apache.hadoop.util.bloom.BloomFilter
 [findbugs]   org.python.core.PyFrame
 [findbugs]   org.apache.hadoop.hbase.filter.CompareFilter
 [findbugs]   org.apache.hadoop.util.VersionInfo
 [findbugs]   org.python.core.PyString
 [findbugs]   org.apache.hadoop.io.Text$Comparator
 [findbugs]   org.jruby.runtime.Block
 [findbugs]   org.antlr.runtime.MismatchedSetException
 [findbugs]   org.apache.hadoop.io.BytesWritable
 [findbugs]   org.apache.hadoop.fs.FsShell
 [findbugs]   org.joda.time.Months
 [findbugs]   org.mozilla.javascript.ImporterTopLevel
 [findbugs]   org.apache.hadoop.hbase.mapreduce.TableOutputFormat
 [findbugs]   org.apache.hadoop.mapred.TaskReport
 [findbugs]   org.apache.hadoop.security.UserGroupInformation
 [findbugs]   org.antlr.runtime.tree.RewriteRuleSubtreeStream
 [findbugs]   org.apache.commons.cli.HelpFormatter
 [findbugs]   com.google.common.collect.Maps
 [findbugs]   org.joda.time.ReadableInstant
 [findbugs]   org.mozilla.javascript.NativeObject
 [findbugs]   org.apache.hadoop.hbase.HConstants
 [findbugs]   org.apache.hadoop.io.serializer.Deserializer
 [findbugs]   org.antlr.runtime.FailedPredicateException
 [findbugs]   org.apache.hadoop.io.compress.CompressionCodec
 [findbugs]   org.jruby.RubyNil
 [findbugs]   org.apache.hadoop.fs.FileStatus
 [findbugs]   org.apache.hadoop.hbase.client.Result
 [findbugs]   org.apache.hadoop.mapreduce.JobContext
 [findbugs]   org.codehaus.jackson.JsonGenerator
 [findbugs]   org.apache.hadoop.mapreduce.TaskAttemptContext
 [findbugs]   org.apache.hadoop.io.BytesWritable$Comparator
 [findbugs]   org.apache.hadoop.io.LongWritable$Comparator
 [findbugs]   org.codehaus.jackson.map.util.LRUMap
 [findbugs]   org.apache.hadoop.hbase.util.Bytes
 [findbugs]   org.antlr.runtime.MismatchedTokenException
 [findbugs]   org.codehaus.jackson.JsonParser
 [findbugs]   com.jcraft.jsch.UserInfo
 [findbugs]   org.python.core.PyException
 [findbugs]   org.apache.commons.cli.ParseException
 [findbugs]   org.apache.hadoop.io.compress.CompressionOutputStream
 [findbugs]   org.apache.hadoop.hbase.filter.WritableByteArrayComparable
 [findbugs]   org.antlr.runtime.tree.CommonTreeNodeStream
 [findbugs]   org.apache.log4j.Level
 [findbugs]   org.apache.hadoop.hbase.client.Scan
 [findbugs]   org.jruby.anno.JRubyMethod
 [findbugs]   org.apache.hadoop.mapreduce.Job
 [findbugs]   com.google.common.util.concurrent.Futures
 [findbugs]   org.apache.commons.logging.LogFactory
 [findbugs]   org.apache.commons.collections.IteratorUtils
 [findbugs]   org.apache.commons.codec.binary.Base64
 [findbugs]   org.codehaus.jackson.map.ObjectMapper
 [findbugs]   org.apache.hadoop.fs.FileSystem
 [findbugs]   org.jruby.embed.LocalContextScope
 [findbugs]   org.apache.hadoop.hbase.filter.FilterList$Operator
 [findbugs]   org.jruby.RubySymbol
 [findbugs]   org.apache.hadoop.hbase.io.ImmutableBytesWritable
 [findbugs]   org.apache.hadoop.io.serializer.SerializationFactory
 [findbugs]   org.antlr.runtime.tree.TreeAdaptor
 [findbugs]   org.apache.hadoop.mapred.RunningJob
 [findbugs]   org.antlr.runtime.CommonTokenStream
 [findbugs]   org.apache.hadoop.io.DataInputBuffer
 [findbugs]   org.apache.hadoop.io.file.tfile.TFile
 [findbugs]   org.apache.commons.cli.GnuParser
 [findbugs]   org.mozilla.javascript.Context
 [findbugs]   org.apache.hadoop.io.FloatWritable
 [findbugs]   org.antlr.runtime.tree.RewriteEarlyExitException
 [findbugs]   org.apache.hadoop.hbase.HBaseConfiguration
 [findbugs]   org.codehaus.jackson.JsonGenerationException
 [findbugs]   org.apache.hadoop.mapre

Re: Pig 0.11

[Olga Natkovich]

Dmitry, I would be happy to help with the release process. Want to get back 
into this now that I am back at work. Let me know what you would like me to do.
 
Olga




From: Dmitriy Ryaboy 
To: dev@pig.apache.org 
Cc: billgra...@gmail.com 
Sent: Thursday, October 11, 2012 2:44 PM
Subject: Re: Pig 0.11

Ok I will branch 0.11 tomorrow morning unless someone objects.
From then on, committers should be careful to commit bug fixes to both
0.11 branch and trunk; minor polish can go into the branch, but whole
new features should not (we can discuss on the list if something is in
the gray area).

D

On Thu, Oct 11, 2012 at 2:16 PM, Gianmarco De Francisci Morales
 wrote:
> I added it as a dependency as it has already its own Jira.
> I hope it is OK.
>
> Cheers,
> --
> Gianmarco
>
>
>
> On Wed, Oct 10, 2012 at 11:23 PM, Bill Graham  wrote:
>
>> +1 for me.
>>
>> There's https://issues.apache.org/jira/browse/PIG-2756 which tracks a few
>> documentation issues that should block Pig 0.11, but they can also be done
>> on the trunk and merged to the branch. Gianmarco, you can add a rank
>> subtask there to serve as a reminder.
>>
>>
>> On Wed, Oct 10, 2012 at 11:03 PM, Gianmarco De Francisci Morales <
>> g...@apache.org> wrote:
>>
>> > We are missing some documentation on the RANK but I guess we could add
>> that
>> > to the branch and trunk in parallel.
>> > All the patches I was keeping an eye on are in.
>> >
>> > So +1 for me.
>> > --
>> > Gianmarco
>> >
>> >
>> >
>> > On Wed, Oct 10, 2012 at 5:31 PM, Jonathan Coveney > > >wrote:
>> >
>> > > I think all of the major patches are in, no? Now it's just bug testing?
>> > > Just wanted to touch base on where we are at with this.
>> > >
>> >
>>
>>
>>
>> --
>> *Note that I'm no longer using my Yahoo! email address. Please email me at
>> billgra...@gmail.com going forward.*
>>

[jira] [Created] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query

[Koji Noguchi (JIRA)]

Koji Noguchi created PIG-2970:
-

 Summary: Nested foreach getting incorrect schema when having 
unrelated inner query
 Key: PIG-2970
 URL: https://issues.apache.org/jira/browse/PIG-2970
 Project: Pig
  Issue Type: Bug
  Components: parser
Reporter: Koji Noguchi
Assignee: Koji Noguchi
Priority: Minor


While looking at PIG-2968, hit a weird error message.

{noformat}
$ cat -n test/foreach2.pig
 1  daily = load 'nyse' as (exchange, symbol);
 2  grpd = group daily by exchange;
 3  unique = foreach grpd {
 4  sym = daily.symbol;
 5  uniq_sym = distinct sym;
 6  --ignoring uniq_sym result
 7  generate group, daily;
 8  };
 9  describe unique;
10  zzz = foreach unique generate group;
11  explain zzz;

% pig -x local -t ColumnMapKeyPrune test/foreach2.pig
...
unique: {symbol: bytearray}

2012-10-12 16:55:44,226 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 
1025: 
 Invalid field projection. 
Projected field [group] does not exist in schema: symbol:bytearray.
...
{noformat}



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

[Koji Noguchi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Noguchi updated PIG-2968:
--

Component/s: parser

> ColumnMapKeyPrune fails to prune a subtree inside foreach
> -
>
> Key: PIG-2968
> URL: https://issues.apache.org/jira/browse/PIG-2968
> Project: Pig
>  Issue Type: Bug
>  Components: parser
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Minor
> Attachments: pig-2968-trunk_v01.txt
>
>
> Sample code 
> {noformat}
> $ cat test/foreach.pig 
> daily = load 'nyse' as (exchange, symbol);
> grpd = group daily by exchange;
> uniquecnt = foreach grpd {
> sym = daily.symbol;
> uniq_sym = distinct sym;
> generate group, uniq_sym;
> };
> another = FOREACH uniquecnt GENERATE group;
> explain another;
> {noformat}
> This breaks when it tries to prune uniq_sym->sym->innerload_daily
> bq. 2012-10-12 14:54:11,031 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
> ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1283) COUNT on null bag causes failure

[Anand L Ranganathan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475331#comment-13475331
 ] 

Anand L Ranganathan commented on PIG-1283:
--

I will modify test case and make it as   
assertNull(count.exec(tup));

Also, I will clean up the code (remove tab and add space) and update the patch.


> COUNT on null bag causes failure
> 
>
> Key: PIG-1283
> URL: https://issues.apache.org/jira/browse/PIG-1283
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Thejas M Nair
>Assignee: Anand L Ranganathan
>  Labels: newbie
> Attachments: PIG-1283-1.patch, PIG-1283-2.patch
>
>
> grunt>  l = load '/tmp/e.bag' as (b : bag{t: (i : int)}, a : int);
> # b is null for the only row
> grunt> c = foreach l generate COUNT(b);   
> grunt> dump c   
> It results in following exception-
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while 
> computing count in COUNT
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
> ... 12 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Please hold all commits to Pig trunk.

[Dmitriy Ryaboy]

All clear.

Please remember to commit bug fixes to both trunk and 0.11.
Please keep new features and exploratory stuff out of 0.11.

D

On Fri, Oct 12, 2012 at 1:02 PM, Dmitriy Ryaboy  wrote:
> I am branching the 0.11 branch, will let you know when that's done.
>
> D

[jira] [Created] (PIG-2969) Can't get simple macro with two return values to work

[Lucy Cunningham (JIRA)]

Lucy Cunningham created PIG-2969:


 Summary: Can't get simple macro with two return values to work
 Key: PIG-2969
 URL: https://issues.apache.org/jira/browse/PIG-2969
 Project: Pig
  Issue Type: Bug
  Components: piggybank
Affects Versions: 0.11
Reporter: Lucy Cunningham


Hi, I followed the documentation to try and write a pig macro with two output 
relations, but i'm getting errors:

define mymacro (a, b)
returns {c, d} {
$c = $a;
$d = $b;
};

e = load 'thing' as (x:int);
f = load 'thing' as (x:int);

g,h = mymacro(e,f);

dump g;
dump h;
~   

 
~  

Gives me error: 

ERROR 2999: Unexpected internal error. Undefined parameter : c

java.lang.RuntimeException: Undefined parameter : c
at 
org.apache.pig.tools.parameters.PreprocessorContext.substitute(PreprocessorContext.java:232)
at 
org.apache.pig.tools.parameters.PigFileParser.input(PigFileParser.java:65)
at 
org.apache.pig.tools.parameters.PigFileParser.Parse(PigFileParser.java:43)
at 
org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.parsePigFile(ParameterSubstitutionPreprocessor.java:105)
at 
org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:98)
at org.apache.pig.Main.runParamPreprocessor(Main.java:778)
at org.apache.pig.Main.run(Main.java:568)
at org.apache.pig.Main.main(Main.java:154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

Is this something that I am doing wrong or is there a bug?

Thanks!
Lucy

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

[Koji Noguchi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Noguchi updated PIG-2968:
--

Attachment: pig-2968-trunk_v01.txt

ConcurrentModificationException is happening in ColumnPruneVisitor.removeSubTree

{noformat}
List ll = p.getPredecessors(op);
if (ll != null) { 
for(Operator pred: ll) { 
removeSubTree((LogicalRelationalOperator)pred);
} 
} 
{noformat} 
where List ll is updated within (recursive-)removeSubTree
while being traversed.

Attaching a patch that creates a shallow copy of the list.

For the test, attaching one that will fail with Exception.  But I couldn't come 
up with _expected_ raw query that matches the pruned result.

> ColumnMapKeyPrune fails to prune a subtree inside foreach
> -
>
> Key: PIG-2968
> URL: https://issues.apache.org/jira/browse/PIG-2968
> Project: Pig
>  Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Minor
> Attachments: pig-2968-trunk_v01.txt
>
>
> Sample code 
> {noformat}
> $ cat test/foreach.pig 
> daily = load 'nyse' as (exchange, symbol);
> grpd = group daily by exchange;
> uniquecnt = foreach grpd {
> sym = daily.symbol;
> uniq_sym = distinct sym;
> generate group, uniq_sym;
> };
> another = FOREACH uniquecnt GENERATE group;
> explain another;
> {noformat}
> This breaks when it tries to prune uniq_sym->sym->innerload_daily
> bq. 2012-10-12 14:54:11,031 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
> ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Please hold all commits to Pig trunk.

[Dmitriy Ryaboy]

I am branching the 0.11 branch, will let you know when that's done.

D

[jira] [Commented] (PIG-1283) COUNT on null bag causes failure

[Jonathan Coveney (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475295#comment-13475295
 ] 

Jonathan Coveney commented on PIG-1283:
---

Anand,

Thanks for working on this.

Some feedback:

1. make sure the spacing is good. No tabs, four space indents, etc. I realize a 
lot of existing Pig code is janky, but we try to make sure new stuff doesn't 
introduce that.
2. I don't know if your test case is actually testing the change? I mean, the 
change you made has COUNT return null in the case when you give it a null bag, 
right? So why does it return 0?
3. use assertEquals(expected, actual) instead of assertTrue

> COUNT on null bag causes failure
> 
>
> Key: PIG-1283
> URL: https://issues.apache.org/jira/browse/PIG-1283
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Thejas M Nair
>Assignee: Anand L Ranganathan
>  Labels: newbie
> Attachments: PIG-1283-1.patch, PIG-1283-2.patch
>
>
> grunt>  l = load '/tmp/e.bag' as (b : bag{t: (i : int)}, a : int);
> # b is null for the only row
> grunt> c = foreach l generate COUNT(b);   
> grunt> dump c   
> It results in following exception-
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while 
> computing count in COUNT
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
> ... 12 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2947) Documentation for Rank operator

[JIRA]

 [ 
https://issues.apache.org/jira/browse/PIG-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Avendaño updated PIG-2947:


Attachment: patch_02

> Documentation for Rank operator
> ---
>
> Key: PIG-2947
> URL: https://issues.apache.org/jira/browse/PIG-2947
> Project: Pig
>  Issue Type: Improvement
>Reporter: Allan Avendaño
>Assignee: Allan Avendaño
>Priority: Trivial
>  Labels: documentation
> Attachments: patch_01, patch_02
>
>
> User documentation for recently released Rank operator, with some basic 
> explanation of usage and examples

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

[Koji Noguchi (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475247#comment-13475247
 ] 

Koji Noguchi commented on PIG-2968:
---

Log showing
{noformat}
$ cat /Users/knoguchi/git/pig/pig_1350068049281.logPig Stack Trace
---
ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to 
explain alias anotherat 
org.apache.pig.PigServer.explain(PigServer.java:999)
at 
org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:398)
at 
org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:330)
at 
org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:293)
at 
org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:715)
at 
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:342)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)   
 at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)at 
org.apache.pig.Main.run(Main.java:604)
at org.apache.pig.Main.main(Main.java:154)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: 
Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPruneat 
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122)
at 
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:277)
at org.apache.pig.PigServer.compilePp(PigServer.java:1322)at 
org.apache.pig.PigServer.explain(PigServer.java:984)
... 10 more
Caused by: java.util.ConcurrentModificationException
at 
java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at 
org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.removeSubTree(ColumnPruneVisitor.java:451)
at 
org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.removeSubTree(ColumnPruneVisitor.java:452)
at 
org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.visit(ColumnPruneVisitor.java:431)
at 
org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
at 
org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
at 
org.apache.pig.newplan.logical.rules.ColumnMapKeyPrune$ColumnMapKeyPruneTransformer.transform(ColumnMapKeyPrune.java:141)
at 
org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110)
... 13 more

{noformat}

> ColumnMapKeyPrune fails to prune a subtree inside foreach
> -
>
> Key: PIG-2968
> URL: https://issues.apache.org/jira/browse/PIG-2968
> Project: Pig
>  Issue Type: Bug
>Reporter: Koji Noguchi
>Assignee: Koji Noguchi
>Priority: Minor
>
> Sample code 
> {noformat}
> $ cat test/foreach.pig 
> daily = load 'nyse' as (exchange, symbol);
> grpd = group daily by exchange;
> uniquecnt = foreach grpd {
> sym = daily.symbol;
> uniq_sym = distinct sym;
> generate group, uniq_sym;
> };
> another = FOREACH uniquecnt GENERATE group;
> explain another;
> {noformat}
> This breaks when it tries to prune uniq_sym->sym->innerload_daily
> bq. 2012-10-12 14:54:11,031 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
> ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

[Koji Noguchi (JIRA)]

Koji Noguchi created PIG-2968:
-

 Summary: ColumnMapKeyPrune fails to prune a subtree inside foreach
 Key: PIG-2968
 URL: https://issues.apache.org/jira/browse/PIG-2968
 Project: Pig
  Issue Type: Bug
Reporter: Koji Noguchi
Assignee: Koji Noguchi
Priority: Minor


Sample code 

{noformat}
$ cat test/foreach.pig 
daily = load 'nyse' as (exchange, symbol);
grpd = group daily by exchange;
uniquecnt = foreach grpd {
sym = daily.symbol;
uniq_sym = distinct sym;
generate group, uniq_sym;
};
another = FOREACH uniquecnt GENERATE group;
explain another;

{noformat}

This breaks when it tries to prune uniq_sym->sym->innerload_daily

bq. 2012-10-12 14:54:11,031 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2967) Fix Glob_local test failure for Pig E2E Test Framework

[Sushant Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475221#comment-13475221
 ] 

Sushant Joshi commented on PIG-2967:


The fix adds studenttab20m data in LocalDeployer.pm.  

-- Snippet from the patch file

{
'name' => "studenttab20m",
'filetype' => "studenttab",
'rows' => 2000,
'outfile' => "singlefile/studenttab20m",
},

I have tested this patch on a test grid through a Jenkins build, and the 
checksums match, tests pass.

> Fix Glob_local test failure for Pig E2E Test Framework
> --
>
> Key: PIG-2967
> URL: https://issues.apache.org/jira/browse/PIG-2967
> Project: Pig
>  Issue Type: Test
>  Components: e2e harness
>Affects Versions: 0.10.1
>Reporter: Sushant Joshi
>Priority: Minor
> Fix For: 0.10.1
>
> Attachments: glob_local.patch
>
>
> The Glob_3_local, Glob_4_local, Glob_5_local E2E tests fails due check sum 
> mismatch with benchmark data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2967) Fix Glob_local test failure for Pig E2E Test Framework

[Sushant Joshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sushant Joshi updated PIG-2967:
---

Attachment: glob_local.patch

Patch for fixing Glob_{3,4,5}_local tests. Fix is in LocalDeployer.pm.

> Fix Glob_local test failure for Pig E2E Test Framework
> --
>
> Key: PIG-2967
> URL: https://issues.apache.org/jira/browse/PIG-2967
> Project: Pig
>  Issue Type: Test
>  Components: e2e harness
>Affects Versions: 0.10.1
>Reporter: Sushant Joshi
>Priority: Minor
> Fix For: 0.10.1
>
> Attachments: glob_local.patch
>
>
> The Glob_3_local, Glob_4_local, Glob_5_local E2E tests fails due check sum 
> mismatch with benchmark data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2967) Fix Glob_local test failure for Pig E2E Test Framework

[Sushant Joshi (JIRA)]

Sushant Joshi created PIG-2967:
--

 Summary: Fix Glob_local test failure for Pig E2E Test Framework
 Key: PIG-2967
 URL: https://issues.apache.org/jira/browse/PIG-2967
 Project: Pig
  Issue Type: Test
  Components: e2e harness
Affects Versions: 0.10.1
Reporter: Sushant Joshi
Priority: Minor
 Fix For: 0.10.1


The Glob_3_local, Glob_4_local, Glob_5_local E2E tests fails due check sum 
mismatch with benchmark data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Cutting Pig-11 branch at 1pm PST

[Dmitriy Ryaboy]

I will begin branching Pig 0.11 around 1pm PST, in about 2 hours.
I will send another email when I start, and another when I finish;
lease refrain from committing any patches between those two messages.

-Dmitriy

[jira] [Updated] (PIG-1283) COUNT on null bag causes failure

[Dmitriy V. Ryaboy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy V. Ryaboy updated PIG-1283:
---

Status: Patch Available  (was: Open)

marking as patch available

> COUNT on null bag causes failure
> 
>
> Key: PIG-1283
> URL: https://issues.apache.org/jira/browse/PIG-1283
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Thejas M Nair
>Assignee: Anand L Ranganathan
>  Labels: newbie
> Attachments: PIG-1283-1.patch, PIG-1283-2.patch
>
>
> grunt>  l = load '/tmp/e.bag' as (b : bag{t: (i : int)}, a : int);
> # b is null for the only row
> grunt> c = foreach l generate COUNT(b);   
> grunt> dump c   
> It results in following exception-
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while 
> computing count in COUNT
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
> ... 12 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2965) RANDOM should allow seed initialization for ease of testing

[Jonathan Coveney (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Coveney resolved PIG-2965.
---

Resolution: Fixed

> RANDOM should allow seed initialization for ease of testing
> ---
>
> Key: PIG-2965
> URL: https://issues.apache.org/jira/browse/PIG-2965
> Project: Pig
>  Issue Type: Bug
>Reporter: Aneesh Sharma
>Assignee: Dmitriy V. Ryaboy
>  Labels: newbie
> Attachments: PIG-2965-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2965) RANDOM should allow seed initialization for ease of testing

[Dmitriy V. Ryaboy (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475176#comment-13475176
 ] 

Dmitriy V. Ryaboy commented on PIG-2965:


+1

> RANDOM should allow seed initialization for ease of testing
> ---
>
> Key: PIG-2965
> URL: https://issues.apache.org/jira/browse/PIG-2965
> Project: Pig
>  Issue Type: Bug
>Reporter: Aneesh Sharma
>Assignee: Dmitriy V. Ryaboy
>  Labels: newbie
> Attachments: PIG-2965-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk #1333

[Apache Jenkins Server]

See 

Changes:

[billgraham] PIG-2964: Add helper method getJobList() to PigStats.JobGraph. 
Extend visibility of couple methods on same class (prkommireddi via billgraham)

[sms] PIG-2579: Support for multiple input schemas in AvroStorage (cheolsoo via 
sms)

--
[...truncated 35907 lines...]
[junit] at 
org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:71)
[junit] at 
org.apache.hadoop.hdfs.server.datanode.FSDataset.shutdown(FSDataset.java:1934)
[junit] at 
org.apache.hadoop.hdfs.server.datanode.DataNode.shutdown(DataNode.java:788)
[junit] at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdownDataNodes(MiniDFSCluster.java:566)
[junit] at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:550)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:87)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:141)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 12/10/12 10:36:07 WARN datanode.FSDatasetAsyncDiskService: 
AsyncDiskService has already shut down.
[junit] 12/10/12 10:36:07 INFO mortbay.log: Stopped 
SelectChannelConnector@localhost:0
[junit] 12/10/12 10:36:07 INFO ipc.Server: Stopping server on 60319
[junit] 12/10/12 10:36:07 INFO ipc.Server: IPC Server handler 1 on 60319: 
exiting
[junit] 12/10/12 10:36:07 INFO ipc.Server: IPC Server handler 2 on 60319: 
exiting
[junit] 12/10/12 10:36:07 INFO ipc.Server: IPC Server handler 0 on 60319: 
exiting
[junit] 12/10/12 10:36:07 INFO ipc.Server: Stopping IPC Server Responder
[junit] 12/10/12 10:36:07 INFO metrics.RpcInstrumentation: shut down
[junit] 12/10/12 10:36:07 INFO ipc.Server: Stopping IPC Server listener on 
60319
[junit] 12/10/12 10:36:07 INFO datanode.DataNode: Waiting for threadgroup 
to exit, active threads is 1
[junit] 12/10/12 10:36:07 WARN datanode.DataNode: 
DatanodeRegistration(127.0.0.1:58890, 
storageID=DS-1072995515-67.195.138.20-58890-1350037664961, infoPort=60166, 
ipcPort=60319):DataXceiveServer:java.nio.channels.AsynchronousCloseException
[junit] at 
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:185)
[junit] at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:159)
[junit] at 
sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:84)
[junit] at 
org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:131)
[junit] at java.lang.Thread.run(Thread.java:662)
[junit] 
[junit] 12/10/12 10:36:07 INFO datanode.DataNode: Exiting DataXceiveServer
[junit] 12/10/12 10:36:08 INFO datanode.DataBlockScanner: Exiting 
DataBlockScanner thread.
[junit] 12/10/12 10:36:08 INFO datanode.DataNode: Waiting for threadgroup 
to exit, active threads is 0
[junit] 12/10/12 10:36:08 INFO datanode.DataNode: 
DatanodeRegistration(127.0.0.1:58890, 
storageID=DS-1072995515-67.195.138.20-58890-1350037664961, infoPort=60166, 
ipcPort=60319):Finishing DataNode in: 
FSDataset{dirpath='
[junit] 12/10/12 10:36:08 INFO ipc.Server: Stopping server on 60319
[junit] 12/10/12 10:36:08 INFO metrics.RpcInstrumentation: shut down
[junit] 12/10/12 10:3

[jira] [Updated] (PIG-1283) COUNT on null bag causes failure

[Anand L Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anand L Ranganathan updated PIG-1283:
-

Attachment: PIG-1283-2.patch

Please find the patch for this bug

> COUNT on null bag causes failure
> 
>
> Key: PIG-1283
> URL: https://issues.apache.org/jira/browse/PIG-1283
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Thejas M Nair
>Assignee: Anand L Ranganathan
>  Labels: newbie
> Attachments: PIG-1283-1.patch, PIG-1283-2.patch
>
>
> grunt>  l = load '/tmp/e.bag' as (b : bag{t: (i : int)}, a : int);
> # b is null for the only row
> grunt> c = foreach l generate COUNT(b);   
> grunt> dump c   
> It results in following exception-
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while 
> computing count in COUNT
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
> ... 12 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Work stopped] (PIG-1283) COUNT on null bag causes failure

[Anand L Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on PIG-1283 stopped by Anand L Ranganathan.

> COUNT on null bag causes failure
> 
>
> Key: PIG-1283
> URL: https://issues.apache.org/jira/browse/PIG-1283
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Reporter: Thejas M Nair
>Assignee: Anand L Ranganathan
>  Labels: newbie
> Attachments: PIG-1283-1.patch, PIG-1283-2.patch
>
>
> grunt>  l = load '/tmp/e.bag' as (b : bag{t: (i : int)}, a : int);
> # b is null for the only row
> grunt> c = foreach l generate COUNT(b);   
> grunt> dump c   
> It results in following exception-
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while 
> computing count in COUNT
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
> at 
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
> Caused by: java.lang.NullPointerException
> at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
> ... 12 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira