Re: Pig 0.11.1 OutOfMemory error

2013-09-09 Thread Shubham Chopra
Hi Koji/Rohini/Bill, I tried the same script with Apache Pig version 0.10.1 (r1426677) compiled Dec 28 2012, 16:46:13. The script compiled without problems. Increasing heap size didn't help with Pig-0.11. I have opened a jira for this issue (https://issues.apache.org/jira/browse/PIG-3455) Thanks

Re: Pig 0.11.1 OutOfMemory error

2013-09-06 Thread Bill Graham
The getSignature method basically generates a string representation of the logical plan and they computes it's hash. In your case it seems the logical plan is too large for the amount of memory you have. Try increasing the heap even more. On Fri, Sep 6, 2013 at 1:10 PM, Koji Noguchi wrote: > Se

Re: Pig 0.11.1 OutOfMemory error

2013-09-06 Thread Rohini Palaniswamy
I think we should fix it in pig if it is a regression from pig 0.10. Shubam, If the script works fine for you in pig 0.10, can you open a jira for the issue with 0.11 ? Regards, Rohini On Fri, Sep 6, 2013 at 1:51 PM, Bill Graham wrote: > The getSignature method basically generates a string

Re: Pig 0.11.1 OutOfMemory error

2013-09-06 Thread Koji Noguchi
Seems to be happening inside the method introduced in 0.11 "org.apache.pig.newplan.logical.relational.LogicalPlan.getSignature" https://issues.apache.org/jira/browse/PIG-2587 Maybe a coincidence but can we ask Bill to help us? Shubham, can you try your query on pig 0.10.* and see if you don't h

Pig 0.11.1 OutOfMemory error

2013-09-04 Thread Shubham Chopra
Hi, I have a relatively large pig scripts (around 1.5k lines, 85 assignments). Around 150 columns are getting projected, joined, grouped and aggregated ending in multiple stores. Pig 0.11.1 fails with the following error even before any jobs are fired: Pig Stack Trace --- ERROR 2998: