Thanks for fixing this Jon. It works now. Thanks -- Prasanth
On Oct 8, 2012, at 2:15 PM, Jonathan Coveney <jcove...@gmail.com> wrote: > Prashanth, > > The fix is in. Please let me know if you have any more issues > > Jon > > 2012/10/7 Prasanth J <buckeye.prasa...@gmail.com> > >> Thanks Jon. >> >> Thanks >> -- Prasanth >> >> On Oct 7, 2012, at 11:46 PM, Jon Coveney <jcove...@gmail.com> wrote: >> >>> My guess is this is my doing. Will tackle this tomorrow. >>> >>> Sent from my iPhone >>> >>> On Oct 7, 2012, at 8:15 PM, Prasanth J <buckeye.prasa...@gmail.com> >> wrote: >>> >>>> Hello everyone >>>> >>>> I recently updated my patch(PIG-2831) to work on the latest trunk and >> noticed that new trunk breaks my code. My code uses secondary sort on a >> tuple inside a databag. The issue seems to be related to BinInterSedes >> comparator. I have pasted the stack trace for reference. Can anyone let me >> know if this is side-effect of some other patch that is committed recently? >>>> >>>> java.io.IOException: Spill failed >>>> at >> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1029) >>>> at >> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691) >>>> at >> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) >>>> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:123) >>>> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) >>>> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) >>>> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) >>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) >>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) >>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) >>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:396) >>>> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) >>>> at org.apache.hadoop.mapred.Child.main(Child.java:249) >>>> Caused by: java.lang.RuntimeException: Unexpected data type 39 found in >> stream. >>>> at >> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.getGeneralizedDataType(BinInterSedes.java:1166) >>>> at >> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:911) >>>> at >> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:730) >>>> at >> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695) >>>> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78) >>>> at >> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111) >>>> at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30) >>>> at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83) >>>> at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59) >>>> at >> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399) >>>> at >> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:853) >>>> at >> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1344) >>>> >>>> >>>> Thanks >>>> -- Prasanth >>>> >> >>
