[ https://issues.apache.org/jira/browse/PIG-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044888#comment-15044888 ]
Colonel.Hou commented on PIG-282: --------------------------------- Daniel Dai, thank you for your answer, all exception is below: Backend error message --------------------- Error: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer at com.compcc.ps.pig.TotalSortPartitioner.getPartition(TotalSortPartitioner.java:21) at com.compcc.ps.pig.TotalSortPartitioner.getPartition(TotalSortPartitioner.java:12) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:692) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:135 ) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) My write partition is : public class TotalSortPartitioner extends Partitioner<PigNullableWritable, Writable> { public static final Log log = LogFactory.getLog(TotalSortPartitioner.class); public static final int ONE_HOUR_MINUTE = 60; public int getPartition(PigNullableWritable key, Writable value, int numPartitions) { int minute = -1; try { minute = Integer.parseInt(key.getValueAsPigType().toString()); } catch (RuntimeException e) { log.error("Convert String minute to Integer error. " + e.getMessage()); return minute; } ... return xxx; } } > Custom Partitioner > ------------------ > > Key: PIG-282 > URL: https://issues.apache.org/jira/browse/PIG-282 > Project: Pig > Issue Type: New Feature > Affects Versions: 0.7.0 > Reporter: Amir Youssefi > Assignee: Aniket Mokashi > Priority: Minor > Fix For: 0.8.0 > > Attachments: CustomPartitioner.patch, CustomPartitionerFinale.patch, > CustomPartitionerTest.patch > > > By adding custom partitioner we can give control over which output partition > a key (/value) goes to. We can add keywords to language e.g. > PARTITION BY UDF(...) > or a similar syntax. UDF returns a number between 0 and n-1 where n is number > of output partitions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)