[ https://issues.apache.org/jira/browse/MAPREDUCE-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph Shraibman resolved MAPREDUCE-2564. ----------------------------------------- Resolution: Invalid Fix Version/s: 0.20.203.0 > NullPointerException in WritableComparator > ------------------------------------------ > > Key: MAPREDUCE-2564 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2564 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: task > Affects Versions: 0.20.203.0 > Environment: java version "1.6.0_23" > Java(TM) SE Runtime Environment (build 1.6.0_23-b05) > Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode) > hadoop-0.20.203.0rc1 > Reporter: Joseph Shraibman > Priority: Blocker > Fix For: 0.20.203.0 > > > java.lang.NullPointerException > at > org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:96) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1110) > at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70) > at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1398) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1297) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:698) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:765) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) > It is easy to see why this is happening. The WritableComparator is created > in JobConf line 776: > {code:title=JobConf.java} > return > WritableComparator.get(getMapOutputKeyClass().asSubclass(WritableComparable.class)); > } > {code} > which calls > {code:title=WritableComparator.java|borderStyle=solid} > protected WritableComparator(Class<? extends WritableComparable> keyClass) { > this(keyClass, false); > } > protected WritableComparator(Class<? extends WritableComparable> keyClass, > boolean createInstances) { > this.keyClass = keyClass; > if (createInstances) { > key1 = newKey(); > key2 = newKey(); > buffer = new DataInputBuffer(); > } else { > key1 = key2 = null; > buffer = null; > } > } > {code} > key1, key2, and buffer end up being null. When compare() is called the NPE is > thrown because buffer is null > {code:title=WritableComparator.java|borderStyle=solid} > public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) { > try { > buffer.reset(b1, s1, l1); // parse key1 > key1.readFields(buffer); > > buffer.reset(b2, s2, l2); // parse key2 > key2.readFields(buffer); > > } catch (IOException e) { > throw new RuntimeException(e); > } > > return compare(key1, key2); // compare them > } > {code} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira