David Dreyfus created PIG-4012:
----------------------------------
Summary: java.lang.IllegalArgumentException: Comparison method
violates its general contract! SpillableMemoryManager
Key: PIG-4012
URL: https://issues.apache.org/jira/browse/PIG-4012
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.12.0
Environment: java version "1.7.0_60-ea"
Java(TM) SE Runtime Environment (build 1.7.0_60-ea-b02)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b04, mixed mode)
Reporter: David Dreyfus
java.lang.IllegalArgumentException: Comparison method violates its general
contract!
at java.util.TimSort.mergeHi(TimSort.java:868)
at java.util.TimSort.mergeAt(TimSort.java:485)
at java.util.TimSort.mergeForceCollapse(TimSort.java:426)
at java.util.TimSort.sort(TimSort.java:223)
at java.util.TimSort.sort(TimSort.java:173)
at java.util.Arrays.sort(Arrays.java:659)
at java.util.Collections.sort(Collections.java:217)
at
org.apache.pig.impl.util.SpillableMemoryManager.handleNotification(SpillableMemoryManager.java:199)
at
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:156)
at sun.management.MemoryImpl.createNotification(MemoryImpl.java:168)
at
sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:301)
at sun.management.Sensor.trigger(Sensor.java:137)
>From SpillableMemoryManager.java:
/**
* We don't lock anything, so this sort may not be stable if a
WeakReference suddenly
* becomes null, but it will be close enough.
* Also between the time we sort and we use these spillables,
they
* may actually change in size - so this is just best effort
*/
Issue may be due to Java 7 and reporting vs ignoring the exception.
Trying
-Djava.util.Arrays.useLegacyMergeSort=true
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6804124
suggests the newer MergeSort is much faster.
Someone may want to make the sorting stable in SpillableMemoryManager so that
the new merge sort can be used without failure.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
