I assume you tested 2.0 with SPARK-12181 . Related code from Platform.java if java.nio.Bits#unaligned() throws exception:
// We at least know x86 and x64 support unaligned access. String arch = System.getProperty("os.arch", ""); //noinspection DynamicRegexReplaceableByCompiledPattern _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64)$"); Can you give us some detail on how the code runs for JDKs on zSystems ? Thanks On Fri, Apr 15, 2016 at 7:01 AM, Adam Roberts <arobe...@uk.ibm.com> wrote: > Hi, I'm testing Spark 2.0.0 on various architectures and have a question, > are we sure if > core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java > <https://github.com/apache/spark/blob/96941b12f8b465df21423275f3cd3ade579b4fa1/core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java> > really is attempting to use unaligned memory access (for the > BytesToBytesMapOffHeapSuite tests specifically)? > > Our JDKs on zSystems for example return false for the > java.nio.Bits.unaligned() method and yet if I skip this check and add s390x > to the supported architectures (for zSystems), all thirteen tests here > pass. > > The 13 tests here all fail as we do not pass the unaligned requirement > (but perhaps incorrectly): > > core/src/test/java/org/apache/spark/unsafe/map/BytesToBytesMapOffHeapSuite.java > <https://github.com/apache/spark/blob/d6dc12ef0146ae409834c78737c116050961f350/core/src/test/java/org/apache/spark/unsafe/map/BytesToBytesMapOffHeapSuite.java> > and I know the unaligned checking is at > common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java > <https://github.com/apache/spark/blob/master/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java> > > Either our JDK's method is returning false incorrectly or this test isn't > using unaligned memory access (so the requirement is invalid), there's no > mention of alignment in the test itself. > > Any guidance would be very much appreciated, cheers > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >