GitHub user michaelkamprath opened a pull request:
https://github.com/apache/spark/pull/16403
[SPARK-18819][CORE] Double byte alignment on ARM platforms
## What changes were proposed in this pull request?
On 32-bit ARM platforms, doubles require 8-byte alignment. The use of
Java's `sun.misc.Unsafe` library for memory block management exposes Spark to
misaligned doubles when using `org.apache.spark.unsafe.Platform.getDouble()` to
extract a double from a memory block with arbitrary offset locations. For the
alignment sensitive platforms, this proposed change creates an properly aligned
buffer that the bytes to be cast to a double are first copied to, then the
double the double is extracted via the `sun.misc.Unsafe.getDouble()` method. On
all other platforms the previous behavior is maintained, which was to cast the
double value from the memory block regardless of offset alignment .
## How was this patch tested?
All unit tests pass, plus tested on both x86 and ARM71 platforms. For the
ARM71 platform, it is verified that this fix removes the problem as described
in SPARK-18819.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/michaelkamprath/spark
fix/SPARK-18819_double-alignment
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16403.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16403
----
commit 30c6c997ca8e864a31d2f4dfa55d47b5aa629596
Author: Michael Kamprath <mich...@claireware.com>
Date: 2016-12-25T21:54:26Z
[SPARK-18819] Double alignment on ARM
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
