GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/20191
[SPARK-22997] Add additional defenses against use of freed MemoryBlocks
## What changes were proposed in this pull request?
This patch modifies Spark's `MemoryAllocator` implementations so that
`free(MemoryBlock)` mutates the passed block to clear pointers (in the off-heap
case) or null out references to backing `long[]` arrays (in the on-heap case).
The goal of this change is to add an extra layer of defense against
use-after-free bugs because currently it's hard to detect corruption caused by
blind writes to freed memory blocks.
## How was this patch tested?
New unit tests in `PlatformSuite`, including new tests for existing
functionality because we did not have sufficient mutation coverage of the
on-heap memory allocator's pooling logic.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark
SPARK-22997-add-defenses-against-use-after-free-bugs-in-memory-allocator
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20191.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20191
----
commit a7f8c07fb5158f39bbb6cc1f23cfb13a0d473536
Author: Josh Rosen <joshrosen@...>
Date: 2018-01-08T23:50:18Z
Add additional defenses against use of freed MemoryBlocks
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
