GitHub user ConeyLiu opened a pull request:
https://github.com/apache/spark/pull/20026
[SPARK-22838][Core] Avoid unnecessary copying of data
## What changes were proposed in this pull request?
If we read data from FileChannel to HeapByteBuffer, there is a need to copy
the data from the off-heap to the on-heap, you can see the follow code:
```java
static int read(FileDescriptor var0, ByteBuffer var1, long var2,
NativeDispatcher var4) throws IOException {
if(var1.isReadOnly()) {
throw new IllegalArgumentException("Read-only buffer");
} else if(var1 instanceof DirectBuffer) {
return readIntoNativeBuffer(var0, var1, var2, var4);
} else {
ByteBuffer var5 = Util.getTemporaryDirectBuffer(var1.remaining());
int var7;
try {
int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
var5.flip();
if(var6 > 0) {
var1.put(var5);
}
var7 = var6;
} finally {
Util.offerFirstTemporaryDirectBuffer(var5);
}
return var7;
}
}
```
## How was this patch tested?
Existing UT.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ConeyLiu/spark datacopy
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20026.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20026
----
commit 27d2b1b3991602a7126d85cfc69c3da1ff84d599
Author: Xianyang Liu <xianyang.liu@...>
Date: 2017-12-20T02:32:44Z
small fix
commit 51c32c8c6cc89a3249b3ef856c41f3c238b59f4a
Author: Xianyang Liu <xianyang.liu@...>
Date: 2017-12-20T03:44:00Z
fix code style
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
