[ https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327513#comment-16327513 ]
ASF GitHub Bot commented on DRILL-6002: --------------------------------------- Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1058#discussion_r161842736 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/spill/SpillSet.java --- @@ -104,9 +107,17 @@ * nodes provide insufficient local disk space) */ + // The buffer size is calculated as LCM of the Hadoop internal checksum buffer (9 * checksum length), where --- End diff -- @paul-rogers Changed `TRANSFER_SIZE` to 72K. It is calculated as LCM between Hadoop internal checksum buffer and MapR FS page size to make writes aligned on internal buffer boundaries. > Avoid memory copy from direct buffer to heap while spilling to local disk > ------------------------------------------------------------------------- > > Key: DRILL-6002 > URL: https://issues.apache.org/jira/browse/DRILL-6002 > Project: Apache Drill > Issue Type: Improvement > Reporter: Vlad Rozov > Assignee: Vlad Rozov > Priority: Major > > When spilling to a local disk or to any file system that supports > WritableByteChannel it is preferable to avoid copy from off-heap to java heap > as WritableByteChannel can work directly with the off-heap memory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)