[
https://issues.apache.org/jira/browse/DRILL-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877425#comment-15877425
]
ASF GitHub Bot commented on DRILL-5275:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/754#discussion_r102378905
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
---
@@ -57,6 +57,12 @@
private BatchSchema.SelectionVectorMode svMode =
BatchSchema.SelectionVectorMode.NONE;
private SelectionVector2 sv2;
+ /**
+ * Disk I/O buffer used for all reads and writes of DrillBufs.
+ */
+
+ private byte buffer[] = new byte[32*1024];
--- End diff --
Read a 18 GB file on disk using just a test program that uses various size
buffers.
{code}
32K buffer: Rate: 799 MB/s
64K buffer: Rate: 766 MB/s
{code}
So, seems no advantage of a larger buffer. (Tests with a smaller buffer do
slow things down, hence the 32K size.)
On direct memory: can't use direct memory as the fundamental problem is
that data is in a direct memory DrillBuf, and must be copied to heap memory for
writing. The original code does the copy by allocating a heap buffer the same
size as the vector (16 MB, 32 MB or larger.) This code does the copying by
reusing the same buffer over and over.
No need to hold the buffer on the operator. This class is used for an
entire spill/read session.
What may be an issue, however, is the merge phase of a sort when many files
are open and so many buffers are created. The reads are synchronous, so they
could share a buffer.
> Sort spill serialization is slow due to repeated buffer allocations
> -------------------------------------------------------------------
>
> Key: DRILL-5275
> URL: https://issues.apache.org/jira/browse/DRILL-5275
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Fix For: 1.10.0
>
>
> Drill provides a sort operator that spills to disk. The spill and read
> operations use the serialization code in the
> {{VectorAccessibleSerializable}}. This code, in turn, uses the
> {{DrillBuf.getBytes()}} method to write to an output stream. (Yes, the "get"
> method writes, and the "write" method reads...)
> The DrillBuf method turns around and calls the UDLE method that does:
> {code}
> byte[] tmp = new byte[length];
> PlatformDependent.copyMemory(addr(index), tmp, 0, length);
> out.write(tmp);
> {code}
> That is, for each write the code allocates a heap buffer. Since Drill buffers
> can be quite large (4, 8, 16 MB or larger), the above rapidly fills the heap
> and causes GC.
> The result is slow performance. On a Mac, with an SSD that can do 700 MB/s of
> I/O, we get only about 40 MB/s. Very likely because of excessive CPU cost and
> GC.
> The solution is to allocate a single read or write buffer, then use that same
> buffer over and over when reading or writing. This must be done in
> {{VectorAccessibleSerializable}} as it is a per-thread class that has
> visibility to all the buffers to be written.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
