[
https://issues.apache.org/jira/browse/AVRO-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807183#comment-16807183
]
ASF subversion and git services commented on AVRO-813:
------------------------------------------------------
Commit a5e0f4ad01703eb4a724e71dfcf99306819c6bf3 in avro's branch
refs/heads/master from Daniel Kulp
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=a5e0f4a ]
[AVRO-813] Keep the InputStream contract of -1 for eof
Patch from Bruno Dumon applied
> EOFException is thrown during normal operation
> ----------------------------------------------
>
> Key: AVRO-813
> URL: https://issues.apache.org/jira/browse/AVRO-813
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Reporter: Bruno Dumon
> Assignee: Bruno Dumon
> Priority: Major
> Labels: memex
> Fix For: 1.9.0
>
> Attachments: avro-813-patch.txt
>
>
> In an application that uses Avro as RPC mechanism (with the NettyTransceiver,
> but that's irrelevant), I've noticed in jprofiler that during normal
> operation quite some time was spent creating EOFExceptions:
> {noformat}
> 5.4% - 2,004 ms org.apache.avro.ipc.generic.GenericResponder.readRequest
> 5.0% - 1,871 ms org.apache.avro.generic.GenericDatumReader.read
> 4.9% - 1,832 ms org.apache.avro.generic.GenericDatumReader.read
> 4.9% - 1,832 ms org.apache.avro.generic.GenericDatumReader.readRecord
> 4.5% - 1,670 ms org.apache.avro.generic.GenericDatumReader.read
> 4.5% - 1,670 ms org.apache.avro.generic.GenericDatumReader.readRecord
> 4.3% - 1,596 ms org.apache.avro.generic.GenericDatumReader.read
> 2.8% - 1,048 ms org.apache.avro.generic.GenericDatumReader.readArray
> 1.3% - 477 ms org.apache.avro.io.ValidatingDecoder.arrayNext
> 1.3% - 471 ms org.apache.avro.io.BinaryDecoder.arrayNext
> 1.3% - 466 ms org.apache.avro.io.BinaryDecoder.doReadItemCount
> 1.3% - 466 ms org.apache.avro.io.BinaryDecoder.readLong
> 1.3% - 466 ms org.apache.avro.io.BinaryDecoder.ensureBounds
> 1.3% - 466 ms org.apache.avro.io.BinaryDecoder$ByteSource.compactAndFill
> 1.3% - 466 ms
> org.apache.avro.io.BinaryDecoder$InputStreamByteSource.tryReadRaw
> 1.3% - 466 ms org.apache.avro.util.ByteBufferInputStream.read
> 1.3% - 466 ms org.apache.avro.util.ByteBufferInputStream.getBuffer
> 1.3% - 466 ms java.io.EOFException.<init>
> 1.3% - 466 ms java.io.IOException.<init>
> 1.2% - 460 ms java.lang.Exception.<init>
> 1.2% - 460 ms java.lang.Throwable.<init>
> 1.2% - 460 ms java.lang.Throwable.fillInStackTrace
> {noformat}
> These exceptions are produced by the ByteBufferInputStream (which modifies
> InputStream's contract: return -1 at eof), but are catched higher up by the
> tryReadRaw method.
> What happens is this:
> The message in question has an (empty) array at the end of its message, thus
> the reader tries to read the size of this array in BinaryDecoder.readLong.
> This calls ensureBounds(10), whose contract is that it should read 10 bytes
> if they are available, and otherwise be quiet. ensureBounds calls via
> compactAndFill the tryReadRaw method. It is this method which catches the
> EOFException, because it only 'tries' to read so many bytes.
> Note that InputStreamByteSource.readRaw (without the 'try' part) does itself
> check if read < 0 in order to throw EOFException, making the throwing of
> EOFException in ByteBufferInputStream unnecessary (for this particular usage).
> There was some talk about EOFException in AVRO-392 too, though it seems this
> particular common case was not mentioned there. When using Avro RPC, or more
> in general, when using Avro to read small messages rather than large files,
> it seems like one can very easily run into this EOFException situation, which
> hurts performance.
> I'll attach a patch which simply removes the throwing of EOFException in
> ByteBufferInputStream, but this will likely break other cases which rely on
> the EOFException being thrown (haven't researched this to the bottom).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
