[ 
https://issues.apache.org/jira/browse/AVRO-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800778#action_12800778
 ] 

Doug Cutting commented on AVRO-327:
-----------------------------------

> If we have a stream with metadata and data (like our DataFile) and if the 
> metadata is encoded in non-avro format we'll have trouble with buffering.

The current DataFile implementation uses a Decoder for all input, so the only 
problem there would be that we'd need to reset the buffer when seeking the 
underlying input stream, which would not be hard.

More generally, since BinaryDecoder has a read(byte[]) method, it is possible 
to implement a Java InputStream on top of a Decoder if one needed to perform 
other non-Avro input from the same source.  So I cannot currently imagine an 
application that could not work with this.  Am I missing something?

So, we have two candidate approaches:
 - Add a new class, BufferredBinaryDecoder, and change internal uses (like 
DataFileReader) to use it, but retain the non-buffering BinaryDecoder for 
applications that need to intermix Avro and other input.
 - add buffering to BinaryDecoder and perhaps provide a non-buffering 
InputStream implementation that wraps a BinaryDecoder, for applications that 
intermix Avro and other data.

I'm currently leaning towards the latter approach.

ValidatingDecoder and ResovlingDecoder each wrap a Decoder.  If they're 
wrapping a BinaryDecoder that buffers, and an application intermixes Avro and 
non-Avro data, then that application would need to use the buffered 
BinaryDecoder for its input, not the ValidatingDecoder or the ResolvingDecoder, 
since those accept only call sequences in accord with a declared schema.






> Performance improvements to BinaryDecoder.readLong()
> ----------------------------------------------------
>
>                 Key: AVRO-327
>                 URL: https://issues.apache.org/jira/browse/AVRO-327
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>            Reporter: Thiruvalluvan M. G.
>            Assignee: Thiruvalluvan M. G.
>
> AVRO-315 proposed performance improvements to readLong(), readFloat() and 
> readDouble(). readLong() did not improve performance well for all. Scott 
> proposed a better method (but requires a change in semantics and API). We'll 
> carry on the discussion on that proposal here. AVRO-315 will be committed 
> with changes for readFloat() and readDouble().

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to