Victor created AVRO-3768:
----------------------------
Summary: Provide InputStream implementation that wraps an Iterator
of Record
Key: AVRO-3768
URL: https://issues.apache.org/jira/browse/AVRO-3768
Project: Apache Avro
Issue Type: New Feature
Components: java
Reporter: Victor
Hello,
I have some code that generates avro record (using GenericData.Record
precisely) record per record (I used an Iterator<GenericData.Record> in
practice, but we could imagine anything similar including an actual
java.util.Stream or even an avro-provided interface), and I would like to
serialized it to some external system (an http request in my particular case).
So basically the data is generated as it is pulled.
Right now, the only option I found is to use a combination of
java.io.PipedInputStream and PipedOutputStream, wrapping the later inside a
DataFileWriter and then feeding the records to the writer (in a separate
thread) so that the PipedInputStream can be read by anything else.
As you can see this is a bit cumbersome and a more straightforward approach
would be welcomed. I tried to implement this myself but I admit I got lost in
all the moving pieces and I couldn't find a simple way of doing that without
implementing a lot of low-level stuff. So maybe I'm missing something or we
could add something to avro :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
