The various Avro writer / readers are not thread safe. You will need to do
some sort of external synchronization. If the threads are in the same JVM,
the easiest way to write from multiple threads safely will be to
synchronize on the DataFileStream instance.
e.g.
synchronized(myDataFileWriter) {
myDataFileWriter.append(datum);
}
On Fri, Mar 13, 2015 at 1:05 PM, Shruthi Jeganathan <
shruthi.jeganat...@tapjoy.com> wrote:
> Hi,
>
> I have multiple threads writing to same avro output file(out.avro). When
> deserializing out.avro, I get this exception:
>
> org.apache.avro.AvroRuntimeException: java.io.IOException: Invalid sync!
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:210)
> at com.example.Main.deserialize(Main.java:80)
> at com.example.Main.main(Main.java:50)Caused by: java.io.IOException:
> Invalid sync!
> at
> org.apache.avro.file.DataFileStream.nextRawBlock(DataFileStream.java:293)
> at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:198)
> ... 2 more
>
> Is this because I'm concurrently writing to out.avro? If it's an issue, is
> there a way for multiple threads to simultaneously write to out.avro?
> Please provide code samples, if possible.
>
> Thanks.
>
>
--
Sean
