Since the exception is thrown from java.io.FileInputStream#open, it's
trying to append to a local file, not one in HDFS.
You're passing 'new File(...)' to appendTo, when you should probably
be passing 'new FsInput(...)'.
Doug
On Mon, Jul 8, 2013 at 9:29 AM, TrevniUser dipti.de...@cerner.com
Thanks for replying. You are correct. I followed this example
https://gist.github.com/QwertyManiac/4724582
--
View this message in context:
http://apache-avro.679487.n3.nabble.com/Is-it-possible-to-append-to-an-already-existing-avro-file-tp3762049p4027789.html
Sent from the Avro - Users
I assume by non-trivial you meant the extra Seekable stuff I needed to
wrap around the DFS output streams to let Avro take it as append-able?
I don't think its possible for Avro to carry it since Avro (core) does
not reverse-depend on Hadoop. Should we document it somewhere though?
Do you have any
I confess to being a user of rather than a developer of open source, but
perhaps you could elaborate on what depends on means and what the
consequences are?
Isn't it -- or couldn't it be made -- a run-time binding, so that only those
who try to use the HDFS append functionality would be
The avro-mapred module includes a Seekable implementation that works
with HDFS called FsInput:
http://avro.apache.org/docs/current/api/java/org/apache/avro/mapred/FsInput.html
With this, your example can be made considerably smaller.
Doug
On Thu, Feb 7, 2013 at 8:28 AM, Harsh J
Thanks so much for the code -- it works great!
Since it is a non-trivial amount of code required to achieve append, I suggest
attaching that code to AVRO-1035, in the hopes that someone will come up with
an interface that requires just one line of user code to achieve append.
--- On Wed,
It will work on an OutputStream that supports append.
http://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileWriter.html#appendTo(org.apache.avro.file.SeekableInput,
java.io.OutputStream)
So it depends on how well HDFS implements FileSystem#append(), not on
any changes in
I don't believe a Hadoop FileSystem is a Java OutputStream?
--- On Tue, 2/5/13, Doug Cutting cutt...@apache.org wrote:
From: Doug Cutting cutt...@apache.org
Subject: Re: Is it possible to append to an already existing avro file
To: user@avro.apache.org
Date: Tuesday, February 5, 2013, 5:27
