Since the exception is thrown from java.io.FileInputStream#open, it's trying to append to a local file, not one in HDFS.
You're passing 'new File(...)' to appendTo, when you should probably be passing 'new FsInput(...)'. Doug On Mon, Jul 8, 2013 at 9:29 AM, TrevniUser <dipti.de...@cerner.com> wrote: > I was following this thread for a problem I am facing while using > SortedKeyValueFiles. > > Below is the piece of code that tries to obtain the appropriate writer based > on whether I am appending or creating a new file: > > OutputStream dataOutputStream; > if (!fileSystem.exists(dataFilePath)) { > dataOutputStream = fileSystem.create(dataFilePath); > mDataFileWriter = new > DataFileWriter<GenericRecord>(datumWriter).setSyncInterval(1 << > 20).create(mRecordSchema, dataOutputStream); > } else { > dataOutputStream = fileSystem.append(dataFilePath); > mDataFileWriter = new > DataFileWriter<GenericRecord>(datumWriter).setSyncInterval(1 << > 20).appendTo(new File(options.getPath() + DATA_FILENAME)); > } > > but it fails with this: > > java.io.FileNotFoundException: /CHANGELOG/data (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:120) > at > org.apache.avro.file.SeekableFileInput.<init>(SeekableFileInput.java:29) > at > org.apache.avro.file.DataFileWriter.appendTo(DataFileWriter.java:149) > at > com.abc.kepler.datasink.hdfs.util.SortedKeyValueFile$Writer.<init>(SortedKeyValueFile.java:597) > at > com.abc.kepler.datasink.hdfs.util.ChangeLogUtil.getChangeLogWriter(ChangeLogUtil.java:84) > at > com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLog.append(HDFSDataSinkChangeLog.java:219) > at > com.abc.kepler.datasink.hdfs.HDFSDataSinkChangesTest.writeDataSingleEntityKeyDefaultLocation(HDFSDataSinkChangesTest.java:1036) > at > com.abc.kepler.datasink.hdfs.HDFSDataSinkChangesTest.javadocExampleTest(HDFSDataSinkChangesTest.java:645) > > So, is the avro writer it not able to locate the file on hdfs? Could you > please share some pointers what could be leading to this? > > > > -- > View this message in context: > http://apache-avro.679487.n3.nabble.com/Is-it-possible-to-append-to-an-already-existing-avro-file-tp3762049p4027785.html > Sent from the Avro - Users mailing list archive at Nabble.com.