I was following this thread for a problem I am facing while using
SortedKeyValueFiles.
Below is the piece of code that tries to obtain the appropriate writer based
on whether I am appending or creating a new file:
OutputStream dataOutputStream;
if (!fileSystem.exists(dataFilePath)) {
dataOutputStream = fileSystem.create(dataFilePath);
mDataFileWriter = new
DataFileWriter<GenericRecord>(datumWriter).setSyncInterval(1 <<
20).create(mRecordSchema, dataOutputStream);
} else {
dataOutputStream = fileSystem.append(dataFilePath);
mDataFileWriter = new
DataFileWriter<GenericRecord>(datumWriter).setSyncInterval(1 <<
20).appendTo(new File(options.getPath() + DATA_FILENAME));
}
but it fails with this:
java.io.FileNotFoundException: /CHANGELOG/data (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at
org.apache.avro.file.SeekableFileInput.<init>(SeekableFileInput.java:29)
at org.apache.avro.file.DataFileWriter.appendTo(DataFileWriter.java:149)
at
com.abc.kepler.datasink.hdfs.util.SortedKeyValueFile$Writer.<init>(SortedKeyValueFile.java:597)
at
com.abc.kepler.datasink.hdfs.util.ChangeLogUtil.getChangeLogWriter(ChangeLogUtil.java:84)
at
com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLog.append(HDFSDataSinkChangeLog.java:219)
at
com.abc.kepler.datasink.hdfs.HDFSDataSinkChangesTest.writeDataSingleEntityKeyDefaultLocation(HDFSDataSinkChangesTest.java:1036)
at
com.abc.kepler.datasink.hdfs.HDFSDataSinkChangesTest.javadocExampleTest(HDFSDataSinkChangesTest.java:645)
So, is the avro writer it not able to locate the file on hdfs? Could you
please share some pointers what could be leading to this?
--
View this message in context:
http://apache-avro.679487.n3.nabble.com/Is-it-possible-to-append-to-an-already-existing-avro-file-tp3762049p4027785.html
Sent from the Avro - Users mailing list archive at Nabble.com.
