[ https://issues.apache.org/jira/browse/AVRO-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573711#comment-13573711 ]
Michael Malak commented on AVRO-1035: ------------------------------------- ha...@cloudera.com has provided example code on how to accomplish HDFS Avro append at https://gist.github.com/QwertyManiac/4724582 > Add the possibility to append to existing avro files > ------------------------------------------------------ > > Key: AVRO-1035 > URL: https://issues.apache.org/jira/browse/AVRO-1035 > Project: Avro > Issue Type: New Feature > Reporter: Vyacheslav Zholudev > > Currently it is not possible to append to avro files that were written and > closed. > Here is a Scott Carey's reply on the mailing list: > {quote} > It is not possible without modifying DataFileWriter. Please open a JIRA > ticket. > It could not simply append to an OutputStream, since it must either: > * Seek to the start to validate the schemas match and find the sync > marker, or > * Trust that the schemas match and find the sync marker from the last block > DataFileWriter cannot refer to Hadoop classes such as FileSystem, but we > could add something to the mapred module that takes a Path and FileSystem > and returns > something that implemements an interface that DataFileWriter can append > to. This would be something that is both a > http://avro.apache.org/docs/1.6.2/api/java/org/apache/avro/file/SeekableInp > ut.html > and an OutputStream, or has both an InputStream from the start of the > existing file and an OutputStream at the end. > {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira