BytesWritable / SequenceFile yields dummy linefeed at end as soon as content has one or more linefeeds. -------------------------------------------------------------------------------------------------------
Key: HADOOP-7760 URL: https://issues.apache.org/jira/browse/HADOOP-7760 Project: Hadoop Common Issue Type: Bug Components: record Affects Versions: 0.20.2 Environment: Easily reproducable on Debian Linux cluster but also on my Arch Linux desktop. I am aware there are some newer releases in the 0.20 series, but all changelogs and release note links for those @ http://hadoop.apache.org/common/releases.html are broken, so I can't check if this has been fixed and/or whether it's safe to upgrade. Reporter: Dieter Plaetinck Priority: Minor I create SequenceFiles which have BytesWritable as values. I notice that if I store content which contains no linefeeds ("\n") or one linefeed, in the value, the value can also be read out of the sequencefile properly. However, as soon as I store input which contains two or more linefeeds (which is actually pretty much always the case), during the process of writing to the sequencefile and reading my data back, one *extra* linefeed is yielded at the end of the value, a linefeed which did not exist in the input. So this effectively corrupts my data, although i could write a hacky workaround for it. I have written a program that demonstrates the behavior, by showing what happens when writing 2 sequencefiles: one that has a record which value contains one linefeeds. another that has a record which value contains two linefeeds. Upon reading, the latter value will contain 3 linefeeds. Test file is : http://pastie.org/2728797 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira