[
https://issues.apache.org/jira/browse/HDFS-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371459#comment-14371459
]
Sverre Bakke commented on HDFS-7941:
------------------------------------
Ok, I just tested this will BLOCK compression, RECORD compression as well as
NONE compression, and same result on all modes. Furthermore, I tested without
sequence files (i.e. normal files) and also here it only syncs at the very
beginning and never again until file is closed.
> SequenceFile.Writer.hsync() not working?
> ----------------------------------------
>
> Key: HDFS-7941
> URL: https://issues.apache.org/jira/browse/HDFS-7941
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.6.0
> Environment: HDP 2.2 running on Redhat
> Reporter: Sverre Bakke
>
> When using SequenceFile.Writer and appending+syncing to file repeatedly, the
> sync does not appear to work other than:
> - once after writing headers
> - when closing.
> Imagine the following test case:
> http://pastebin.com/Y9xysCRX
> This code would append a new record every second and then immediately sync
> it. One would also imagine that the file would grow for every append,
> however, this does not happen.
> After watching the behavior I have noticed that it only syncs the headers at
> the very beginning (providing a file of 164 bytes) and then never again until
> its closed. This despite it is asked to hsync() after every append.
> Looking into the debug logs, this also claims the same behavior (executed the
> provided code example and grepped for "sync"):
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> 2015-03-17 15:55:14 DEBUG ProtobufRpcEngine:253 - Call: fsync took 11ms
> This was the only time the code ran fsync throughout the entire execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
