Re: Block missing due to power failure

2022-12-30 Thread Viraj Jasani
Agree. For some reason, if you would not like to use more than one datanode (let alone datanodes across multiple racks for fault tolerance) for some non-critical usecase, it's still recommended to use hsync over the output stream for on-disk persistence (unless the single DN setup is being used onl

Re: Block missing due to power failure

2022-12-30 Thread Ayush Saxena
The file was in progress? In that case this is possible, once the data gets persisted on the disk of the datanode then the data loss ain’t possible.If someone did a hflush and not hsync while writing and the power loss happens immediately after that, so in that case also I feel there is a possibili

Block missing due to power failure

2022-12-30 Thread hehaore...@gmail.com
Hi,A 1-replica HDFS cluster with a single DataNode. When the DataNode was restarted after power failure, it found a file with a missing block. The size of the block and mate files found in the storage path is empty, and the last modification time is the power off time. Besides the fact that the fil