[
https://issues.apache.org/jira/browse/HDFS-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967794#comment-16967794
]
Wei-Chiu Chuang commented on HDFS-14947:
----------------------------------------
Hey [~abhishek.sahani] thanks for the details.
bq. Firstly the connector task creates a temporary file for partition assigned
to it in hdfs inmemory file system and later after certain rotation time
temporary file is closed and persisted to filesystem and later the temp file is
also renamed in hdfs.
In other words, is RAM_DISK / LAZY_PERSIST used
(https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/MemoryStorage.html)?
That can be an issue. Both Cloudera and Hortonworks don't support this feature
officially and I know it's not as robust as it should be. Still, a file goes
missing without a reason doesn't make sense to me.
> infrequent data loss due to rename functionality breaking
> ---------------------------------------------------------
>
> Key: HDFS-14947
> URL: https://issues.apache.org/jira/browse/HDFS-14947
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.7.3
> Reporter: abhishek sahani
> Priority: Critical
>
> We are facing an issue where data is getting lost from hdfs during rename ,
> in namenode logs we check file is renamed successfully but in hdfs after
> rename file is not present at destination location and thus we are loosing
> the data.
>
> namenode logs:
> 19/10/31 16:54:09 DEBUG top.TopAuditLogger: ------------------- logged event
> for top service: allowed=true ugi=root (auth:SIMPLE) ip=/*.*.*.* cmd=rename
> src=/topics/+tmp/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/351bffa9-15e3-427b-9e02-c9e8823d68d6_tmp.parquet
>
> dst=/topics/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic+9+0000000000+0000000099.parquet
> perm=root:supergroup:rw-r--r--
>
> 19/10/31 16:54:09 DEBUG hdfs.StateChange: DIR* NameSystem.renameTo:
> /topics/+tmp/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/351bffa9-15e3-427b-9e02-c9e8823d68d6_tmp.parquet
> to
> /topics/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic+9+0000000000+0000000099.parquet
> 19/10/31 16:54:09 DEBUG ipc.Server: IPC Server handler 8 on 9000: responding
> to org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from
> *.*.*.*:39854 Call#48333 Retry#0
> 19/10/31 16:54:09 DEBUG hdfs.StateChange: DIR* FSDirectory.renameTo:
> /topics/+tmp/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/351bffa9-15e3-427b-9e02-c9e8823d68d6_tmp.parquet
> to
> /topics/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic+9+0000000000+0000000099.parquet
> 19/10/31 16:54:09 DEBUG ipc.Server: IPC Server handler 6 on 9000:
> org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from *.*.*.*:39854
> Call#48337 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER
> 19/10/31 16:54:09 DEBUG hdfs.StateChange: DIR*
> FSDirectory.unprotectedRenameTo:
> /topics/+tmp/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/351bffa9-15e3-427b-9e02-c9e8823d68d6_tmp.parquet
> is renamed to
> /topics/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic/tenant=5da59e664cedfd00090d3757/groupid=5da59e664cedfd00090d3758/project=5da59e664cedfd00090d3759/name=dataPipeLineEvent_17/year=2019/month=10/day=16/hour=17/datapipelinefinaltest14.5da59e664cedfd00090d3757.dataPipeLineEvent_17.topic+9+0000000000+0000000099.parquet
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
