rangareddy commented on issue #6009:
URL: https://github.com/apache/hudi/issues/6009#issuecomment-5102637444
This issue was reviewed as part of the JIRA-migrated backlog triage.
Findings: The attached trace points at HDFS rather than at Hudi. The failure
is
```
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException):
File does not exist:
/tmp/huditest/partition_0/.hoodie_partition_metadata_0 (inode 16466)
[Lease. Holder: DFSClient_NONMAPREDUCE_-535797949_164, pending creates: 1]
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease
```
raised from `FSNamesystem.checkLease` during a `DFSClient` fsync -- the
namenode no longer held a valid lease on the partition-metadata file that the
writer was still creating. That is a lease/namenode-state condition in the HDFS
cluster; nothing in the trace identifies Hudi as the cause.
Diagnosing it further would need the namenode log for that inode and a
listing of the table's `.hoodie` directory at the time, which were requested on
2022-06-30 and never provided. The reproduction as given -- running the
`hudi-kafka-connect/demo` scripts against `hudi-master` as of mid-2022 with
Hudi 0.10.0 / Hadoop 3.2.1 -- is not reconstructable today.
Closing as stale. If you still hit this, please open a fresh issue on a
current Hudi version with the namenode log alongside the connect worker log,
and note whether the target path is on HDFS with any non-default lease or
append settings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
