[
https://issues.apache.org/jira/browse/HDFS-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983259#comment-13983259
]
Colin Patrick McCabe commented on HDFS-1309:
--------------------------------------------
Is this still an issue in branch 2? It looks like
{{DistributedFileSystem#rename}} throws {{IOException}} if something goes
wrong, and has for a while. Unless I'm missing something, rename is not a
filesystem operation that fails silently.
> FileSystem.rename will fail silently
> ------------------------------------
>
> Key: HDFS-1309
> URL: https://issues.apache.org/jira/browse/HDFS-1309
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 0.20.2
> Environment: Linux version 2.6.31-302-ec2 (buildd@yellow) (gcc
> version 4.4.1 (Ubuntu 4.4.1-4ubuntu7) ) #7-Ubuntu SMP Tue Oct 13 19:55:22 UTC
> 2009
> Reporter: Kris Nuttycombe
> Priority: Minor
>
> Some filesystem operations (such as rename) will fail silently. In the
> attached example, a failure message will be written to the hadoop log, but it
> would be much better if the operation were to "fail fast" by throwing a
> checked exception and forcing the caller to handle the problem; failing to do
> so can easily lead to inadvertent data corruption.
> val coalesceBasePath = new Path(eventLog.basePath, coalesceTo)
> val backupBasePath = new Path(eventLog.basePath, relocateTo)
> eventLog.fs.mkdirs(backupBasePath)
> for (path <- coalesced; time <- HDFSEventLog.timePart(path, eventType)) {
> val backupPath = HDFSEventLog.path(backupBasePath, eventType, time)
> log.info("Relocating " + path + " to " + backupPath)
> eventLog.fs.rename(path, backupPath)
> }
> INF [20100715-16:11:20.727] reporting: Relocating
> hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226067707 to
> hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226067707
> INF [20100715-16:11:20.752] reporting: Relocating
> hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226077707 to
> hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226077707
> INF [20100715-16:11:20.754] reporting: Relocating
> hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226457707 to
> hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226457707
> INF [20100715-16:11:20.757] reporting: Relocating
> hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279229126727 to
> hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279229126727
> Complete.
> [knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls
> hdfs://localhost:9000/test-batchEventLog/
> Found 3 items
> drwxr-xr-x - knuttycombe supergroup 0 2010-07-15 14:54
> /test-batchEventLog/coalesced
> drwxr-xr-x - knuttycombe supergroup 0 2010-07-15 14:35
> /test-batchEventLog/metrics
> drwxr-xr-x - knuttycombe supergroup 0 2010-07-15 16:11
> /test-batchEventLog/pre-coalesce
> [knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls
> hdfs://localhost:9000/test-batchEventLog/metrics
> Found 4 items
> -rw-r--r-- 3 knuttycombe supergroup 2017122 2010-07-15 14:34
> /test-batchEventLog/metrics/metrics_1279226067707
> -rw-r--r-- 3 knuttycombe supergroup 4122951 2010-07-15 14:34
> /test-batchEventLog/metrics/metrics_1279226077707
> -rw-r--r-- 3 knuttycombe supergroup 5999912 2010-07-15 14:35
> /test-batchEventLog/metrics/metrics_1279226457707
> -rw-r--r-- 3 knuttycombe supergroup 8638301 2010-07-15 14:26
> /test-batchEventLog/metrics/metrics_1279229126727
> [knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls
> hdfs://localhost:9000/test-batchEventLog/pre-coalesce
> [knuttycombe@floorshow reporting (reporting-coalesce)]$
--
This message was sent by Atlassian JIRA
(v6.2#6252)
