[jira] Created: (HDFS-1593) Allow a datanode to copy a block to a datanode on a foreign HDFS cluster.

2011-01-24 Thread dhruba borthakur (JIRA)
Allow a datanode to copy a block to a datanode on a foreign HDFS cluster. - Key: HDFS-1593 URL: https://issues.apache.org/jira/browse/HDFS-1593 Project: Hadoop HDFS Issu

[jira] Updated: (HDFS-1593) Allow a datanode to copy a block to a datanode on a foreign HDFS cluster.

2011-01-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-1593: --- Attachment: copyBlockTrunk1.txt This patch is useful for HighTide because it can be used to tr

[jira] Commented: (HDFS-1593) Allow a datanode to copy a block to a datanode on a foreign HDFS cluster.

2011-01-24 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985567#action_12985567 ] Arun C Murthy commented on HDFS-1593: - Dhruba, what are the security implications here?

[jira] Commented: (HDFS-1593) Allow a datanode to copy a block to a datanode on a foreign HDFS cluster.

2011-01-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985573#action_12985573 ] dhruba borthakur commented on HDFS-1593: Arun: The HighTide node is running with the

[jira] Updated: (HDFS-1557) Separate Storage from FSImage

2011-01-24 Thread Ivan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1557: - Attachment: HDFS-1557.diff > Separate Storage from FSImage > - > >

[jira] Updated: (HDFS-1557) Separate Storage from FSImage

2011-01-24 Thread Ivan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1557: - Status: Open (was: Patch Available) > Separate Storage from FSImage > - > >

[jira] Updated: (HDFS-1557) Separate Storage from FSImage

2011-01-24 Thread Ivan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1557: - Status: Patch Available (was: Open) According to the checklist, its Sun convention which actually says bo

[jira] Updated: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated HDFS-1594: Attachment: hadoop-root-namenode-linux124.log > When the disk becomes full Namenode is getting shutdown and

[jira] Created: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able recover

2011-01-24 Thread Devaraj K (JIRA)
When the disk becomes full Namenode is getting shutdown and not able recover Key: HDFS-1594 URL: https://issues.apache.org/jira/browse/HDFS-1594 Project: Hadoop HDFS

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985686#action_12985686 ] Devaraj K commented on HDFS-1594: - When the disk becomes full, name node file system (fsimag

[jira] Updated: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated HDFS-1594: Summary: When the disk becomes full Namenode is getting shutdown and not able to recover (was: When the dis

[jira] Updated: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated HDFS-1594: Attachment: HDFS-1594.patch > When the disk becomes full Namenode is getting shutdown and not able to > rec

[jira] Updated: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated HDFS-1594: Affects Version/s: 0.21.1 0.21.0 Release Note: Implemented a daemon thread t

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985692#action_12985692 ] Hadoop QA commented on HDFS-1594: - -1 overall. Here are the results of testing the latest a

[jira] Commented: (HDFS-1593) Allow a datanode to copy a block to a datanode on a foreign HDFS cluster.

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985858#action_12985858 ] Todd Lipcon commented on HDFS-1593: --- I think for this to work correctly with security, the

[jira] Commented: (HDFS-1084) TestDFSShell fails in trunk.

2011-01-24 Thread Po Cheung (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985874#action_12985874 ] Po Cheung commented on HDFS-1084: - testFilePermissions has the same failure on Windows becau

[jira] Assigned: (HDFS-982) TestDelegationToken#testDelegationTokenWithRealUser is failing

2011-01-24 Thread Po Cheung (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Po Cheung reassigned HDFS-982: -- Assignee: Po Cheung > TestDelegationToken#testDelegationTokenWithRealUser is failing > --

[jira] Created: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
DFSClient may incorrectly detect datanode failure - Key: HDFS-1595 URL: https://issues.apache.org/jira/browse/HDFS-1595 Project: Hadoop HDFS Issue Type: Bug Components: data-node, hdf

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985898#action_12985898 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- Here is my thought for fixing t

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985906#action_12985906 ] Todd Lipcon commented on HDFS-1595: --- Could we solve this with a blacklist-like feature? ie

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985929#action_12985929 ] Koji Noguchi commented on HDFS-1595: Can we describe the problem users faced before desc

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985939#action_12985939 ] Todd Lipcon commented on HDFS-1595: --- {quote} How about we change DFSClient so that if the

[jira] Commented: (HDFS-863) Potential deadlock in TestOverReplicatedBlocks

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985942#action_12985942 ] Todd Lipcon commented on HDFS-863: -- Seems the new writeUnlock() calls should be inside a fin

[jira] Assigned: (HDFS-1084) TestDFSShell fails in trunk.

2011-01-24 Thread Po Cheung (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Po Cheung reassigned HDFS-1084: --- Assignee: Po Cheung > TestDFSShell fails in trunk. > > >

[jira] Updated: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-1595: - Description: Suppose a source datanode S is writing to a destination datanode D in

[jira] Commented: (HDFS-1084) TestDFSShell fails in trunk.

2011-01-24 Thread Po Cheung (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986036#action_12986036 ] Po Cheung commented on HDFS-1084: - Some helpful links on chmod under cygwin... http://cygwi

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986035#action_12986035 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- {quote} Could we solve this wit

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986039#action_12986039 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- {quote} How about we change DFS

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986040#action_12986040 ] Todd Lipcon commented on HDFS-1595: --- bq. Consequently, the normal cases are penalized and

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986041#action_12986041 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- {quote} What about something li

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986046#action_12986046 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- {quote} What about something li

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986097#action_12986097 ] Todd Lipcon commented on HDFS-1595: --- bq. There is no completeBlock() call to the NN. We on

[jira] Updated: (HDFS-1566) Test that covers full partition

2011-01-24 Thread Konstantin Boudnik (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated HDFS-1566: - Attachment: HDFS-1566.sh A pretty poor attempt to automate this regression test is attache

[jira] Updated: (HDFS-1566) Test that covers full partition

2011-01-24 Thread Konstantin Boudnik (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated HDFS-1566: - Attachment: HDFS-1566.sh Fixing a debug message (not important) > Test that covers full p

[jira] Commented: (HDFS-1566) Test that covers full partition

2011-01-24 Thread Konstantin Boudnik (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986190#action_12986190 ] Konstantin Boudnik commented on HDFS-1566: -- I propose to resolve this JIRA as "Late

[jira] Commented: (HDFS-1391) Exiting safemode takes a long time when there are lots of blocks in the HDFS

2011-01-24 Thread Matt Foley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986193#action_12986193 ] Matt Foley commented on HDFS-1391: -- There seems to be a glitch: attachment excessReplicas3.

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986197#action_12986197 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- > Sorry, I meant completeFile()

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986198#action_12986198 ] Tsz Wo (Nicholas), SZE commented on HDFS-1595: -- The third solution would be: re

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986222#action_12986222 ] Devaraj K commented on HDFS-1594: - It passed in my system and shown +1 but it failed to appl

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Konstantin Boudnik (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986225#action_12986225 ] Konstantin Boudnik commented on HDFS-1594: -- You test has failed the application to

[jira] Updated: (HDFS-1595) DFSClient may incorrectly detect datanode failure

2011-01-24 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-1595: -- Attachment: hdfs-1595-idea.txt Since I wasn't doing a good job of explaining my idea, I just wrote a qui

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Konstantin Boudnik (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986233#action_12986233 ] Konstantin Boudnik commented on HDFS-1594: -- Coupla nits in your patch: - {{+res

[jira] Commented: (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover

2011-01-24 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986238#action_12986238 ] Devaraj K commented on HDFS-1594: - Thanks Konstantin. I will do all the changes and test wit