[jira] Commented: (HADOOP-2757) Should DFS outputstream's close wait forever?

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712650#action_12712650 ] dhruba borthakur commented on HADOOP-2757: -- > 1. rpc timeout: the patch seems to

[jira] Updated: (HADOOP-5896) Remove the dependency of GenericOptionsParser on Option.withArgPattern

2009-05-24 Thread Jothi Padmanabhan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated HADOOP-5896: -- Assignee: Sharad Agarwal > Remove the dependency of GenericOptionsParser on Option.wit

[jira] Commented: (HADOOP-5896) Remove the dependency of GenericOptionsParser on Option.withArgPattern

2009-05-24 Thread Jothi Padmanabhan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712648#action_12712648 ] Jothi Padmanabhan commented on HADOOP-5896: --- The following test is not running w

[jira] Updated: (HADOOP-5252) Streaming overrides -inputformat option

2009-05-24 Thread Sharad Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sharad Agarwal updated HADOOP-5252: --- Resolution: Fixed Fix Version/s: 0.21.0 Hadoop Flags: [Reviewed] Statu

[jira] Commented: (HADOOP-5623) Streaming: process provided status messages are overwritten every 10 seoncds

2009-05-24 Thread Jothi Padmanabhan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712642#action_12712642 ] Jothi Padmanabhan commented on HADOOP-5623: --- Changes look fine. However, the tes

[jira] Updated: (HADOOP-5438) Merge FileSystem.create and FileSystem.append

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-5438: - Resolution: Fixed Fix Version/s: 0.21.0 Status: Resolved (was: Patch A

[jira] Commented: (HADOOP-5890) Use exponential backoff on Thread.sleep during DN shutdown

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712637#action_12712637 ] dhruba borthakur commented on HADOOP-5890: -- +1. Code looks good. > Use exponent

[jira] Commented: (HADOOP-5739) After JobTracker restart Capacity Schduler does not schedules pending tasks from already running tasks.

2009-05-24 Thread Hemanth Yamijala (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712636#action_12712636 ] Hemanth Yamijala commented on HADOOP-5739: -- Some comments: - In JobTracker.proce

[jira] Updated: (HADOOP-5900) Minor correction in HDFS Documentation

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-5900: - Resolution: Fixed Fix Version/s: (was: 0.18.3) (was: 0.

[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments

2009-05-24 Thread Ravi Gummadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Gummadi updated HADOOP-5895: - Attachment: HADOOP-5895.v1.patch Agreed Jothi. Here is the patch with a flag doing the work of c

[jira] Updated: (HADOOP-5883) TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory

2009-05-24 Thread Hemanth Yamijala (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Yamijala updated HADOOP-5883: - Attachment: HADOOP-5883.patch The attached patch file incorporates the changes as mentio

[jira] Updated: (HADOOP-5883) TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory

2009-05-24 Thread Hemanth Yamijala (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Yamijala updated HADOOP-5883: - Status: Patch Available (was: Open) > TaskMemoryMonitorThread might shoot down tasks ev

[jira] Commented: (HADOOP-5859) FindBugs : fix "wait() or sleep() with locks held" warnings in hdfs

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712631#action_12712631 ] dhruba borthakur commented on HADOOP-5859: -- > 2) dataQueue lock and ackQueue lock

[jira] Commented: (HADOOP-5883) TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory

2009-05-24 Thread Hemanth Yamijala (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712630#action_12712630 ] Hemanth Yamijala commented on HADOOP-5883: -- As per some offline discussions with

[jira] Commented: (HADOOP-5895) Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments

2009-05-24 Thread Jothi Padmanabhan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712629#action_12712629 ] Jothi Padmanabhan commented on HADOOP-5895: --- Using a flag to control the number

[jira] Commented: (HADOOP-5710) Counter MAP_INPUT_BYTES missing from new mapreduce api.

2009-05-24 Thread Amareshwari Sriramadasu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712628#action_12712628 ] Amareshwari Sriramadasu commented on HADOOP-5710: - Contrib test failures a

Build failed in Hudson: Hadoop-Patch-vesta.apache.org #394

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/394/ -- [...truncated 507968 lines...] [exec] [junit] 09/05/25 02:57:38 INFO mapred.IndexCache: Map ID attempt_200905250244_0001_m_02_2 not found in cache [exec]

Build failed in Hudson: Hadoop-Patch-vesta.apache.org #393

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/393/changes Changes: [gkesavan] HADOOP-5847. Fixed failing Streaming unit tests -- [...truncated 435061 lines...] [exec] [junit] 09/05/24 22:29:58 WARN mapred.JobInProgre

[jira] Commented: (HADOOP-5472) Distcp does not support globbing of input paths

2009-05-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712611#action_12712611 ] Hadoop QA commented on HADOOP-5472: --- -1 overall. Here are the results of testing the la

[jira] Commented: (HADOOP-5904) Hung on hdfs: writeChunk, DFSClient.java:2126, DataStreamer socketWrite

2009-05-24 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712590#action_12712590 ] Kan Zhang commented on HADOOP-5904: --- > this issue will be fixed in the trunk as part of

Build failed in Hudson: Hadoop-trunk #845

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/845/ -- [...truncated 499674 lines...] [junit] 09/05/24 18:02:50 INFO datanode.DataNode: PacketResponder 0 for block blk_-3110252916069208619_1008 terminating [junit] 09/05/24 18:02:50 INFO

[jira] Commented: (HADOOP-5906) Stream test TestStreamingExitStatus fails with Out of Memory

2009-05-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712569#action_12712569 ] Steve Loughran commented on HADOOP-5906: stack trace. Presumably we just need to s

[jira] Created: (HADOOP-5906) Stream test TestStreamingExitStatus fails with Out of Memory

2009-05-24 Thread Steve Loughran (JIRA)
Stream test TestStreamingExitStatus fails with Out of Memory Key: HADOOP-5906 URL: https://issues.apache.org/jira/browse/HADOOP-5906 Project: Hadoop Core Issue Type: Bug

[jira] Updated: (HADOOP-5847) Streaming unit tests failing for a while on trunk

2009-05-24 Thread Giridharan Kesavan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated HADOOP-5847: --- Resolution: Fixed Fix Version/s: 0.21.0 Status: Resolved (was: Pat

[jira] Commented: (HADOOP-5847) Streaming unit tests failing for a while on trunk

2009-05-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712566#action_12712566 ] Steve Loughran commented on HADOOP-5847: I'm willing to do the commit, but just ne

[jira] Commented: (HADOOP-5847) Streaming unit tests failing for a while on trunk

2009-05-24 Thread Giridharan Kesavan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712563#action_12712563 ] Giridharan Kesavan commented on HADOOP-5847: should we commit this? > Stream

Build failed in Hudson: Hadoop-Patch-vesta.apache.org #392

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/392/ -- [...truncated 544191 lines...] [exec] [junit] 09/05/24 13:27:40 INFO mapred.TaskTracker: attempt_200905241327_0001_m_01_0 0.0% cleanup > map [exec] [ju

[jira] Commented: (HADOOP-5170) Set max map/reduce tasks on a per-job basis, either per-node or cluster-wide

2009-05-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712550#action_12712550 ] Hadoop QA commented on HADOOP-5170: --- -1 overall. Here are the results of testing the la

[jira] Commented: (HADOOP-5847) Streaming unit tests failing for a while on trunk

2009-05-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712540#action_12712540 ] Steve Loughran commented on HADOOP-5847: I'm running these tests at home (OSX) , a

[jira] Updated: (HADOOP-5816) ArrayIndexOutOfBoundsException when using KeyFieldBasedComparator

2009-05-24 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HADOOP-5816: - Status: Patch Available (was: Open) > ArrayIndexOutOfBoundsException when using KeyFieldBasedCo

[jira] Updated: (HADOOP-5816) ArrayIndexOutOfBoundsException when using KeyFieldBasedComparator

2009-05-24 Thread He Yongqiang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HADOOP-5816: - Attachment: hadoop-5816-2009-05-24.patch Talked with Min about this bug two weeks ago. This is a

[jira] Commented: (HADOOP-5438) Merge FileSystem.create and FileSystem.append

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712527#action_12712527 ] dhruba borthakur commented on HADOOP-5438: -- +1 Code looks good. I will commit it

[jira] Commented: (HADOOP-5887) Sqoop should create tables in Hive metastore after importing to HDFS

2009-05-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712526#action_12712526 ] Hadoop QA commented on HADOOP-5887: --- -1 overall. Here are the results of testing the la

Build failed in Hudson: Hadoop-Patch-vesta.apache.org #391

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/391/ -- [...truncated 20 lines...] [exec] [exec] == [exec] =

[jira] Commented: (HADOOP-5620) discp can preserve modification times of files

2009-05-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712525#action_12712525 ] Hadoop QA commented on HADOOP-5620: --- -1 overall. Here are the results of testing the la

Build failed in Hudson: Hadoop-Patch-vesta.apache.org #390

2009-05-24 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/390/ -- [...truncated 514737 lines...] [exec] [junit] 09/05/24 08:53:40 INFO mapred.TaskTracker: attempt_200905240853_0001_m_01_0 0.0% [exec] [junit] 09/05/24

Re: Question regarding HDFS Recovery

2009-05-24 Thread Dhruba Borthakur
If the client and all the datanode(s) for a block are dead, then the file is corrupt. It cannot be recovered and the lease cannot be reclaimed. Is that a problem? > your replication factor is 3 and the NN's minReplication is also 3. In the current trunk, the NN guarantees to keep blocks replicat

[jira] Commented: (HADOOP-5759) IllegalArgumentException when CombineFileInputFormat is used as job InputFormat

2009-05-24 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712524#action_12712524 ] dhruba borthakur commented on HADOOP-5759: -- This patch keeps the original aim of