[jira] Updated: (HADOOP-1127) Speculative Execution and output of Reduce tasks

2007-03-30 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1127: -- Status: Open (was: Patch Available) Needs a minor change... > Speculative Execution and outp

[jira] Assigned: (HADOOP-1189) Still seeing some unexpected 'No space left on device' exceptions

2007-03-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned HADOOP-1189: Assignee: Raghu Angadi > Still seeing some unexpected 'No space left on device' exceptions

[jira] Created: (HADOOP-1189) Still seeing some unexpected 'No space left on device' exceptions

2007-03-30 Thread Raghu Angadi (JIRA)
Still seeing some unexpected 'No space left on device' exceptions - Key: HADOOP-1189 URL: https://issues.apache.org/jira/browse/HADOOP-1189 Project: Hadoop Issue Type: Bug

[jira] Updated: (HADOOP-1090) In SortValidator, the check for whether a file belongs to sort-input or sort-output dir is weak

2007-03-30 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1090: -- Fix Version/s: 0.13.0 Affects Version/s: 0.12.2 Status: Patch Available

[jira] Updated: (HADOOP-1090) In SortValidator, the check for whether a file belongs to sort-input or sort-output dir is weak

2007-03-30 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1090: -- Attachment: HADOOP-1090_20070331_1.patch Simple fix... > In SortValidator, the check for whet

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread Christian Kunz (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485688 ] Christian Kunz commented on HADOOP-1182: Increasing the number of handlers in namenode server seems to help

Re: [jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Nigel Daley
Try this: try { JobClient.runJob(conf); // Will throw exception about mismatched key type throw new AssertionError("descriptive message"); } catch (Exception expected) { // expected } Narrow the caught exception type as appropriate. On Mar 30, 2007, at 4:30 PM, Tahir Hashmi (JIRA) wrot

[jira] Updated: (HADOOP-1127) Speculative Execution and output of Reduce tasks

2007-03-30 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1127: -- Status: Patch Available (was: Open) > Speculative Execution and output of Reduce tasks >

[jira] Updated: (HADOOP-1127) Speculative Execution and output of Reduce tasks

2007-03-30 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1127: -- Attachment: HADOOP-1127_20070331_2.patch Tested and updated to reflect trunk... > Speculative

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-1001: - Attachment: TestMapOutputType.java Here's a JUnit test case that generates the exceptions. Unfor

[jira] Commented: (HADOOP-1187) DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks

2007-03-30 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485679 ] dhruba borthakur commented on HADOOP-1187: -- It could help. But if we store it in the pendingCreates, then w

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485678 ] dhruba borthakur commented on HADOOP-1182: -- That's what i thought earlier. But when I investigated, it appe

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485677 ] Owen O'Malley commented on HADOOP-1182: --- The calls waiting/thread is rarely relevant. Most of the time, it is

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485671 ] Raghu Angadi commented on HADOOP-1182: -- As part of this issue, should we make "calls waiting per thread" confi

[jira] Created: (HADOOP-1188) processIOError() should update fstime file

2007-03-30 Thread Konstantin Shvachko (JIRA)
processIOError() should update fstime file -- Key: HADOOP-1188 URL: https://issues.apache.org/jira/browse/HADOOP-1188 Project: Hadoop Issue Type: Bug Components: dfs Affects Versions: 0.1

[jira] Commented: (HADOOP-1172) Reduce job failed due to error in logging

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485669 ] Owen O'Malley commented on HADOOP-1172: --- -1 This is not logging from the framework. This is user data and if

[jira] Commented: (HADOOP-1187) DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks

2007-03-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485668 ] Raghu Angadi commented on HADOOP-1187: -- Hairong is adding map for hostname to datanode map. Would that help?

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HADOOP-1001: -- Status: Open (was: Patch Available) I'm sorry, I forgot to request a unit test case that test

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HADOOP-1001: -- Comment: was deleted > the output of the map is not type checked against the specified types >

[jira] Commented: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485663 ] Owen O'Malley commented on HADOOP-1001: --- +1 > the output of the map is not type checked against the specified

[jira] Commented: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485661 ] Hadoop QA commented on HADOOP-1001: --- +1, because http://issues.apache.org/jira/secure/attachment/12354644/1001.pat

Re: [jira] Updated: (HADOOP-1180) NNbench test should be able to test the checksumfilesystem as well as the raw filesystem

2007-03-30 Thread Raghu Angadi
Doug Cutting wrote: hairong Kuang wrote: 1. NNBench sets the block size to be 1. Althouth it generates a file with only 1 byte, but the file's checksum file has 16 bytes (12 bytes header plus 4 bytes checksums). Without the checksum file, only 1 block needs to be generated. With the checksum

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-1001: - Attachment: 1001.patch Re-written exception message to be more meaningful. > the output of the

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-1001: - Status: Patch Available (was: Open) > the output of the map is not type checked against the spe

Re: slip 0.13 release to May?

2007-03-30 Thread Doug Cutting
Tom White wrote: And we start backporting 0.12.3 patches into the 0.12 branch now? Yes, I think we can start that at any point. We can use Jira and the CHANGES.txt in that branch to keep track of things. Each backport commit should name the Jira issue(s) involved. Doug

[jira] Created: (HADOOP-1187) DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks

2007-03-30 Thread dhruba borthakur (JIRA)
DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks --- Key: HADOOP-1187 URL: https://issues.apache.org/jira/browse/HADOOP-1187 Project: Hadoop

Re: slip 0.13 release to May?

2007-03-30 Thread Tom White
+1 And we start backporting 0.12.3 patches into the 0.12 branch now? On 30/03/07, Doug Cutting <[EMAIL PROTECTED]> wrote: I think we ought to slip the 0.13 feature freeze from the first Friday in April to the first Friday in May. I also propose that this be the first release where we follow th

slip 0.13 release to May?

2007-03-30 Thread Doug Cutting
I think we ought to slip the 0.13 feature freeze from the first Friday in April to the first Friday in May. I also propose that this be the first release where we follow the new release process that's being discussed in https://issues.apache.org/jira/browse/HADOOP-1161. Thoughts? Doug

[jira] Updated: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1170: - Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available

[jira] Updated: (HADOOP-672) dfs shell enhancements

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-672: Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available)

[jira] Updated: (HADOOP-1011) ConcurrentModificationException in JobHistory

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1011: - Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available

[jira] Updated: (HADOOP-1159) Reducers hang when map output file has a checksum error

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1159: - Fix Version/s: (was: 0.12.3) Affects Version/s: 0.12.3 Status: Open (

[jira] Commented: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485636 ] Hadoop QA commented on HADOOP-1170: --- +1, because http://issues.apache.org/jira/secure/attachment/12354634/1170-v2

[jira] Commented: (HADOOP-672) dfs shell enhancements

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485634 ] Hadoop QA commented on HADOOP-672: -- +1, because http://issues.apache.org/jira/secure/attachment/12354460/fsshellHel

[jira] Updated: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Igor Bolotin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Bolotin updated HADOOP-1170: - Status: Patch Available (was: Open) > Very high CPU usage on data nodes because of FSDataset.ch

[jira] Updated: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Igor Bolotin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Bolotin updated HADOOP-1170: - Attachment: 1170-v2.patch This patch removes all FSDataset.checkDataDir() calls from DataNode as

[jira] Commented: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485626 ] Owen O'Malley commented on HADOOP-1001: --- I'm sorry, I wasn't very clear. The error message should be included

[jira] Commented: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485619 ] Hadoop QA commented on HADOOP-1001: --- +1, because http://issues.apache.org/jira/secure/attachment/12354628/1001.pat

[jira] Updated: (HADOOP-672) dfs shell enhancements

2007-03-30 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-672: Status: Patch Available (was: Open) > dfs shell enhancements > -- > >

[jira] Commented: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485614 ] Tahir Hashmi commented on HADOOP-1001: -- Oh yes, reporting the expected and received type names is the right thi

[jira] Commented: (HADOOP-1159) Reducers hang when map output file has a checksum error

2007-03-30 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485611 ] Tom White commented on HADOOP-1159: --- >We saw the NPE coming from the call to mapOutputIn.read( ) in the >MapOutpu

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HADOOP-1001: -- Fix Version/s: 0.13.0 Status: Open (was: Patch Available) Rather than assume that

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-1001: - Status: Patch Available (was: Open) > the output of the map is not type checked against the spe

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Tahir Hashmi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-1001: - Attachment: 1001.patch Using fields keyClass and valClass instead of calling JobConf methods. >

[jira] Commented: (HADOOP-1134) Block level CRCs in HDFS

2007-03-30 Thread Sameer Paranjpye (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485605 ] Sameer Paranjpye commented on HADOOP-1134: -- > But actually, now that I think about it, if we're primarily n

[jira] Commented: (HADOOP-1186) deadlock in Abstract Metrics Context

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485604 ] Hadoop QA commented on HADOOP-1186: --- -1, because the patch command could not apply the latest attachment http://i

[jira] Updated: (HADOOP-1186) deadlock in Abstract Metrics Context

2007-03-30 Thread David Bowen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Bowen updated HADOOP-1186: Resolution: Duplicate Status: Resolved (was: Patch Available) This is a duplicate of http

[jira] Updated: (HADOOP-1186) deadlock in Abstract Metrics Context

2007-03-30 Thread Michael Bieniosek (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Bieniosek updated HADOOP-1186: -- Status: Patch Available (was: Open) > deadlock in Abstract Metrics Context >

[jira] Updated: (HADOOP-1186) deadlock in Abstract Metrics Context

2007-03-30 Thread Michael Bieniosek (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Bieniosek updated HADOOP-1186: -- Attachment: metrics-deadlock.patch This patch fixes the problem. > deadlock in Abstra

[jira] Created: (HADOOP-1186) deadlock in Abstract Metrics Context

2007-03-30 Thread Michael Bieniosek (JIRA)
deadlock in Abstract Metrics Context Key: HADOOP-1186 URL: https://issues.apache.org/jira/browse/HADOOP-1186 Project: Hadoop Issue Type: Bug Components: metrics Affects Versions: 0.12.1

[jira] Commented: (HADOOP-1178) NullPointer Exception in org.apache.hadoop.dfs.NameNode.isDir on namenode restart

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485594 ] Hadoop QA commented on HADOOP-1178: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1164) TestReplicationPolicy doesn't use port 0 for the NameNode

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485593 ] Hadoop QA commented on HADOOP-1164: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1167) InMemoryFileSystem uses synchronizedtMaps with maps that are locked anyways

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485595 ] Hadoop QA commented on HADOOP-1167: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1169) CopyFiles skips src files of s3 urls

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485590 ] Hadoop QA commented on HADOOP-1169: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1146) "Reduce input records" counter name is misleading

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485592 ] Hadoop QA commented on HADOOP-1146: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1166) Pull the NullOutputFormat into the lib package

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485591 ] Hadoop QA commented on HADOOP-1166: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1165) Code for toString in code generated by Record I/O Compiler can be generic

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485589 ] Hadoop QA commented on HADOOP-1165: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1110) JobTracker WebUI "Map input records" a little off.

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485588 ] Hadoop QA commented on HADOOP-1110: --- Integrated in Hadoop-Nightly #42 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1134) Block level CRCs in HDFS

2007-03-30 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485585 ] Konstantin Shvachko commented on HADOOP-1134: - > The split could include the datanode name, the block ID

[jira] Commented: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Igor Bolotin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485583 ] Igor Bolotin commented on HADOOP-1170: -- I'll prepare patch with all calls removed later today > Very high CPU

[jira] Updated: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Igor Bolotin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Bolotin updated HADOOP-1170: - Status: Open (was: Patch Available) > Very high CPU usage on data nodes because of FSDataset.ch

[jira] Created: (HADOOP-1185) dynamically change dfs log levels

2007-03-30 Thread dhruba borthakur (JIRA)
dynamically change dfs log levels - Key: HADOOP-1185 URL: https://issues.apache.org/jira/browse/HADOOP-1185 Project: Hadoop Issue Type: Improvement Components: dfs Reporter: dhruba bortha

[jira] Updated: (HADOOP-1178) NullPointer Exception in org.apache.hadoop.dfs.NameNode.isDir on namenode restart

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1178: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank

[jira] Updated: (HADOOP-1110) JobTracker WebUI "Map input records" a little off.

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1110: - Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available

[jira] Commented: (HADOOP-1170) Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485578 ] Doug Cutting commented on HADOOP-1170: -- Is there a consensus to commit this as-is, or is someone working on an

[jira] Updated: (HADOOP-1179) task Tracker should be restarted if its jetty http server cannot serve get-map-output files

2007-03-30 Thread Nigel Daley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Daley updated HADOOP-1179: Fix Version/s: 0.12.3 > task Tracker should be restarted if its jetty http server cannot serve >

[jira] Updated: (HADOOP-1111) Job completion notification to a job configured URL

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-: - Status: Open (was: Patch Available) I'd rather wait to commit this until we have some unit test

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485576 ] dhruba borthakur commented on HADOOP-1182: -- There are a few open issues that deal with reducing CPU usage o

[jira] Commented: (HADOOP-1011) ConcurrentModificationException in JobHistory

2007-03-30 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485575 ] Owen O'Malley commented on HADOOP-1011: --- I like Tahir's patch better than blindly replacing with synchronized

[jira] Updated: (HADOOP-1001) the output of the map is not type checked against the specified types

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1001: - Status: Open (was: Patch Available) This is inner-loop code and should use the fields keyClass

[jira] Updated: (HADOOP-1167) InMemoryFileSystem uses synchronizedtMaps with maps that are locked anyways

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1167: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank

[jira] Updated: (HADOOP-1169) CopyFiles skips src files of s3 urls

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1169: - Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available

[jira] Updated: (HADOOP-1166) Pull the NullOutputFormat into the lib package

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1166: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank

[jira] Updated: (HADOOP-1164) TestReplicationPolicy doesn't use port 0 for the NameNode

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1164: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank

[jira] Updated: (HADOOP-1105) Reducers don't make "progress" while iterating through values

2007-03-30 Thread Nigel Daley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Daley updated HADOOP-1105: Fix Version/s: (was: 0.13.0) 0.12.3 Adding to 0.12.3 queue as suggested by

[jira] Updated: (HADOOP-1165) Code for toString in code generated by Record I/O Compiler can be generic

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1165: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thank

[jira] Updated: (HADOOP-1146) "Reduce input records" counter name is misleading

2007-03-30 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated HADOOP-1146: - Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available

[jira] Commented: (HADOOP-1182) DFS Scalability issue with filecache in large clusters

2007-03-30 Thread Christian Kunz (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485560 ] Christian Kunz commented on HADOOP-1182: Indeed, namenode server is on 99.9 CPU. top output during transitio

[jira] Commented: (HADOOP-1011) ConcurrentModificationException in JobHistory

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485557 ] Hadoop QA commented on HADOOP-1011: --- +1, because http://issues.apache.org/jira/secure/attachment/12354497/1011.pat

[jira] Updated: (HADOOP-1105) Reducers don't make "progress" while iterating through values

2007-03-30 Thread Devaraj Das (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HADOOP-1105: Attachment: 1105.patch I am attaching a patch for this. Please review it, and if it seems ok, plea

[jira] Updated: (HADOOP-1011) ConcurrentModificationException in JobHistory

2007-03-30 Thread woyg (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] woyg updated HADOOP-1011: - Attachment: 1011.patch Better use synchronized maps to avoid more such issues, as JobHistory class and its inne

[jira] Updated: (HADOOP-1011) ConcurrentModificationException in JobHistory

2007-03-30 Thread woyg (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] woyg updated HADOOP-1011: - Status: Patch Available (was: Open) > ConcurrentModificationException in JobHistory > -

Re: [jira] Updated: (HADOOP-1180) NNbench test should be able to test the checksumfilesystem as well as the raw filesystem

2007-03-30 Thread Doug Cutting
hairong Kuang wrote: 1. NNBench sets the block size to be 1. Althouth it generates a file with only 1 byte, but the file's checksum file has 16 bytes (12 bytes header plus 4 bytes checksums). Without the checksum file, only 1 block needs to be generated. With the checksum file, 17 blocks need to

Re: Cost of debug statements.

2007-03-30 Thread Enis Soztutar
Doug Cutting wrote: Nigel Daley wrote: Just a caveat: when there is conditional code like this based on the configured logging level, then testing should be done with different logging levels to verify that there are no side effects or exceptions (such as NPE) generated by these conditional code

[jira] Commented: (HADOOP-1158) JobTracker should collect statistics of failed map output fetches, and take decisions to reexecute map tasks and/or restart the (possibly faulty) Jetty server on the Ta

2007-03-30 Thread Devaraj Das (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485481 ] Devaraj Das commented on HADOOP-1158: - Yes, the reduce/fetcher should try a few times (it could be configurable?

[jira] Commented: (HADOOP-1177) Lack of logging of exceptions in MapOutputLocation.getFile

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485479 ] Hadoop QA commented on HADOOP-1177: --- Integrated in Hadoop-Nightly #41 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1123) LocalFileSystem gets a NullPointerException when tries to recover from ChecksumError

2007-03-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485478 ] Hadoop QA commented on HADOOP-1123: --- Integrated in Hadoop-Nightly #41 (See http://lucene.zones.apache.org:8080/hu

Hudson build is back to normal: Hadoop-Nightly #41

2007-03-30 Thread hudson
See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/41/