[jira] Commented: (HADOOP-206) local map-reduce job limited to single reducer

[ https://issues.apache.org/jira/browse/HADOOP-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489938 ] Runping Qi commented on HADOOP-206: --- If LocalJobRunner does not force to set numReduceTasks to 1, unit tests TestM

[jira] Created: (HADOOP-1270) Randomize the fetch of map outputs

Randomize the fetch of map outputs -- Key: HADOOP-1270 URL: https://issues.apache.org/jira/browse/HADOOP-1270 Project: Hadoop Issue Type: Improvement Components: mapred Reporter: Arun C M

[jira] Commented: (HADOOP-1234) map tasks fail because they do not find application in file cache

[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489929 ] Devaraj Das commented on HADOOP-1234: - Could you please tell whether you see this problem consistently? > map t

[jira] Commented: (HADOOP-1269) DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489914 ] Raghu Angadi commented on HADOOP-1269: -- Could you briefly describe the load you tested with. This does not see

[jira] Commented: (HADOOP-1269) DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489911 ] Konstantin Shvachko commented on HADOOP-1269: - I think we still can optimize some things in the name-nod

Fwd: hadoop file permissions

Hello hadoop mailing list. I'm an intern at a software company somewhere that's been tasked with adding file permissions to hadoop. I've begun a discussion with Doug Cutting about how to accomplish that, and he suggested that I move it to the mailing list. So here it is. If you have any suggest

[jira] Updated: (HADOOP-206) local map-reduce job limited to single reducer

[ https://issues.apache.org/jira/browse/HADOOP-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HADOOP-206: - Fix Version/s: 0.13.0 Assignee: Runping Qi (was: Owen O'Malley) > local map-reduce job

[jira] Commented: (HADOOP-1269) DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489905 ] dhruba borthakur commented on HADOOP-1269: -- Different methods may synchronize on different subsets of the n

[jira] Updated: (HADOOP-234) Support for writing Map/Reduce functions in C++

[ https://issues.apache.org/jira/browse/HADOOP-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HADOOP-234: - Attachment: pipes.patch Here is a first pass of this patch. I still need to add unit tests. > Su

[jira] Commented: (HADOOP-1269) DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489890 ] Doug Cutting commented on HADOOP-1269: -- Won't different methods need to synchronize on different subsets of the

[jira] Commented: (HADOOP-1161) need improved release process

[ https://issues.apache.org/jira/browse/HADOOP-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489883 ] Doug Cutting commented on HADOOP-1161: -- I don't believe we should set a release date, since we can't know how l

[jira] Updated: (HADOOP-1190) Fix unchecked warnings

[ https://issues.apache.org/jira/browse/HADOOP-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-1190: -- Status: Open (was: Patch Available) > Fix unchecked warnings > -- > >

[jira] Commented: (HADOOP-1190) Fix unchecked warnings

[ https://issues.apache.org/jira/browse/HADOOP-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489882 ] Tom White commented on HADOOP-1190: --- Committed mapred package fixes in revision 530153. This completes core Hadoop

[jira] Updated: (HADOOP-1111) Job completion notification to a job configured URL

[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-: -- Resolution: Fixed Status: Resolved (was: Patch Available) I've just committed this. Thanks Al

[jira] Assigned: (HADOOP-1269) DFS Scalability: namenode throughout impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur reassigned HADOOP-1269: Assignee: dhruba borthakur > DFS Scalability: namenode throughout impacted becuase

[jira] Updated: (HADOOP-1269) DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-1269: - Summary: DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lo

[jira] Updated: (HADOOP-1269) DFS Scalability: namenode throughout impacted becuase of global FSNamesystem lock

[ https://issues.apache.org/jira/browse/HADOOP-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-1269: - Attachment: serverThreads1.html serverThreads40.html > DFS Scalability:

[jira] Created: (HADOOP-1269) DFS Scalability: namenode throughout impacted becuase of global FSNamesystem lock

DFS Scalability: namenode throughout impacted becuase of global FSNamesystem lock - Key: HADOOP-1269 URL: https://issues.apache.org/jira/browse/HADOOP-1269 Project: Hadoo

[jira] Commented: (HADOOP-1161) need improved release process

[ https://issues.apache.org/jira/browse/HADOOP-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489864 ] Nigel Daley commented on HADOOP-1161: - To summarize, I think we've roughly agreed to the following dev and relea

[jira] Updated: (HADOOP-1127) Speculative Execution and output of Reduce tasks

[ https://issues.apache.org/jira/browse/HADOOP-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1127: -- Attachment: HADOOP-1127_20070419_7.patch Thanks for the review Owen, here is patch incorporati

[jira] Created: (HADOOP-1268) Replace Vector types by ArrayList in FSDirectory

Replace Vector types by ArrayList in FSDirectory Key: HADOOP-1268 URL: https://issues.apache.org/jira/browse/HADOOP-1268 Project: Hadoop Issue Type: Improvement Affects Versions: 0.12.0, 0

[jira] Updated: (HADOOP-1268) Replace Vector types by ArrayList in FSDirectory

[ https://issues.apache.org/jira/browse/HADOOP-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HADOOP-1268: Component/s: dfs > Replace Vector types by ArrayList in FSDirectory >

[jira] Commented: (HADOOP-1066) http://lucene.apache.org/hadoop/ front page is not user-friendly

[ https://issues.apache.org/jira/browse/HADOOP-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489849 ] Nigel Daley commented on HADOOP-1066: - The -1 from Hadoop QA is due to the patch being html and not forrest xml.

[jira] Updated: (HADOOP-1066) http://lucene.apache.org/hadoop/ front page is not user-friendly

[ https://issues.apache.org/jira/browse/HADOOP-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Nicosia updated HADOOP-1066: -- Fix Version/s: 0.13.0 > http://lucene.apache.org/hadoop/ front page is not user-friendly > ---

Re: Checkstyle report

The Hadoop Patch build is also now running checkstyle, however with a modified style sheet (so I can easily scrape the produced html for the number of errors) and with a modified configuration file that currently only checks source code indentation. You can see the report for each build und

Checkstyle report

A checkstyle report is now being generated as part of the Hadoop Nightly build. This uses all the checkstyle rules in src/test/ checkstyle.xml The report is linked from the "Build Artifacts" section of each build. You can see the report linked from the latest build: http://lucene.zones.ap

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489837 ] Runping Qi commented on HADOOP-485: --- You want to control the partitioning as well. For example, let's we have the

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489836 ] Tahir Hashmi commented on HADOOP-485: - Sorry, read that as "...infrastructure for grouping to reduce based on som

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489835 ] Tahir Hashmi commented on HADOOP-485: - Does the infrastructure for grouping to reduce based on some properties of

[jira] Commented: (HADOOP-1267) change default config to be single node rather than "local" for both map/reduce and hdfs

[ https://issues.apache.org/jira/browse/HADOOP-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489834 ] Nigel Daley commented on HADOOP-1267: - >From Doug: We could include a confg/pseudo-cluster.xml file, that folks

Re: [jira] Commented: (HADOOP-1066) http://lucene.apache.org/hadoop/ front page is not user-friendly

Nigel Daley wrote: Agreed. I think what I was suggesting was all user focused documentation that answers user's questions such as: - how do I determine the latest stable release? - when can I expect the next release? - what is getting fixed/added in the next release? +1 Doug

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489832 ] Owen O'Malley commented on HADOOP-485: -- Sorry, I wasn't tracking this discussion fast enough to stop the huge d

Re: [jira] Commented: (HADOOP-1066) http://lucene.apache.org/hadoop/ front page is not user-friendly

On Apr 18, 2007, at 9:30 AM, Doug Cutting wrote: Nigel Daley wrote: Perhaps the releases section could be fleshed out with more info on our "stable" link, our roadmap (link to Jira roadmap?), and our release process (ie, what we agree to in HADOOP-1161). I agree that all this stuff should

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489829 ] Doug Cutting commented on HADOOP-485: - Sorry for spamming this issue, but I finally remembered the discussion tha

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489822 ] Doug Cutting commented on HADOOP-485: - Another simple approach to implementing this would be to, when a value co

[jira] Resolved: (HADOOP-1180) NNbench test should be able to test the checksumfilesystem as well as the raw filesystem

[ https://issues.apache.org/jira/browse/HADOOP-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur resolved HADOOP-1180. -- Resolution: Won't Fix We will continue to run NNbench with the checksum file system.

[jira] Commented: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489818 ] Doug Cutting commented on HADOOP-485: - First, I don't think we can assume that the key comparator will also work

[jira] Commented: (HADOOP-1267) change default config to be single node rather than "local" for both map/reduce and hdfs

[ https://issues.apache.org/jira/browse/HADOOP-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489813 ] Doug Cutting commented on HADOOP-1267: -- Why? Right now, things work out of the box w/o starting daemons, worry

Re: [jira] Commented: (HADOOP-1066) http://lucene.apache.org/hadoop/ front page is not user-friendly

Nigel Daley wrote: Perhaps the releases section could be fleshed out with more info on our "stable" link, our roadmap (link to Jira roadmap?), and our release process (ie, what we agree to in HADOOP-1161). I agree that all this stuff should be somewhere prominent on the site, but we need to b

[jira] Created: (HADOOP-1267) change default config to be single node rather than "local" for both map/reduce and hdfs

change default config to be single node rather than "local" for both map/reduce and hdfs Key: HADOOP-1267 URL: https://issues.apache.org/jira/browse/HADOOP-1267

Build failed in Hudson: Hadoop-Nightly #62

See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/62/ -- [...truncated 22640 lines...] [junit] 2007-04-18 15:56:52,348 INFO mapred.LocalJobRunner (LocalJobRunner.java:progress(188)) - http://lucene.zones.apache.org:8080/hudson/job/Hado

[jira] Commented: (HADOOP-1111) Job completion notification to a job configured URL

[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489797 ] Tom White commented on HADOOP-: --- On 18/04/07, Nigel Daley <[EMAIL PROTECTED]> wrote: > All the tests get test.

Re: [jira] Commented: (HADOOP-1111) Job completion notification to a job configured URL

All the tests get test.build.data from a system property. It's set by ant when the tests are run. On Apr 18, 2007, at 8:23 AM, Tom White (JIRA) wrote: [ https://issues.apache.org/jira/browse/HADOOP-? page=com.atlassian.jira.plugin.system.issuetabpanels:comment- tabpanel#action_124

[jira] Commented: (HADOOP-1111) Job completion notification to a job configured URL

[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489787 ] Tom White commented on HADOOP-: --- I think this should get test.build.data from the Hadoop Configuration object

[jira] Updated: (HADOOP-1190) Fix unchecked warnings

[ https://issues.apache.org/jira/browse/HADOOP-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-1190: -- Attachment: mapred-unchecked.patch > Fix unchecked warnings > -- > >

[jira] Updated: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-485: Attachment: TestUserValueGrouping.java.patch Here's a basic test for this requirement. The mapper g

[jira] Updated: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi updated HADOOP-485: Attachment: Hadoop-485-pre.patch Here's a proof-of-concept patch for this requirement. The intent i

[jira] Assigned: (HADOOP-485) allow a different comparator for grouping keys in calls to reduce

[ https://issues.apache.org/jira/browse/HADOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tahir Hashmi reassigned HADOOP-485: --- Assignee: Tahir Hashmi (was: Owen O'Malley) > allow a different comparator for grouping keys

[jira] Commented: (HADOOP-1154) streaming hang. (PipeMapRed$MROutputThread gone)

[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489694 ] Hadoop QA commented on HADOOP-1154: --- Integrated in Hadoop-Nightly #61 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-1190) Fix unchecked warnings

[ https://issues.apache.org/jira/browse/HADOOP-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489693 ] Hadoop QA commented on HADOOP-1190: --- Integrated in Hadoop-Nightly #61 (See http://lucene.zones.apache.org:8080/hu

[jira] Commented: (HADOOP-968) Reduce shuffle and merge should be done a child JVM

[ https://issues.apache.org/jira/browse/HADOOP-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489692 ] Hadoop QA commented on HADOOP-968: -- Integrated in Hadoop-Nightly #61 (See http://lucene.zones.apache.org:8080/hudso

[jira] Commented: (HADOOP-1148) re-indent all code

[ https://issues.apache.org/jira/browse/HADOOP-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489691 ] Hadoop QA commented on HADOOP-1148: --- Integrated in Hadoop-Nightly #61 (See http://lucene.zones.apache.org:8080/hu

Build failed in Hudson: Hadoop-Nightly #61

See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/61/ Changes: [tomwhite] HADOOP-1190. Fix unchecked warnings in fs and io packages. [tomwhite] HADOOP-1154. Fail a streaming task if the threads reading from or writing to the streaming process fail. Contributed by Koji Noguchi.

[jira] Updated: (HADOOP-1111) Job completion notification to a job configured URL

[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-: --- Status: Patch Available (was: Open) > Job completion notification to a job configur

[jira] Updated: (HADOOP-1111) Job completion notification to a job configured URL

[ https://issues.apache.org/jira/browse/HADOOP-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-: --- Attachment: patch-.txt changes based on Doug feedback > Job completion notifica