[jira] [Updated] (MAPREDUCE-1416) New JIRA components for Map/Reduce project
[ https://issues.apache.org/jira/browse/MAPREDUCE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated MAPREDUCE-1416: - Comment: was deleted (was: Rajesh, the patch looks good. Can you also add a negative testcase for skipped jobs? ) > New JIRA components for Map/Reduce project > -- > > Key: MAPREDUCE-1416 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1416 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Steve Loughran > > We need more JIRA components for the Map/Reduce project for better tracking. > Some missing ones: DistributedCache, TaskController, contrib/vaidya, > contrib/mruit, contrib/dynamic-scheduler, contrib/data_join. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration
[ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025637#comment-13025637 ] Amar Kamat commented on MAPREDUCE-1461: --- Rajesh, the patch looks good. Can you also add a negative testcase for skipped jobs? > Feature to instruct rumen-folder utility to skip jobs worth of specific > duration > > > Key: MAPREDUCE-1461 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: tools/rumen >Reporter: Rajesh Balamohan > Attachments: MR-1461-trunk.patch, mapreduce-1461--2010-02-05.patch, > mapreduce-1461--2010-03-04.patch > > > JSON outputs of rumen on production logs can be huge in the order of multiple > GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON > data. > It would be helpful to have an option in rumen-folder, wherein user can > specify a duration from which rumen-folder should start processing data. > Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1416) New JIRA components for Map/Reduce project
[ https://issues.apache.org/jira/browse/MAPREDUCE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025636#comment-13025636 ] Amar Kamat commented on MAPREDUCE-1416: --- Rajesh, the patch looks good. Can you also add a negative testcase for skipped jobs? > New JIRA components for Map/Reduce project > -- > > Key: MAPREDUCE-1416 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1416 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Steve Loughran > > We need more JIRA components for the Map/Reduce project for better tracking. > Some missing ones: DistributedCache, TaskController, contrib/vaidya, > contrib/mruit, contrib/dynamic-scheduler, contrib/data_join. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2422) Removed unused internal methods from DistributedCache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025549#comment-13025549 ] Hadoop QA commented on MAPREDUCE-2422: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477441/MAPREDUCE-2422.patch against trunk revision 1096649. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/186//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/186//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/186//console This message is automatically generated. > Removed unused internal methods from DistributedCache > - > > Key: MAPREDUCE-2422 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2422 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2422.patch, MAPREDUCE-2422.patch, > MAPREDUCE-2422.patch > > > DistributedCache has a number of deprecated methods that are no longer used > ever since TrackerDistributedCacheManager was introduced in MAPREDUCE-476. > Removing these methods (which are not user-facing) will make it possible to > complete MAPREDUCE-1638 by keeping DistributedCache in the API tree, and > TrackerDistributedCacheManager, TaskDistributedCacheManager in the > implementation tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2455) Remove deprecated JobTracker.State in favour of JobTrackerStatus
[ https://issues.apache.org/jira/browse/MAPREDUCE-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025530#comment-13025530 ] Hadoop QA commented on MAPREDUCE-2455: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477443/MAPREDUCE-2455.patch against trunk revision 1096649. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//console This message is automatically generated. > Remove deprecated JobTracker.State in favour of JobTrackerStatus > > > Key: MAPREDUCE-2455 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2455 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: build, client >Reporter: Tom White >Assignee: Tom White > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2455.patch > > > MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue > is to remove the getter (in favour of getJobTrackerStatus(), which will > remain) so there is no longer a direct dependency of the public API on > JobTracker. This is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2455) Remove deprecated JobTracker.State in favour of JobTrackerStatus
[ https://issues.apache.org/jira/browse/MAPREDUCE-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2455: - Status: Patch Available (was: Open) > Remove deprecated JobTracker.State in favour of JobTrackerStatus > > > Key: MAPREDUCE-2455 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2455 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: build, client >Reporter: Tom White >Assignee: Tom White > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2455.patch > > > MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue > is to remove the getter (in favour of getJobTrackerStatus(), which will > remain) so there is no longer a direct dependency of the public API on > JobTracker. This is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2455) Remove deprecated JobTracker.State in favour of JobTrackerStatus
[ https://issues.apache.org/jira/browse/MAPREDUCE-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2455: - Attachment: MAPREDUCE-2455.patch Here's a patch for this change. > Remove deprecated JobTracker.State in favour of JobTrackerStatus > > > Key: MAPREDUCE-2455 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2455 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: build, client >Reporter: Tom White >Assignee: Tom White > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2455.patch > > > MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue > is to remove the getter (in favour of getJobTrackerStatus(), which will > remain) so there is no longer a direct dependency of the public API on > JobTracker. This is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2455) Remove deprecated JobTracker.State in favour of JobTrackerStatus
Remove deprecated JobTracker.State in favour of JobTrackerStatus Key: MAPREDUCE-2455 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2455 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Tom White Assignee: Tom White Fix For: 0.23.0 MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue is to remove the getter (in favour of getJobTrackerStatus(), which will remain) so there is no longer a direct dependency of the public API on JobTracker. This is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2421) Remove JobHistory's dependency on JobTracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025447#comment-13025447 ] Hadoop QA commented on MAPREDUCE-2421: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477440/MAPREDUCE-2421.patch against trunk revision 1096649. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/185//testReport/ Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/185//console This message is automatically generated. > Remove JobHistory's dependency on JobTracker > > > Key: MAPREDUCE-2421 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2421 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2421.patch > > > The direct dependency can be removed by introducing a callback inteface. This > is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2422) Removed unused internal methods from DistributedCache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2422: - Attachment: MAPREDUCE-2422.patch New patch to add a (@Private) ClientDistributedCacheManager, which is used by the client-side job submission process. > Removed unused internal methods from DistributedCache > - > > Key: MAPREDUCE-2422 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2422 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2422.patch, MAPREDUCE-2422.patch, > MAPREDUCE-2422.patch > > > DistributedCache has a number of deprecated methods that are no longer used > ever since TrackerDistributedCacheManager was introduced in MAPREDUCE-476. > Removing these methods (which are not user-facing) will make it possible to > complete MAPREDUCE-1638 by keeping DistributedCache in the API tree, and > TrackerDistributedCacheManager, TaskDistributedCacheManager in the > implementation tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2421) Remove JobHistory's dependency on JobTracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2421: - Attachment: MAPREDUCE-2421.patch This patch introduces a callback interface to remove the dependency. > Remove JobHistory's dependency on JobTracker > > > Key: MAPREDUCE-2421 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2421 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2421.patch > > > The direct dependency can be removed by introducing a callback inteface. This > is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2421) Remove JobHistory's dependency on JobTracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2421: - Status: Patch Available (was: Open) > Remove JobHistory's dependency on JobTracker > > > Key: MAPREDUCE-2421 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2421 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2421.patch > > > The direct dependency can be removed by introducing a callback inteface. This > is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025422#comment-13025422 ] Owen O'Malley commented on MAPREDUCE-2454: -- You should also look at the work in MAPREDUCE-279. Once the MapReduce library is user code there are a lot more options available. > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Mariappan Asokan >Priority: Minor > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025386#comment-13025386 ] Mariappan Asokan commented on MAPREDUCE-2454: - Sorry about that Chris. Thanks for moving. -- Asokan On 04/26/2011 03:01 PM, Chris Douglas (JIRA) wrote: [ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas moved HADOOP-7242 to MAPREDUCE-2454: -- Affects Version/s: (was: 0.21.0) Key: MAPREDUCE-2454 (was: HADOOP-7242) Project: Hadoop Map/Reduce (was: Hadoop Common) Allow external sorter plugin for MR --- Key: MAPREDUCE-2454 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Mariappan Asokan Priority: Minor Define interfaces and some abstract classes in the Hadoop framework to facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Mariappan Asokan >Priority: Minor > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas moved HADOOP-7242 to MAPREDUCE-2454: -- Affects Version/s: (was: 0.21.0) Key: MAPREDUCE-2454 (was: HADOOP-7242) Project: Hadoop Map/Reduce (was: Hadoop Common) > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Mariappan Asokan >Priority: Minor > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2428) start-mapred.sh script fails if HADOOP_HOME is not set
[ https://issues.apache.org/jira/browse/MAPREDUCE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025210#comment-13025210 ] Hudson commented on MAPREDUCE-2428: --- Integrated in Hadoop-Mapreduce-22-branch #42 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/42/]) MAPREDUCE-2428. svn merge -c 1096649 from trunk > start-mapred.sh script fails if HADOOP_HOME is not set > -- > > Key: MAPREDUCE-2428 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2428 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.21.0 >Reporter: Tom White >Assignee: Tom White >Priority: Blocker > Fix For: 0.22.0, 0.23.0 > > Attachments: MAPREDUCE-2428.patch > > > MapReduce portion of HADOOP-6953 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
[
https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025209#comment-13025209
]
Hudson commented on MAPREDUCE-2317:
---
Integrated in Hadoop-Mapreduce-22-branch #42 (See
[https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/42/])
> HadoopArchives throwing NullPointerException while creating hadoop archives
> (.har files)
>
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
>Affects Versions: 0.20.3
> Environment: windows
>Reporter: Devaraj K
>Assignee: Devaraj K
>Priority: Minor
> Fix For: 0.20.4, 0.21.1, 0.22.0, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch,
> MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it
> is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in
> E:/temp
> {code:xml}
> java.lang.NullPointerException
> at
> org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at
> org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
