[jira] [Commented] (MAPREDUCE-3431) NPE in Resource Manager shutdown
[
https://issues.apache.org/jira/browse/MAPREDUCE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153835#comment-13153835
]
Steve Loughran commented on MAPREDUCE-3431:
---
the root cause of this is the service {{stop()}} method assumes the service
started up correctly, and hence that all member fields are non-null. Each field
should be assessed for null-ness before use, and set to null afterwards, to
stop re-entrant calls. All yarn services will need reviewing. Testing is hard
as it's tricky to generate all code-paths needed to stress every action in the
{{stop()}} method
NPE in Resource Manager shutdown
Key: MAPREDUCE-3431
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3431
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.0
Environment: OS/X w/ java6
Reporter: Steve Loughran
Priority: Minor
bringing up a resource manager failed; shutdown triggered an NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2377) task-controller fails to parse configuration if it doesn't end in \n
[ https://issues.apache.org/jira/browse/MAPREDUCE-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-2377: --- Component/s: task-controller Priority: Major (was: Critical) Target Version/s: 0.22.0 Fix Version/s: (was: 0.22.0) 0.20.206.0 Hadoop Flags: Reviewed I've committed to branch-20-security. Leaving open for 22. task-controller fails to parse configuration if it doesn't end in \n Key: MAPREDUCE-2377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2377 Project: Hadoop Map/Reduce Issue Type: Bug Components: task-controller Affects Versions: 0.22.0 Reporter: Todd Lipcon Fix For: 0.20.206.0 Attachments: mr-2377-20.txt If the task-controller.cfg file doesn't end in a newline, it fails to parse properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2377) task-controller fails to parse configuration if it doesn't end in \n
[ https://issues.apache.org/jira/browse/MAPREDUCE-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153875#comment-13153875 ] Eli Collins commented on MAPREDUCE-2377: Forgot to mention, aside from the test I also tested a build from a tarball running a couple MR jobs using the LTC. task-controller fails to parse configuration if it doesn't end in \n Key: MAPREDUCE-2377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2377 Project: Hadoop Map/Reduce Issue Type: Bug Components: task-controller Affects Versions: 0.22.0 Reporter: Todd Lipcon Fix For: 0.20.206.0 Attachments: mr-2377-20.txt If the task-controller.cfg file doesn't end in a newline, it fails to parse properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2059) RecoveryManager attempts to add jobtracker.info
[ https://issues.apache.org/jira/browse/MAPREDUCE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-2059: --- Attachment: MAPREDUCE-2059.patch I was impatient. It runs for about 5 minutes. But the new test was failing, because the previous test case testJobTrackerInfoCreation() was not closing MiniDFSCluter. I added the shutdown statement, and cleaned up some deprecations in the new test. Also change job completion threshold from 50% to 20%, which reduced running time from 290 sec to 150. RecoveryManager attempts to add jobtracker.info --- Key: MAPREDUCE-2059 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2059 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.203.0, 0.22.0 Environment: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 Reporter: Dan Adkins Labels: hadoop Fix For: 0.22.0 Attachments: MAPREDUCE-2059.patch, MAPREDUCE-2059.patch The jobtracker is treating the file 'jobtracker.info' in the system data directory as a job to be recovered, resulting in the following: 10/09/09 18:06:02 WARN mapred.JobTracker: Failed to add the job jobtracker.info java.lang.IllegalArgumentException: JobId string : jobtracker.info is not properly formed at org.apache.hadoop.mapreduce.JobID.forName(JobID.java:158) at org.apache.hadoop.mapred.JobID.forName(JobID.java:84) at org.apache.hadoop.mapred.JobTracker$RecoveryManager.addJobForRecovery(JobTracker.java:1057) at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1565) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:275) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:267) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:262) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4256) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2059) RecoveryManager attempts to add jobtracker.info
[ https://issues.apache.org/jira/browse/MAPREDUCE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153882#comment-13153882 ] Konstantin Shvachko commented on MAPREDUCE-2059: Otherwise code looks good +1. RecoveryManager attempts to add jobtracker.info --- Key: MAPREDUCE-2059 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2059 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.203.0, 0.22.0 Environment: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 Reporter: Dan Adkins Labels: hadoop Fix For: 0.22.0 Attachments: MAPREDUCE-2059.patch, MAPREDUCE-2059.patch The jobtracker is treating the file 'jobtracker.info' in the system data directory as a job to be recovered, resulting in the following: 10/09/09 18:06:02 WARN mapred.JobTracker: Failed to add the job jobtracker.info java.lang.IllegalArgumentException: JobId string : jobtracker.info is not properly formed at org.apache.hadoop.mapreduce.JobID.forName(JobID.java:158) at org.apache.hadoop.mapred.JobID.forName(JobID.java:84) at org.apache.hadoop.mapred.JobTracker$RecoveryManager.addJobForRecovery(JobTracker.java:1057) at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1565) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:275) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:267) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:262) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4256) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2059) RecoveryManager attempts to add jobtracker.info
[ https://issues.apache.org/jira/browse/MAPREDUCE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153890#comment-13153890 ] Konstantin Shvachko commented on MAPREDUCE-2059: I just committed this to 0.22 branch. Thanks Subroto. Keeping it open until the inclusion to 0.20.security is decided. RecoveryManager attempts to add jobtracker.info --- Key: MAPREDUCE-2059 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2059 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.203.0, 0.22.0 Environment: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 Reporter: Dan Adkins Labels: hadoop Fix For: 0.22.0 Attachments: MAPREDUCE-2059.patch, MAPREDUCE-2059.patch The jobtracker is treating the file 'jobtracker.info' in the system data directory as a job to be recovered, resulting in the following: 10/09/09 18:06:02 WARN mapred.JobTracker: Failed to add the job jobtracker.info java.lang.IllegalArgumentException: JobId string : jobtracker.info is not properly formed at org.apache.hadoop.mapreduce.JobID.forName(JobID.java:158) at org.apache.hadoop.mapred.JobID.forName(JobID.java:84) at org.apache.hadoop.mapred.JobTracker$RecoveryManager.addJobForRecovery(JobTracker.java:1057) at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1565) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:275) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:267) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:262) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4256) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2059) RecoveryManager attempts to add jobtracker.info
[ https://issues.apache.org/jira/browse/MAPREDUCE-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153932#comment-13153932 ] Hudson commented on MAPREDUCE-2059: --- Integrated in Hadoop-Mapreduce-22-branch #91 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/91/]) MAPREDUCE-2059. RecoveryManager excludes jobtracker.info from the list of jobs to be recovered. Contributed by Subroto Sanyal. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204275 Files : * /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt * /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestRecoveryManager.java RecoveryManager attempts to add jobtracker.info --- Key: MAPREDUCE-2059 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2059 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.203.0, 0.22.0 Environment: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 Reporter: Dan Adkins Labels: hadoop Fix For: 0.22.0 Attachments: MAPREDUCE-2059.patch, MAPREDUCE-2059.patch The jobtracker is treating the file 'jobtracker.info' in the system data directory as a job to be recovered, resulting in the following: 10/09/09 18:06:02 WARN mapred.JobTracker: Failed to add the job jobtracker.info java.lang.IllegalArgumentException: JobId string : jobtracker.info is not properly formed at org.apache.hadoop.mapreduce.JobID.forName(JobID.java:158) at org.apache.hadoop.mapred.JobID.forName(JobID.java:84) at org.apache.hadoop.mapred.JobTracker$RecoveryManager.addJobForRecovery(JobTracker.java:1057) at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1565) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:275) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:267) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:262) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4256) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3121) NodeManager should handle disk-failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153950#comment-13153950 ] Eli Collins commented on MAPREDUCE-3121: Looks like the change has similar assumptions as MR1, eg the boot disk is either raided or we're using a health checker script to stop the services if the boot disk fails. Worth mentioning this in the docs. I think it would make more sense to name the classes LocalDir* instead of Disk* since we're checking local dirs and not disks. For example, we only check the given dirs so if there's a failure on another sector of the disk it won't notice. The NM won't handle boot disk failures even if it detects a failure on a dir hosted on the boot disk because it's dir-centric (ie doesn't know that the disk has failed, just that a dir has). Similarly the local dirs and log dirs may of course reside on the same disk so if we were checking disks we wouldn't need to check them independently. The DN calls this volume checking for the same rationale, something similar here would make sense as well. I'd call it LocalDirChecker and have it live in common next to LocalDirAllocator. This way HDFS could re-use the code. 5% seems pretty low. How did you arrive at that? Are you sure you want a 12 disk host with only 1 working disk to keep running? NodeManager should handle disk-failures --- Key: MAPREDUCE-3121 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3121 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Ravi Gummadi Priority: Blocker Fix For: 0.23.1 Attachments: 3121.patch, 3121.v1.1.patch, 3121.v1.patch, 3121.v2.patch This is akin to MAPREDUCE-2413 but for YARN's NodeManager. We want to minimize the impact of transient/permanent disk failures on containers. With larger number of disks per node, the ability to continue to run containers on other disks is crucial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3438) TestRaidNode fails because of Too many open files
TestRaidNode fails because of Too many open files --- Key: MAPREDUCE-3438 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3438 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/raid Affects Versions: 0.22.0 Reporter: Konstantin Shvachko Fix For: 0.22.0 TestRaidNode fails because it opens many connections. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3438) TestRaidNode fails because of Too many open files
[
https://issues.apache.org/jira/browse/MAPREDUCE-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153953#comment-13153953
]
Konstantin Shvachko commented on MAPREDUCE-3438:
This is the last failing test for 0.22. See last several builds for
Hadoop-Mapreduce-22-branch.
The failure is because of the following exception:
{code}
11/11/21 01:05:26 INFO hdfs.DFSClient: Failed to connect to /127.0.0.1:45905,
add to deadNodes and continue
java.net.SocketException: Too many open files
at sun.nio.ch.Net.socket0(Native Method)
at sun.nio.ch.Net.socket(Net.java:97)
at sun.nio.ch.SocketChannelImpl.init(SocketChannelImpl.java:84)
at
sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:37)
at java.nio.channels.SocketChannel.open(SocketChannel.java:105)
at
org.apache.hadoop.net.StandardSocketFactory.createSocket(StandardSocketFactory.java:63)
at
org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:702)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:390)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)
at java.io.DataInputStream.read(DataInputStream.java:132)
at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:122)
at org.apache.hadoop.raid.RaidUtils.copyBytes(RaidUtils.java:93)
at org.apache.hadoop.raid.Decoder.decodeFile(Decoder.java:133)
at org.apache.hadoop.raid.RaidNode.unRaid(RaidNode.java:867)
at org.apache.hadoop.raid.RaidNode.recoverFile(RaidNode.java:333)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:349)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1482)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1478)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1476)
{code}
Which leads to {{BlockMissingException}} and failure of
{{TestRaidNode.testPathFilter}} in the end.
The fix is either
# to increase ulimit on Jenkins machines, which I did on my box and everything
passed, or
# to scale down the test itself.
TestRaidNode fails because of Too many open files
---
Key: MAPREDUCE-3438
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3438
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/raid
Affects Versions: 0.22.0
Reporter: Konstantin Shvachko
Fix For: 0.22.0
TestRaidNode fails because it opens many connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3424) MR-2415 cleanup
[ https://issues.apache.org/jira/browse/MAPREDUCE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153970#comment-13153970 ] Todd Lipcon commented on MAPREDUCE-3424: Looks good. Did you run MR unit tests and the task-controller tests locally? MR-2415 cleanup --- Key: MAPREDUCE-3424 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3424 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Affects Versions: 0.20.205.0 Reporter: Eli Collins Assignee: Eli Collins Priority: Minor Attachments: mapreduce-3424-1.patch, mapreduce-3424-2.patch MR-2415 had some tabs and weird indenting and spacing. Also would be more clear if LTC explicitly overrides createLogDir. Let's clean this up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3438) TestRaidNode fails because of Too many open files
[ https://issues.apache.org/jira/browse/MAPREDUCE-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153972#comment-13153972 ] Konstantin Boudnik commented on MAPREDUCE-3438: --- +1 on the first option. Jenkins slaves are using default settings for ulimit, which isn't a viable option once you're dealing with applications at scale. TestRaidNode fails because of Too many open files --- Key: MAPREDUCE-3438 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3438 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/raid Affects Versions: 0.22.0 Reporter: Konstantin Shvachko Fix For: 0.22.0 TestRaidNode fails because it opens many connections. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3439) [Gridmix] Support addons in Gridmix
[Gridmix] Support addons in Gridmix --- Key: MAPREDUCE-3439 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3439 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Fix For: 0.24.0 At times there is a need to benchmark certain Hadoop client APIs. Often, this is done by running simple standard sort-like programs on Hadoop and then using an external utility to benchmark the APIs. But then the benchmarking results tend to be off from reality as the load on the cluster doesn't match the actual load. We believe that Gridmix3 - which is a Hadoop workload simulator - can prove useful here. Gridmix3 already provides a mechanism to load the cluster - often called as a 'test cluster' - using a real trace thus mimicking the real-life workload. Currently, Gridmix3 consumes a representative workload trace and loads the Hadoop cluster to match what is seen in the trace. Gridmix3 can be enhanced to also support user scripts (hereby referred as 'addons') which will be loaded within Gridmix3 and will get updates like 1. Job submission 2. Job completion 3. Cluster status These addons can also ping/access a live, close-to-real-life Hadoop cluster. This will allow users to benchmark the Hadoop cluster while it is running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3424) MR-2415 cleanup
[ https://issues.apache.org/jira/browse/MAPREDUCE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-3424: --- Attachment: mapreduce-3424-3.patch Updated patch attached. Minor update on the last one. I ran the tt test as well as the full suite and ran some jobs on 3 nodes w the the ltc enabled. test-patch results comming. MR-2415 cleanup --- Key: MAPREDUCE-3424 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3424 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Affects Versions: 0.20.205.0 Reporter: Eli Collins Assignee: Eli Collins Priority: Minor Attachments: mapreduce-3424-1.patch, mapreduce-3424-2.patch, mapreduce-3424-3.patch MR-2415 had some tabs and weird indenting and spacing. Also would be more clear if LTC explicitly overrides createLogDir. Let's clean this up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3424) Some LinuxTaskController cleanup
[ https://issues.apache.org/jira/browse/MAPREDUCE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-3424: --- Summary: Some LinuxTaskController cleanup (was: MR-2415 cleanup) Some LinuxTaskController cleanup Key: MAPREDUCE-3424 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3424 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Affects Versions: 0.20.205.0 Reporter: Eli Collins Assignee: Eli Collins Priority: Minor Attachments: mapreduce-3424-1.patch, mapreduce-3424-2.patch, mapreduce-3424-3.patch MR-2415 had some tabs and weird indenting and spacing. Also would be more clear if LTC explicitly overrides createLogDir. Let's clean this up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/MAPREDUCE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-3437: --- Resolution: Fixed Fix Version/s: 0.23.1 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch-0.23 Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3437-branch-0.23.patch [INFO] Scanning for projects... [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/MAPREDUCE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154021#comment-13154021 ] Todd Lipcon commented on MAPREDUCE-3437: +1, thanks Jonathan. Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3437-branch-0.23.patch [INFO] Scanning for projects... [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3424) Some LinuxTaskController cleanup
[
https://issues.apache.org/jira/browse/MAPREDUCE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154023#comment-13154023
]
Eli Collins commented on MAPREDUCE-3424:
{noformat}
[exec]
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new
or modified tests.
[exec] Please justify why no tests are needed for
this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] -1 findbugs. The patch appears to introduce 2 new Findbugs
(version 1.3.9) warnings.
[exec]
{noformat}
The findbugs warning came from a previous change (see them w/ a null patch).
Existing tests suffice since this is cleanup.
Some LinuxTaskController cleanup
Key: MAPREDUCE-3424
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3424
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: tasktracker
Affects Versions: 0.20.205.0
Reporter: Eli Collins
Assignee: Eli Collins
Priority: Minor
Attachments: mapreduce-3424-1.patch, mapreduce-3424-2.patch,
mapreduce-3424-3.patch
MR-2415 had some tabs and weird indenting and spacing. Also would be more
clear if LTC explicitly overrides createLogDir. Let's clean this up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/MAPREDUCE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154025#comment-13154025 ] Hudson commented on MAPREDUCE-3437: --- Integrated in Hadoop-Common-0.23-Commit #191 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/191/]) MAPREDUCE-3437. Fix examples pom to refer to the correct 0.23 snapshot version. Contributed by Jonathan Eagles. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204368 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3437-branch-0.23.patch [INFO] Scanning for projects... [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/MAPREDUCE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154029#comment-13154029 ] Hudson commented on MAPREDUCE-3437: --- Integrated in Hadoop-Mapreduce-0.23-Commit #202 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/202/]) MAPREDUCE-3437. Fix examples pom to refer to the correct 0.23 snapshot version. Contributed by Jonathan Eagles. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204368 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3437-branch-0.23.patch [INFO] Scanning for projects... [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/MAPREDUCE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154028#comment-13154028 ] Hudson commented on MAPREDUCE-3437: --- Integrated in Hadoop-Hdfs-0.23-Commit #189 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/189/]) MAPREDUCE-3437. Fix examples pom to refer to the correct 0.23 snapshot version. Contributed by Jonathan Eagles. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204368 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3437-branch-0.23.patch [INFO] Scanning for projects... [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
