[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Status: Open (was: Patch Available) > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.2 > > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch, > MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Attachment: MR-3796.3.patch Made min/max settings global at the RM level instead of the per-scheduler settings. > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.2 > > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch, > MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Status: Patch Available (was: Open) > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.2 > > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch, > MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Hadoop Flags: Incompatible change > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.2 > > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch, > MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3941) Improve distributed shell application to work on a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3941: --- Description: (was: Improve the distributed shell application to be able to work on a secure cluster.) Summary: Improve distributed shell application to work on a secure cluster (was: Improve distributed shell example to work on a secure cluster) > Improve distributed shell application to work on a secure cluster > - > > Key: MAPREDUCE-3941 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3941 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Hitesh Shah >Assignee: Hitesh Shah > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3922) Document solution for potential problem compiling 32 bit binaries on a x86_64 host.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3922:
---
Attachment: MR-3922.1.patch
Attaching a patch which actually addresses the issue in question. The pom now
has a placeholder which can be modified as per the build's requirements.
Pass -Dcontainer-executor.additional_cflags=" -m32 " if a 32-bit build is
needed. By default, this value is unset.
> Document solution for potential problem compiling 32 bit binaries on a x86_64
> host.
> ---
>
> Key: MAPREDUCE-3922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3922
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
>Priority: Minor
> Labels: bigtop, build, doc, mrv2
> Attachments: MAPREDUCE-3922.patch, MR-3922.1.patch
>
>
> MAPREDUCE-3880 specifies that {{-m32}} be passed to gcc to ensure that a
> 32-bit binary is built. On my Amazon Linux with a x86_64 architecture, I
> could not build the native container-executor because the configure script
> exited with an error when trying to use gcc (version 4.4.5) with this flag.
> Adds a comment to
> {{hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml}}
> regarding package requirement for glibc-devel.i686 or glibc-devel.i386
> package on yum-based (RHEL, Centos, Amazon) Linux distributions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3614) finalState UNDEFINED if AM is killed by hand
[
https://issues.apache.org/jira/browse/MAPREDUCE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3614:
---
Summary: finalState UNDEFINED if AM is killed by hand (was: 55)
> finalState UNDEFINED if AM is killed by hand
> -
>
> Key: MAPREDUCE-3614
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3614
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: MAPREDUCE-3614.branch-0.23.patch
>
>
> Courtesy [~dcapwell]
> {quote}
> If the AM is running and you kill the process (sudo kill #pid), the State in
> Yarn would be FINISHED and FinalStatus is UNDEFINED. The Tracking UI would
> say "History" and point to the proxy url (which will redirect to the history
> server).
> The state should be more descriptive that the job failed and the tracker url
> shouldn't point to the history server.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Attachment: MR-3796.2.patch max/min allocation field variables are added into findbugs-exclude as they are initialized only once and from then read-only. > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Open (was: Patch Available) > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.1.patch, MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Attachment: MR-3709.2.patch Real fix this time. > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah > Attachments: MR-3709.1.patch, MR-3709.2.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Status: Patch Available (was: Open) > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah > Attachments: MR-3709.1.patch, MR-3709.2.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Status: Open (was: Patch Available) > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah > Attachments: MR-3709.1.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Status: Patch Available (was: Open) > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah > Attachments: MR-3709.1.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Priority: Major (was: Blocker) > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah > Attachments: MR-3709.1.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3709: --- Attachment: MR-3709.1.patch @Eli, given that MAPREDUCE-3765 is now in, the latest patch should be able to use smaller sized containers which hopefully should solve the problem. Could you please try the patch and let me know if it fixes the issue in your env. I am also downgrading this to a non-blocker as it is not reproducible in most environments. Please let me know if you see any issues with doing this. > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah >Priority: Blocker > Attachments: MR-3709.1.patch > > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Attachment: MR-3796.1.patch > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.1.patch, MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.1.patch, MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3796: --- Attachment: MR-3796.wip.patch Still a work in progress. Tests pending. Changing the state machine slightly in case anyone wants to take a quick look. > Scheduler.normalizeRequest does not account for allocation requests that > exceed maximumAllocation limits > - > > Key: MAPREDUCE-3796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Attachments: MR-3796.wip.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Attachment: MR-3765.3.patch Added a findbugs exclude warning for minimumAllocation error. The minimumAllocation is initialized only once in reinitialize() and never changed on subsequent calls once initialized. > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3775: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3775: --- Attachment: MR-3775.1.patch Trivial patch to fix cases where nested classes are used with miniyarncluster. > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Open (was: Patch Available) Cancelling patch to re-work the way minAlloc and maxAlloc can be changed at run-time > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3780) RM assigns containers to killed applications
[ https://issues.apache.org/jira/browse/MAPREDUCE-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3780: --- Attachment: MR-3780.1.patch > RM assigns containers to killed applications > > > Key: MAPREDUCE-3780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3780 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3780.1.patch > > > RM attempts to assign containers to killed applications. The applications > were killed when they were inactive and waiting for AM allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3780) RM assigns containers to killed applications
[ https://issues.apache.org/jira/browse/MAPREDUCE-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3780: --- Status: Patch Available (was: Open) > RM assigns containers to killed applications > > > Key: MAPREDUCE-3780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3780 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3780.1.patch > > > RM attempts to assign containers to killed applications. The applications > were killed when they were inactive and waiting for AM allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3780) RM assigns containers to killed applications
[ https://issues.apache.org/jira/browse/MAPREDUCE-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3780: --- Priority: Blocker (was: Major) > RM assigns containers to killed applications > > > Key: MAPREDUCE-3780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3780 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > > RM attempts to assign containers to killed applications. The applications > were killed when they were inactive and waiting for AM allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3427) streaming tests fail with MR2
[
https://issues.apache.org/jira/browse/MAPREDUCE-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3427:
---
Status: Patch Available (was: Open)
> streaming tests fail with MR2
> -
>
> Key: MAPREDUCE-3427
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3427
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming, mrv2
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Alejandro Abdelnur
>Assignee: Hitesh Shah
>Priority: Blocker
> Fix For: 0.23.1, 0.24.0
>
> Attachments: MR-3427.1.patch
>
>
> After Mavenizing streaming and getting its testcases to use the MiniMRCluster
> wrapper (MAPREDUCE-3169), 4 testcases fail to pass.
> Following is an assessment of those failures. Note that the testcases have
> been tweaked only to set the streaming JAR and yarn as the framework.
>
> (If these issues are unrelated we should create sub-tasks for each one of
> them).
> *TestStreamingCombiner*, fails because returned counters don't match
> assertion. However, counters printed in the test output indicate values that
> would satisfy the assertion. As Tom has indicated it seems MR/YARN are not
> passing back counter information to the client API.
> *TestStreamingBadRecords*, the job is failing with the following exception
> {code}
> Application application_1321575850006_0001 failed 1 times due to AM Container
> for
> appattempt_1321575850006_0001_01 exited with exitCode: 127 due to:
> .Failing this attempt.. Failing the application.
> {code}
> Difficult to troubleshoot because there are not task logs from Mini MR/YARN
> run.
> *TestStreamingStatus* fails in validateTaskStatus() in the following assertion
> {code}
> expected:<[before consuming input > sort]> but was:<[SUCCEEDED]>
> {code}
> *TestUlimit* fails with
> {code}
> org.junit.ComparisonFailure: output is wrong expected:<[786432]> but
> was:<[unlimited]>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3427) streaming tests fail with MR2
[
https://issues.apache.org/jira/browse/MAPREDUCE-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3427:
---
Attachment: MR-3427.1.patch
Patch to comment out testulimit and teststreamingbadrecords tests.
> streaming tests fail with MR2
> -
>
> Key: MAPREDUCE-3427
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3427
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming, mrv2
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Alejandro Abdelnur
>Assignee: Hitesh Shah
>Priority: Blocker
> Fix For: 0.23.1, 0.24.0
>
> Attachments: MR-3427.1.patch
>
>
> After Mavenizing streaming and getting its testcases to use the MiniMRCluster
> wrapper (MAPREDUCE-3169), 4 testcases fail to pass.
> Following is an assessment of those failures. Note that the testcases have
> been tweaked only to set the streaming JAR and yarn as the framework.
>
> (If these issues are unrelated we should create sub-tasks for each one of
> them).
> *TestStreamingCombiner*, fails because returned counters don't match
> assertion. However, counters printed in the test output indicate values that
> would satisfy the assertion. As Tom has indicated it seems MR/YARN are not
> passing back counter information to the client API.
> *TestStreamingBadRecords*, the job is failing with the following exception
> {code}
> Application application_1321575850006_0001 failed 1 times due to AM Container
> for
> appattempt_1321575850006_0001_01 exited with exitCode: 127 due to:
> .Failing this attempt.. Failing the application.
> {code}
> Difficult to troubleshoot because there are not task logs from Mini MR/YARN
> run.
> *TestStreamingStatus* fails in validateTaskStatus() in the following assertion
> {code}
> expected:<[before consuming input > sort]> but was:<[SUCCEEDED]>
> {code}
> *TestUlimit* fails with
> {code}
> org.junit.ComparisonFailure: output is wrong expected:<[786432]> but
> was:<[unlimited]>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Attachment: MR-3765.2.patch > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Open (was: Patch Available) > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3756) Make single shuffle limit configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3756: --- Attachment: MR-3756.1.patch Trivial patch. > Make single shuffle limit configurable > -- > > Key: MAPREDUCE-3756 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3756 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Hitesh Shah > Attachments: MR-3756.1.patch > > > Make single shuffle limit configurable, currently it's hard-coded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3756) Make single shuffle limit configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3756: --- Status: Patch Available (was: Open) > Make single shuffle limit configurable > -- > > Key: MAPREDUCE-3756 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3756 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Hitesh Shah > Attachments: MR-3756.1.patch > > > Make single shuffle limit configurable, currently it's hard-coded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Attachment: MR-3765.1.patch Addressing issue seen when looking into MR-3709 > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3765: --- Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Status: Patch Available (was: Open) > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Attachment: MR-3699.3.patch Also, add default val for MR-AM heartbeat to RM into mapred-default.xml > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch, MR-3699.2.patch, MR-3699.3.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Status: Open (was: Patch Available) > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch, MR-3699.2.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Attachment: MR-3699.2.patch modified yarn-default.xml and added missing entry to mapred-default.xml > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch, MR-3699.2.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Status: Patch Available (was: Open) > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch, MR-3699.2.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Status: Patch Available (was: Open) > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3699) Default RPC handlers are very low for YARN servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3699: --- Attachment: MR-3699.1.patch Trivial patch based on findings from performance benchmarks run on MRv2. > Default RPC handlers are very low for YARN servers > -- > > Key: MAPREDUCE-3699 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3699 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3699.1.patch > > > Mainly NM has a default of 5, RM has 10 and AM also has 10 irrespective of > num-slots, num-nodes and num-tasks respectively. Though ideally we want to > scale according to slots/nodes/tasks, for now increasing the defaults should > be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3718) Default AM heartbeat interval should be one second
[ https://issues.apache.org/jira/browse/MAPREDUCE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3718: --- Attachment: MR-3718.1.patch Trivial patch based on findings from performance benchmarks. > Default AM heartbeat interval should be one second > -- > > Key: MAPREDUCE-3718 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3718 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3718.1.patch > > > Helps in improving app performance. RM should be able to handle this, as the > heartbeats aren't really costly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3718) Default AM heartbeat interval should be one second
[ https://issues.apache.org/jira/browse/MAPREDUCE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3718: --- Status: Patch Available (was: Open) > Default AM heartbeat interval should be one second > -- > > Key: MAPREDUCE-3718 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3718 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3718.1.patch > > > Helps in improving app performance. RM should be able to handle this, as the > heartbeats aren't really costly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3426: --- Status: Patch Available (was: Open) > uber-jobs tried to write outputs into wrong dir > --- > > Key: MAPREDUCE-3426 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3426.1.patch, MR-3426.2.patch, MR-3426.3.patch > > > Incorrect setup of the uber tasks causes tasks to try to write intermidiate > outputs into dirs that the user does not have permissions to write to on a > secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3426: --- Attachment: MR-3426.3.patch addressed comments. > uber-jobs tried to write outputs into wrong dir > --- > > Key: MAPREDUCE-3426 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3426.1.patch, MR-3426.2.patch, MR-3426.3.patch > > > Incorrect setup of the uber tasks causes tasks to try to write intermidiate > outputs into dirs that the user does not have permissions to write to on a > secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3426: --- Attachment: MR-3426.2.patch Re-based patch. > uber-jobs tried to write outputs into wrong dir > --- > > Key: MAPREDUCE-3426 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3426.1.patch, MR-3426.2.patch > > > Incorrect setup of the uber tasks causes tasks to try to write intermidiate > outputs into dirs that the user does not have permissions to write to on a > secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3426: --- Attachment: MR-3426.1.patch > uber-jobs tried to write outputs into wrong dir > --- > > Key: MAPREDUCE-3426 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3426.1.patch > > > Incorrect setup of the uber tasks causes tasks to try to write intermidiate > outputs into dirs that the user does not have permissions to write to on a > secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3465) org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23
[ https://issues.apache.org/jira/browse/MAPREDUCE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3465: --- Status: Patch Available (was: Open) > org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23 > > > Key: MAPREDUCE-3465 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3465 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3465.1.patch > > > Running org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin > Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.121 sec <<< > FAILURE! > Tests in error: > > testParsingProcStatAndCpuFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): > > /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/CPUINFO_943711651 > (No such file or directory) > > testParsingProcMemFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): > > /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/MEMINFO_943711651 > (No such file or directory) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3465) org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23
[ https://issues.apache.org/jira/browse/MAPREDUCE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3465: --- Attachment: MR-3465.1.patch Patch for 0.23 branch only. Seems like http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-project/pom.xml?r1=1164779&r2=1173739 was not merged to 0.23. > org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23 > > > Key: MAPREDUCE-3465 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3465 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3465.1.patch > > > Running org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin > Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.121 sec <<< > FAILURE! > Tests in error: > > testParsingProcStatAndCpuFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): > > /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/CPUINFO_943711651 > (No such file or directory) > > testParsingProcMemFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): > > /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/MEMINFO_943711651 > (No such file or directory) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3454) [Gridmix] TestDistCacheEmulation is broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3454: --- Status: Patch Available (was: Open) > [Gridmix] TestDistCacheEmulation is broken > -- > > Key: MAPREDUCE-3454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3454 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.24.0 >Reporter: Amar Kamat >Assignee: Hitesh Shah > Labels: gridmix > Fix For: 0.23.1 > > Attachments: MR-3454.1.patch > > > TestDistCacheEmulation is broken as 'MapReduceTestUtil' no longer exists. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3454) [Gridmix] TestDistCacheEmulation is broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3454: --- Attachment: MR-3454.1.patch > [Gridmix] TestDistCacheEmulation is broken > -- > > Key: MAPREDUCE-3454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3454 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.24.0 >Reporter: Amar Kamat >Assignee: Hitesh Shah > Labels: gridmix > Fix For: 0.23.1 > > Attachments: MR-3454.1.patch > > > TestDistCacheEmulation is broken as 'MapReduceTestUtil' no longer exists. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3444) trunk/0.23 builds broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3444: --- Attachment: MR-3444.1.patch fix ivy and build.xml for recent changes. > trunk/0.23 builds broken > - > > Key: MAPREDUCE-3444 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3444 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3444.1.patch > > > https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/208/ > https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1310/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3444) trunk/0.23 builds broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3444: --- Status: Patch Available (was: Open) > trunk/0.23 builds broken > - > > Key: MAPREDUCE-3444 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3444 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3444.1.patch > > > https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/208/ > https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1310/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3434) Nightly build broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3434: --- Status: Patch Available (was: Open) > Nightly build broken > - > > Key: MAPREDUCE-3434 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3434 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3434.1.patch > > > https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/901/ > Results : > Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs) > Tests in error: > org.apache.hadoop.mapreduce.v2.TestMROldApiJobs: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMROldApiJobs > org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMRJobs > Likely due to either of: > MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR resolution > (tucu) > MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides > client APIs cross MR1 and MR2. (Ahmed via tucu) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3434) Nightly build broken
[ https://issues.apache.org/jira/browse/MAPREDUCE-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3434: --- Attachment: MR-3434.1.patch JobConf#findContainingJar requires jobclient-tests.jar to be in the classpath to work correctly. > Nightly build broken > - > > Key: MAPREDUCE-3434 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3434 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MR-3434.1.patch > > > https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/901/ > Results : > Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs) > Tests in error: > org.apache.hadoop.mapreduce.v2.TestMROldApiJobs: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMROldApiJobs > org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMRJobs > Likely due to either of: > MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR resolution > (tucu) > MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides > client APIs cross MR1 and MR2. (Ahmed via tucu) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3366) Mapreduce component should use consistent directory structure layout as HDFS/common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3366:
---
Status: Open (was: Patch Available)
> Mapreduce component should use consistent directory structure layout as
> HDFS/common
> ---
>
> Key: MAPREDUCE-3366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
> Environment: Java, Maven
>Reporter: Eric Yang
>Assignee: Eric Yang
> Attachments: MAPREDUCE-3366.patch
>
>
> Directory structure for MRv2 layout looks like:
> {noformat}
> hadoop-mapreduce-0.23.0-SNAPSHOT/bin
> /conf
> /lib
> /modules
> {noformat}
> The directory structure layout should be updated to reflect changes
> implemented in HADOOP-6255.
> {noformat}
> hadoop-mapreduce-0.23.0-SNAPSHOT/bin
> /etc/hadoop
> /lib
> /libexec
> /sbin
> /share/hadoop
> /share/hadoop/lib
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3407) Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3407: --- Status: Patch Available (was: Open) > Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster > > > Key: MAPREDUCE-3407 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3407 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3407.1.patch > > > pom for mapreduce-client-jobclient sets system property to incorrect jar > name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3407) Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3407: --- Attachment: MR-3407.1.patch Copied patch from MAPREDUCE-3393. > Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster > > > Key: MAPREDUCE-3407 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3407 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3407.1.patch > > > pom for mapreduce-client-jobclient sets system property to incorrect jar > name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3213) AM should kill all running tasks on a node when it gets marked as blacklisted
[ https://issues.apache.org/jira/browse/MAPREDUCE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3213: --- Priority: Major (was: Critical) > AM should kill all running tasks on a node when it gets marked as blacklisted > - > > Key: MAPREDUCE-3213 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3213 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3102) NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3102: --- Attachment: MR-3102.2.patch Addressed comments. > NodeManager should fail fast with wrong configuration or permissions for > LinuxContainerExecutor > --- > > Key: MAPREDUCE-3102 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3102 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3102.1.patch, MR-3102.2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3102) NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3102: --- Status: Patch Available (was: Open) > NodeManager should fail fast with wrong configuration or permissions for > LinuxContainerExecutor > --- > > Key: MAPREDUCE-3102 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3102 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3102.1.patch, MR-3102.2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3393) TestMRJobs, TestMROldApiJobs, and TestUberAM failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3393: --- Attachment: MR-3393.2.patch adding java_home into env settings. > TestMRJobs, TestMROldApiJobs, and TestUberAM failures > - > > Key: MAPREDUCE-3393 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3393 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Hitesh Shah > Attachments: MR-3393.1.patch, MR-3393.2.patch > > > Check out branch 0.23 and run mvn test from hadoop-mapreduce-project directory > --- > T E S T S > --- > Running org.apache.hadoop.mapred.TestClientServiceDelegate > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.717 sec > Running org.apache.hadoop.mapred.TestClientRedirect > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.436 sec > Running org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 sec > Running org.apache.hadoop.mapreduce.v2.TestMRJobs > Tests run: 4, Failures: 3, Errors: 1, Skipped: 0, Time elapsed: 67.999 sec > <<< FAILURE! > Running org.apache.hadoop.mapreduce.v2.TestYARNRunner > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.976 sec > Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 31.879 sec > <<< FAILURE! > Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService > ^NRunning org.apache.hadoop.mapreduce.v2.TestUberAM > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 101.096 sec > <<< FAILURE! > Results : > Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testJobSucceed(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs): Job > expected to succeed failed > testJobFail(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs) > Tests in error: > testFailingMapper(org.apache.hadoop.mapreduce.v2.TestMRJobs): 0 > org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMRJobs > Tests run: 19, Failures: 5, Errors: 2, Skipped: 0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3393) TestMRJobs, TestMROldApiJobs, and TestUberAM failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3393: --- Attachment: MR-3393.1.patch Part 1 of the final patch: - the system property setting for path to app jar is being set incorrect to jobclient.jar instead of mr-client-app.jar. Builds currently are effectively broken as the tests are silently failing as the appropriate jar cannot be found. > TestMRJobs, TestMROldApiJobs, and TestUberAM failures > - > > Key: MAPREDUCE-3393 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3393 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves > Attachments: MR-3393.1.patch > > > Check out branch 0.23 and run mvn test from hadoop-mapreduce-project directory > --- > T E S T S > --- > Running org.apache.hadoop.mapred.TestClientServiceDelegate > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.717 sec > Running org.apache.hadoop.mapred.TestClientRedirect > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.436 sec > Running org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 sec > Running org.apache.hadoop.mapreduce.v2.TestMRJobs > Tests run: 4, Failures: 3, Errors: 1, Skipped: 0, Time elapsed: 67.999 sec > <<< FAILURE! > Running org.apache.hadoop.mapreduce.v2.TestYARNRunner > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.976 sec > Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 31.879 sec > <<< FAILURE! > Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService > ^NRunning org.apache.hadoop.mapreduce.v2.TestUberAM > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 101.096 sec > <<< FAILURE! > Results : > Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs) > testJobSucceed(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs): Job > expected to succeed failed > testJobFail(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs) > Tests in error: > testFailingMapper(org.apache.hadoop.mapreduce.v2.TestMRJobs): 0 > org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start > org.apache.hadoop.mapreduce.v2.TestMRJobs > Tests run: 19, Failures: 5, Errors: 2, Skipped: 0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3120) JobHistory is not providing correct count failed,killed task
[ https://issues.apache.org/jira/browse/MAPREDUCE-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3120: --- Priority: Critical (was: Major) > JobHistory is not providing correct count failed,killed task > > > Key: MAPREDUCE-3120 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3120 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Subroto Sanyal >Assignee: Subroto Sanyal >Priority: Critical > Fix For: 0.23.1 > > Attachments: JobFail.PNG > > > Please refer the attachment JobFail.PNG. > Here the Job (WordCount) Failed as all Map Attempts were > killed(intensionally) but, still the Table in UI shows 0 Killed Attempts and > no reason for Failure is also available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3314) AM should have a way of telling RM to not allocate containers on some nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3314: --- Priority: Critical (was: Major) > AM should have a way of telling RM to not allocate containers on some nodes > --- > > Key: MAPREDUCE-3314 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3314 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.1 > > > The AM blacklists certain nodes due to task failures but the RM does not know > of this and can allocate blacklisted nodes to the AM again and again. > RM should support a protocol that allows AM to inform the RM of nodes that it > should not allocate to it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3213) AM should kill all running tasks on a node when it gets marked as blacklisted
[ https://issues.apache.org/jira/browse/MAPREDUCE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3213: --- Priority: Critical (was: Major) > AM should kill all running tasks on a node when it gets marked as blacklisted > - > > Key: MAPREDUCE-3213 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3213 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Priority: Critical > Fix For: 0.23.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3350) Per-app RM page should have the list of application-attempts like on the app JHS page
[ https://issues.apache.org/jira/browse/MAPREDUCE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3350: --- Priority: Critical (was: Major) > Per-app RM page should have the list of application-attempts like on the app > JHS page > - > > Key: MAPREDUCE-3350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, webapps >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Priority: Critical > Fix For: 0.23.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3272) Lost NMs fail to rejoin
[
https://issues.apache.org/jira/browse/MAPREDUCE-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated MAPREDUCE-3272:
---
Priority: Critical (was: Major)
> Lost NMs fail to rejoin
> ---
>
> Key: MAPREDUCE-3272
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3272
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Priority: Critical
> Fix For: 0.23.1
>
>
> Lost nodemanagers fail to join back.
> When the NM is lost, RM log reads
> {noformat}
> INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor:
> Expired: Timed out after 600 secs
> INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
> Processing of type EXPIRE
> INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Removed
> Node
> INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
> Node Transitioned from RUNNING to LOST
> {noformat}
> When the NM joins back, RM log reads
> {noformat}
> INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService:
> Node not found rebooting
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3271) Lost nodes list and count not updated
[ https://issues.apache.org/jira/browse/MAPREDUCE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3271: --- Priority: Critical (was: Major) > Lost nodes list and count not updated > - > > Key: MAPREDUCE-3271 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3271 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Priority: Critical > Fix For: 0.23.1 > > > When nodemanagers are lost, the "Lost Nodes" list and the count is not > incremented. Either we, > 1. Fix the lost nodes list when a nodemanager is lost - The problem with > tracking lost nodes is, if the nodemanager joins back, there would be > duplicate entries in active and lost nodes with different port numbers. > 2. Do not track lost nodemanagers -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3353) Need a RM->AM channel to inform AMs about faulty/unhealthy/lost nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3353: --- Priority: Critical (was: Major) > Need a RM->AM channel to inform AMs about faulty/unhealthy/lost nodes > - > > Key: MAPREDUCE-3353 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3353 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster, mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Priority: Critical > Fix For: 0.23.1 > > > When a node gets lost or turns faulty, AM needs to know about that event so > that it can take some action like for e.g. re-executing map tasks whose > intermediate output live on that faulty node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3368) compile-mapred-test fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3368: --- Attachment: MR-3368.1.patch 0.23 branch build breaking as the test fixes for MAPREDUCE-3217 and MAPREDUCE-3221 were using hdfs protocol related changes that have not merged into 0.23. This patch is only for branch-0.23. Effectively reverts some of the changes done for fixing the tests. Will need to revert/fix these tests again if the hdfs protocol related changes are merged to 0.23. > compile-mapred-test fails > - > > Key: MAPREDUCE-3368 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3368 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.0 > > Attachments: MR-3368.1.patch > > > compile-mapred-test target is failing once again. > Details: > https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-0.23-Build/83/consoleFull -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3345) Race condition in ResourceManager causing TestContainerManagerSecurity to fail sometimes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3345: --- Status: Patch Available (was: Open) > Race condition in ResourceManager causing TestContainerManagerSecurity to > fail sometimes > > > Key: MAPREDUCE-3345 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3345 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3345.1.patch > > > See > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1247//testReport/org.apache.hadoop.yarn.server/TestContainerManagerSecurity/testUnauthorizedUser/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3345) Race condition in ResourceManager causing TestContainerManagerSecurity to fail sometimes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3345: --- Attachment: MR-3345.1.patch No tests - trivial fix. Running failing test in a loop for over an hour now with no failures. > Race condition in ResourceManager causing TestContainerManagerSecurity to > fail sometimes > > > Key: MAPREDUCE-3345 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3345 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3345.1.patch > > > See > https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1247//testReport/org.apache.hadoop.yarn.server/TestContainerManagerSecurity/testUnauthorizedUser/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3102) NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3102: --- Component/s: mrv2 Affects Version/s: 0.23.0 > NodeManager should fail fast with wrong configuration or permissions for > LinuxContainerExecutor > --- > > Key: MAPREDUCE-3102 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3102 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3102.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3102) NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3102: --- Status: Patch Available (was: Open) Verified manually on a linux cluster that NM does not come up with a bad conf file ( tried missing group setting ) > NodeManager should fail fast with wrong configuration or permissions for > LinuxContainerExecutor > --- > > Key: MAPREDUCE-3102 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3102 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: security >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3102.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3102) NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3102: --- Attachment: MR-3102.1.patch > NodeManager should fail fast with wrong configuration or permissions for > LinuxContainerExecutor > --- > > Key: MAPREDUCE-3102 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3102 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: security >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.1 > > Attachments: MR-3102.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3219) ant test TestDelegationToken failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3219: --- Attachment: MR-3219.1.patch > ant test TestDelegationToken failing on trunk > - > > Key: MAPREDUCE-3219 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3219 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3219.1.patch > > > Testcase: testDelegationToken took 2.043 sec > Caused an ERROR > Client Hitesh tries to renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > org.apache.hadoop.security.AccessControlException: Client Hitesh tries to > renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57) > at > org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:399) > at > org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:475) > at org.apache.hadoop.security.token.Token.renew(Token.java:310) > at > org.apache.hadoop.mapred.JobClient.renewDelegationToken(JobClient.java:1088) > at > org.apache.hadoop.mapreduce.security.token.delegation.TestDelegationToken.testDelegationToken(TestDelegationToken.java:89) > Caused by: org.apache.hadoop.security.AccessControlException: Client Hitesh > tries to renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > at org.apache.hadoop.ipc.Client.call(Client.java:1085) > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244) > at $Proxy11.renewDelegationToken(Unknown Source) > at > org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:397) -- This message is a
[jira] [Updated] (MAPREDUCE-3219) ant test TestDelegationToken failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3219: --- Status: Patch Available (was: Open) > ant test TestDelegationToken failing on trunk > - > > Key: MAPREDUCE-3219 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3219 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3219.1.patch > > > Testcase: testDelegationToken took 2.043 sec > Caused an ERROR > Client Hitesh tries to renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > org.apache.hadoop.security.AccessControlException: Client Hitesh tries to > renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57) > at > org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:399) > at > org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:475) > at org.apache.hadoop.security.token.Token.renew(Token.java:310) > at > org.apache.hadoop.mapred.JobClient.renewDelegationToken(JobClient.java:1088) > at > org.apache.hadoop.mapreduce.security.token.delegation.TestDelegationToken.testDelegationToken(TestDelegationToken.java:89) > Caused by: org.apache.hadoop.security.AccessControlException: Client Hitesh > tries to renew a token with renewer specified as alice > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) > at > org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) > at org.apache.hadoop.ipc.Client.call(Client.java:1085) > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244) > at $Proxy11.renewDelegationToken(Unknown Source) > at > org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:397) -- This mes
[jira] [Updated] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3215: --- Status: Patch Available (was: Open) > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk > -- > > Key: MAPREDUCE-3215 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3215 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.24.0 > > Attachments: MR-3215.1.patch > > > Testcase: testNoJobSetupCleanup took 13.271 sec > FAILED > Number of part-files is 0 and not 1 > junit.framework.AssertionFailedError: Number of part-files is 0 and not 1 > at > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.submitAndValidateJob(TestNoJobSetupCleanup.java:60) > at > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.testNoJobSetupCleanup(TestNoJobSetupCleanup.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3215: --- Attachment: MR-3215.1.patch > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk > -- > > Key: MAPREDUCE-3215 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3215 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.24.0 > > Attachments: MR-3215.1.patch > > > Testcase: testNoJobSetupCleanup took 13.271 sec > FAILED > Number of part-files is 0 and not 1 > junit.framework.AssertionFailedError: Number of part-files is 0 and not 1 > at > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.submitAndValidateJob(TestNoJobSetupCleanup.java:60) > at > org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.testNoJobSetupCleanup(TestNoJobSetupCleanup.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3332) contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling
[ https://issues.apache.org/jira/browse/MAPREDUCE-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3332: --- Attachment: MR-3332.1.patch Trivial fix. > contrib/raid compile breaks due to changes in > hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling > > > Key: MAPREDUCE-3332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Trivial > Fix For: 0.23.1 > > Attachments: MR-3332.1.patch > > > [javac] > /Users/Hitesh/dev/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:783: > > writeBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Token,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],org.apache.hadoop.hdfs.protocol.DatanodeInfo,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long,org.apache.hadoop.util.DataChecksum) > in org.apache.hadoop.hdfs.protocol.datatransfer.Sender cannot be applied to > (org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Token,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long) > [javac] new Sender(out).writeBlock(block.getBlock(), > block.getBlockToken(), "", > [javac]^ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3332) contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling
[ https://issues.apache.org/jira/browse/MAPREDUCE-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3332: --- Status: Patch Available (was: Open) > contrib/raid compile breaks due to changes in > hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling > > > Key: MAPREDUCE-3332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Trivial > Fix For: 0.23.1 > > Attachments: MR-3332.1.patch > > > [javac] > /Users/Hitesh/dev/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:783: > > writeBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Token,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],org.apache.hadoop.hdfs.protocol.DatanodeInfo,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long,org.apache.hadoop.util.DataChecksum) > in org.apache.hadoop.hdfs.protocol.datatransfer.Sender cannot be applied to > (org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Token,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long) > [javac] new Sender(out).writeBlock(block.getBlock(), > block.getBlockToken(), "", > [javac]^ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Open (was: Patch Available) Disabling tests addressed in MR-3321. > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch, MR-3214.2.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3321) Disable some failing legacy tests for MRv2 builds to go through
[ https://issues.apache.org/jira/browse/MAPREDUCE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3321: --- Status: Patch Available (was: Open) > Disable some failing legacy tests for MRv2 builds to go through > --- > > Key: MAPREDUCE-3321 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3321 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.0 > > Attachments: MR-3321.1.patch > > > By-product of MR-3214. Disable tests for the short term until fixes are > available for all tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3321) Disable some failing legacy tests for MRv2 builds to go through
[ https://issues.apache.org/jira/browse/MAPREDUCE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3321: --- Attachment: MR-3321.1.patch Copy of MR-3214.2.patch > Disable some failing legacy tests for MRv2 builds to go through > --- > > Key: MAPREDUCE-3321 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3321 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.0 > > Attachments: MR-3321.1.patch > > > By-product of MR-3214. Disable tests for the short term until fixes are > available for all tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Patch Available (was: Open) > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch, MR-3214.2.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Attachment: MR-3214.2.patch Updating patch with fix (provided by Sid) for ant tests timing out due to changes from MR-3035. > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch, MR-3214.2.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Open (was: Patch Available) > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Patch Available (was: Open) > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Open (was: Patch Available) > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Status: Patch Available (was: Open) > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3214) ant mapreduce tests failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3214: --- Attachment: MR-3214.1.patch > ant mapreduce tests failing > --- > > Key: MAPREDUCE-3214 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.24.0 > > Attachments: MR-3214.1.patch > > > Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3318) ant test TestSeveral timing out in commit builds
[ https://issues.apache.org/jira/browse/MAPREDUCE-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3318: --- Summary: ant test TestSeveral timing out in commit builds (was: TestSeveral timing out in commit builds) > ant test TestSeveral timing out in commit builds > > > Key: MAPREDUCE-3318 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3318 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah > Fix For: 0.23.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3262: --- Priority: Critical (was: Major) > Handle missing events in failure scenarios > -- > > Key: MAPREDUCE-3262 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Critical > Fix For: 0.23.0 > > Attachments: MR-3262.1.patch > > > Need to handle kill container event in localization failed state. > Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3313: --- Attachment: MR-3313.2.patch > TestResourceTrackerService failing in trunk some times > -- > > Key: MAPREDUCE-3313 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ravi Gummadi >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3313.1.patch, MR-3313.2.patch > > > TestResourceTrackerService is failing in trunk sometimes with the following > error: > testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) > Time elapsed: 0.876 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) > at > org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3313: --- Fix Version/s: 0.23.0 Affects Version/s: 0.23.0 Status: Patch Available (was: Open) > TestResourceTrackerService failing in trunk some times > -- > > Key: MAPREDUCE-3313 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ravi Gummadi >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3313.1.patch, MR-3313.2.patch > > > TestResourceTrackerService is failing in trunk sometimes with the following > error: > testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) > Time elapsed: 0.876 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) > at > org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3313: --- Component/s: mrv2 > TestResourceTrackerService failing in trunk some times > -- > > Key: MAPREDUCE-3313 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ravi Gummadi >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3313.1.patch, MR-3313.2.patch > > > TestResourceTrackerService is failing in trunk sometimes with the following > error: > testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) > Time elapsed: 0.876 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) > at > org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3313: --- Attachment: MR-3313.1.patch > TestResourceTrackerService failing in trunk some times > -- > > Key: MAPREDUCE-3313 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Ravi Gummadi >Assignee: Hitesh Shah > Attachments: MR-3313.1.patch > > > TestResourceTrackerService is failing in trunk sometimes with the following > error: > testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) > Time elapsed: 0.876 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) > at > org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3262: --- Status: Patch Available (was: Open) > Handle missing events in failure scenarios > -- > > Key: MAPREDUCE-3262 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3262.1.patch > > > Need to handle kill container event in localization failed state. > Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
