[jira] [Comment Edited] (YARN-6927) Add support for individual resource types requests in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205011#comment-16205011
]
Sunil G edited comment on YARN-6927 at 10/15/17 4:01 AM:
-
Yes. This sounds a better syntax {{(memory=x unit, vcore=y unit, gpu=z unit,
...)}}. With more resource-types getting added in YARN, defining arbitrary
resource names here will be inlining with the same resource name in
resource-types.xml. With YARN-7307, I assume that server and client will have
same synonymous meaning for resource types.
I am not very sure whether its a problem or not.
{{mapreduce.map/reduce.resource.}} here resource vectors are
{noformat}
memory
vcores
disk-space
{noformat}
looks like a very clean config model.
Could we consider other resource specifications like
{{yarn.app.mapreduce.am.resource.mb/cpu-vcores}} as a standard way to define
CPU and Memory if users want to use, rather than an overwriting model? In case
when both are specified (old and new config for cpu/memory), i am more in favor
of throwing an exception to end user asking to specify in one format.
was (Author: sunilg):
Yes. This sounds a better syntax {{(memory=x unit, vcore=y unit, gpu=z unit,
...)}}. With more resource-types getting added in YARN, defining arbitrary
resource names here will be inlining with the same resource name in
resource-types.xml. With YARN-7307, I assume that server and client will have
same synonymous meaning for resource types.
I am not very sure whether its a problem or not.
{{mapreduce.map/reduce.resource.}} here resource vectors are
{noformat}
memory
vcores
disk-space
{noformat}
looks like a very clean config model.
Could we consider other resource specifications like
{{yarn.app.mapreduce.am.resource.mb/cpu-vcores}} as a standard way to define
CPU and Memory is users wants to use, rather than an overwriting model? In case
when both are specified, i am more in favor of throwing an exception to end
user asking to specify in one format.
> Add support for individual resource types requests in MapReduce
> ---
>
> Key: YARN-6927
> URL: https://issues.apache.org/jira/browse/YARN-6927
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Attachments: YARN-6927.000.patch, YARN-6927.001.patch
>
>
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource
> profiles don't give users much flexibility in their resource requests. To
> satisfy users' needs, MapReduce should also allow users to specify arbitrary
> resource requests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4373) Jobs can be temporarily forgotten during recovery
[ https://issues.apache.org/jira/browse/YARN-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved YARN-4373. Resolution: Invalid This issue was actually that if the name node is unreachable, there's a window after a job finishes where the RM will redirect to the JHS, but because the name node isn't reachable, the job's history hasn't been transferred yet. That's not something that we can easily resolve, so I'm closing this as invalid. > Jobs can be temporarily forgotten during recovery > - > > Key: YARN-4373 > URL: https://issues.apache.org/jira/browse/YARN-4373 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.7.1 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Critical > > The RM becomes available to service requests before state store recovery is > started. Before recovery and during the recovery period, it's possible for a > client to request an application report for a running application to which > the RM will respond that the application in unknown. > I'm seeing this issue with Oozie during an RM failover. Until the active > finishes recovery, it reports erroneous information to Oozie, which doesn't > have context to know that it should just try again later. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205011#comment-16205011
]
Sunil G commented on YARN-6927:
---
Yes. This sounds a better syntax {{(memory=x unit, vcore=y unit, gpu=z unit,
...)}}. With more resource-types getting added in YARN, defining arbitrary
resource names here will be inlining with the same resource name in
resource-types.xml. With YARN-7307, I assume that server and client will have
same synonymous meaning for resource types.
I am not very sure whether its a problem or not.
{{mapreduce.map/reduce.resource.}} here resource vectors are
{noformat}
memory
vcores
disk-space
{noformat}
looks like a very clean config model.
Could we consider other resource specifications like
{{yarn.app.mapreduce.am.resource.mb/cpu-vcores}} as a standard way to define
CPU and Memory is users wants to use, rather than an overwriting model? In case
when both are specified, i am more in favor of throwing an exception to end
user asking to specify in one format.
> Add support for individual resource types requests in MapReduce
> ---
>
> Key: YARN-6927
> URL: https://issues.apache.org/jira/browse/YARN-6927
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Attachments: YARN-6927.000.patch, YARN-6927.001.patch
>
>
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource
> profiles don't give users much flexibility in their resource requests. To
> satisfy users' needs, MapReduce should also allow users to specify arbitrary
> resource requests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205010#comment-16205010 ] Daniel Templeton commented on YARN-6927: Agree about the space. I see the point that a user you specifies both the old and new options might be confused by the result. On the other hand, it's confusing to have some of the options use the new format, and some the old. I think I'd rather not throw an exception on the new format. If you're really worried about confusing the user, we could throw an exception if the user specifies both the new and old memory values (or same for CPU). > Add support for individual resource types requests in MapReduce > --- > > Key: YARN-6927 > URL: https://issues.apache.org/jira/browse/YARN-6927 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Daniel Templeton >Assignee: Gergo Repas > Attachments: YARN-6927.000.patch, YARN-6927.001.patch > > > YARN-6504 adds support for resource profiles in MapReduce jobs, but resource > profiles don't give users much flexibility in their resource requests. To > satisfy users' needs, MapReduce should also allow users to specify arbitrary > resource requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7254) UI and metrics changes related to absolute resource configuration
[ https://issues.apache.org/jira/browse/YARN-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205006#comment-16205006 ] Wangda Tan commented on YARN-7254: -- +1, thanks [~sunilg]! > UI and metrics changes related to absolute resource configuration > - > > Key: YARN-7254 > URL: https://issues.apache.org/jira/browse/YARN-7254 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-7254.001.patch, YARN-7254.002.patch, > YARN-7254.YARN-5881.002.patch, YARN-7254.YARN-5881.003.patch, > YARN-7254.YARN-5881.004.patch, YARN-7254.YARN-5881.005.patch, > YARN-7254.YARN-5881.006.patch, YARN-7254.YARN-5881.007.patch, > YARN-7254.YARN-5881.008.patch, YARN-7254.YARN-5881.009.patch > > > Impact on UI and metrics related to absolute resource configuration on CS -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205005#comment-16205005
]
Wangda Tan commented on YARN-6927:
--
[~templedf], since MR only supports specifying individual resource types, I
think we can add the support of specifying all types in single field in the
future.
Before this patch, user can specify
{code}
yarn.app.mapreduce.am.resource.mb for AM's memory
yarn.app.mapreduce.am.cpu-vcores for AM's vcores
mapreduce.map/reduce.memory.mb for mapper/reducer's memory
mapreduce.map/reduce.cpu.vcores for mapper/reducer's vcores.
{code}
After this patch, user can specify:
{code}
For AM's memory:
yarn.app.mapreduce.am.resource.mb OR
yarn.app.mapreduce.am.resource.memory-mb = 1024 MB
For AM's vcores:
yarn.app.mapreduce.am.cpu-vcores OR
yarn.app.mapreduce.am.resource.vcores = 5
For Other AM resources:
yarn.app.mapreduce.am.resource.disk-space = 12 GB
For Mapper/Reducer's Memory:
mapreduce.map/reduce.memory.mb OR
mapreduce.map/reduce.resource.memory-mb = 1024 MB
For Mapper/Reducer's vcore:
mapreduce.map/reduce.cpu.vcores OR
mapreduce.map/reduce.resource.vcores = 3
For other Mapper/Reducer's resource:
mapreduce.map/reduce.resource.disk-space = 30 GB
{code}
First,
I think we should throw exception when following options are specified:
{code}
yarn.app.mapreduce.am.resource.memory-mb
yarn.app.mapreduce.am.resource.vcores
mapreduce.map/reduce.resource.memory-mb
mapreduce.map/reduce.resource.vcores
{code}
Because overwriting logic (from the implementation, new options like
mapreduce.map/reduce.resource.vcores overwrite the old one. This is very
confusing to end users.
Second,
It might be better to support specifying space between number and unit. E.g.
1024MB, 1024 MB are all valid and mean the same thing. currently it doesn't
allow adding a space.
Thoughts?
> Add support for individual resource types requests in MapReduce
> ---
>
> Key: YARN-6927
> URL: https://issues.apache.org/jira/browse/YARN-6927
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Attachments: YARN-6927.000.patch, YARN-6927.001.patch
>
>
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource
> profiles don't give users much flexibility in their resource requests. To
> satisfy users' needs, MapReduce should also allow users to specify arbitrary
> resource requests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204986#comment-16204986
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
55s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 103 new or modified
test files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
52s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
36s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m
58s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site hadoop-yarn-project . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 25m
10s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
32s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 50m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 22m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 22m 12s{color}
| {color:red} root generated 1 new + 1445 unchanged - 0 fixed = 1446 total (was
1445) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
6m 0s{color} | {color:orange} root: The patch generated 136 new + 3549
unchanged - 298 fixed = 3685 total (was 3847) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
3s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
17s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
3s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
15s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server . hadoop-yarn-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 32m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m
14s{color} | {color:red} root generated 1 new + 53 unchanged - 0 fixed = 54
total (was 53) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 25m 44s{color}
|
[jira] [Assigned] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime
[ https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang reassigned YARN-7066: --- Resolution: Duplicate Assignee: Eric Yang > Add ability to specify volumes to mount for DockerContainerRuntime > -- > > Key: YARN-7066 > URL: https://issues.apache.org/jira/browse/YARN-7066 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Affects Versions: 3.0.0-beta1 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: YARN-7066.001.patch, YARN-7066.002.patch > > > Yarnfile describes environment, docker image, and configuration template for > launching docker containers in YARN. It would be nice to have ability to > specify the volumes to mount. This can be used in combination to > AMBARI-21748 to mount HDFS as data directories to docker containers. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime
[ https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204985#comment-16204985 ] Eric Yang commented on YARN-7066: - [~shaneku...@gmail.com] Sorry, I did not know YARN-5534 already includes user defined mount. I probably should have read the patch before this was opened. We can close this as a dupe. Thanks > Add ability to specify volumes to mount for DockerContainerRuntime > -- > > Key: YARN-7066 > URL: https://issues.apache.org/jira/browse/YARN-7066 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Affects Versions: 3.0.0-beta1 >Reporter: Eric Yang > Attachments: YARN-7066.001.patch, YARN-7066.002.patch > > > Yarnfile describes environment, docker image, and configuration template for > launching docker containers in YARN. It would be nice to have ability to > specify the volumes to mount. This can be used in combination to > AMBARI-21748 to mount HDFS as data directories to docker containers. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204882#comment-16204882
]
Hadoop QA commented on YARN-7190:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
55s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-5355_branch2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
36s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
57s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
17s{color} | {color:green} YARN-5355_branch2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
2s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
8s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 40m 12s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 33s{color}
| {color:red} hadoop-yarn-server-timelineservice in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 37s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 40m 13s{color}
| {color:red} hadoop-yarn-project in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
25s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}228m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler |
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| | hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler |
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| Timed out junit tests |
[jira] [Commented] (YARN-7013) merge related work for YARN-3926 branch
[
https://issues.apache.org/jira/browse/YARN-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204866#comment-16204866
]
Hadoop QA commented on YARN-7013:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 46 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
10s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
14s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
38s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
49s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
21m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
23s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
11s{color} | {color:green} branch-3.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 11m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 11m 27s{color}
| {color:red} root generated 1 new + 1272 unchanged - 0 fixed = 1273 total (was
1272) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 41s{color} | {color:orange} root: The patch generated 57 new + 2325
unchanged - 67 fixed = 2382 total (was 2392) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
16s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} hadoop-yarn-project_hadoop-yarn generated 0 new +
5400 unchanged - 6 fixed = 5400 total (was 5406) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common
generated 0 new + 4458 unchanged - 5 fixed = 4458 total (was 4463) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color}
[jira] [Commented] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204865#comment-16204865
]
Hadoop QA commented on YARN-7190:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-5355_branch2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
11s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
58s{color} | {color:green} YARN-5355_branch2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
36s{color} | {color:green} YARN-5355_branch2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
7s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
9s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 43s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 38s{color}
| {color:red} hadoop-yarn-server-timelineservice in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 48m 48s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 38m 54s{color}
| {color:red} hadoop-yarn-project in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
19s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler |
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| | hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler |
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| Timed out junit tests |
org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServices
|
|
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204864#comment-16204864
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
54s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 103 new or modified
test files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
22s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
39s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
17s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site hadoop-yarn-project . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 13m
9s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
3s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 8m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 8m 14s{color}
| {color:red} root generated 1 new + 1445 unchanged - 0 fixed = 1446 total (was
1445) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 48s{color} | {color:orange} root: The patch generated 135 new + 3553
unchanged - 297 fixed = 3688 total (was 3850) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
8s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
3s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server . hadoop-yarn-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 18m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m
20s{color} | {color:red} root generated 1 new + 53 unchanged - 0 fixed = 54
total (was 53) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 53s{color}
|
[jira] [Commented] (YARN-5516) Add REST API for periodicity
[
https://issues.apache.org/jira/browse/YARN-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204857#comment-16204857
]
Hadoop QA commented on YARN-5516:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 7s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 36s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
39s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 48m 31s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0de40f0 |
| JIRA Issue | YARN-5516 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892234/YARN-5516.v001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 610a61bab112 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204833#comment-16204833 ] Daniel Templeton commented on YARN-6927: Interesting idea, [~leftnoteasy]. Looking at other resource managers (Grid Engine, LSF, PBSPro, et al), it looks like most support both ways, e.g. "qsub -l mem=4G -l cpu=4" or "qsub -l mem=4G,cpu=4", which seems sane to me. Can we get this one in and then follow up with another JIRA to add a resource string parser as you suggested? > Add support for individual resource types requests in MapReduce > --- > > Key: YARN-6927 > URL: https://issues.apache.org/jira/browse/YARN-6927 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Daniel Templeton >Assignee: Gergo Repas > Attachments: YARN-6927.000.patch, YARN-6927.001.patch > > > YARN-6504 adds support for resource profiles in MapReduce jobs, but resource > profiles don't give users much flexibility in their resource requests. To > satisfy users' needs, MapReduce should also allow users to specify arbitrary > resource requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5516) Add REST API for periodicity
[ https://issues.apache.org/jira/browse/YARN-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-5516: -- Attachment: YARN-5516.v001.patch Added first version of the patch. > Add REST API for periodicity > > > Key: YARN-5516 > URL: https://issues.apache.org/jira/browse/YARN-5516 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sangeetha Abdu Jyothi >Assignee: Sean Po > Attachments: YARN-5516.v001.patch > > > YARN-5516 changing REST API of the reservation system to support periodicity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated YARN-7190:
---
Attachment: YARN-7190-YARN-5355_branch2.02.patch
> Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user
> classpath
>
>
> Key: YARN-7190
> URL: https://issues.apache.org/jira/browse/YARN-7190
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineclient, timelinereader, timelineserver
>Reporter: Vrushali C
>Assignee: Varun Saxena
> Attachments: YARN-7190-YARN-5355_branch2.01.patch,
> YARN-7190-YARN-5355_branch2.02.patch, YARN-7190-YARN-5355_branch2.02.patch
>
>
> [~jlowe] had a good observation about the user classpath getting extra jars
> in hadoop 2.x brought in with TSv2. If users start picking up Hadoop 2,x's
> version of HBase jars instead of the ones they shipped with their job, it
> could be a problem.
> So when TSv2 is to be used in 2,x, the hbase related jars should come into
> only the NM classpath not the user classpath.
> Here is a list of some jars
> {code}
> commons-csv-1.0.jar
> commons-el-1.0.jar
> commons-httpclient-3.1.jar
> disruptor-3.3.0.jar
> findbugs-annotations-1.3.9-1.jar
> hbase-annotations-1.2.6.jar
> hbase-client-1.2.6.jar
> hbase-common-1.2.6.jar
> hbase-hadoop2-compat-1.2.6.jar
> hbase-hadoop-compat-1.2.6.jar
> hbase-prefix-tree-1.2.6.jar
> hbase-procedure-1.2.6.jar
> hbase-protocol-1.2.6.jar
> hbase-server-1.2.6.jar
> htrace-core-3.1.0-incubating.jar
> jamon-runtime-2.4.1.jar
> jasper-compiler-5.5.23.jar
> jasper-runtime-5.5.23.jar
> jcodings-1.0.8.jar
> joni-2.1.2.jar
> jsp-2.1-6.1.14.jar
> jsp-api-2.1-6.1.14.jar
> jsr311-api-1.1.1.jar
> metrics-core-2.2.0.jar
> servlet-api-2.5-6.1.14.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204823#comment-16204823 ] Varun Saxena commented on YARN-7295: Patch was not applying due to line endings in yarn.cmd file. Updated a new patch. > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: (was: YARN-7295-branch-2.01.patch) > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: YARN-7295-branch-2.01.patch > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204822#comment-16204822
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 12s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892231/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17937/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204821#comment-16204821
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892231/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17936/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204818#comment-16204818
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892231/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17934/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204820#comment-16204820
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892231/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17935/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: YARN-7295-branch-2.01.patch > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: (was: YARN-7295-branch-2.01.patch) > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204814#comment-16204814
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892230/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17933/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: (was: YARN-7295-branch-2.01.patch) > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: YARN-7295-branch-2.01.patch > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7295) Backport ATSv2 to branch-2
[
https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204796#comment-16204796
]
Hadoop QA commented on YARN-7295:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} YARN-7295 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7295 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892215/YARN-7295-branch-2.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17932/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Backport ATSv2 to branch-2
> --
>
> Key: YARN-7295
> URL: https://issues.apache.org/jira/browse/YARN-7295
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Varun Saxena
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-7295-branch-2.01.patch
>
>
> This JIRA is to run QA on patch with ATSv2 changes on branch-2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7127) Merge yarn-native-service branch into trunk
[ https://issues.apache.org/jira/browse/YARN-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204792#comment-16204792 ] Allen Wittenauer commented on YARN-7127: bq. It's the command to interact with ResourceManager which does listing / updating of application metadata to YARN's point of view. Although it's called application, It's NOT a command specific to the app. (i.e. AM). bq. However for 'yarn service', it's the command to interact with the service framework, i.e. the special AM we wrote. Users don't care about the internals of a command. They want cohesion. bq. there has to be a differentiator. It seems as though a lot of the issues raised here are because the RM isn't keeping track of which frameworks that YARN provides that a given application is using or used to launch. This is a "record keeping" problem. Passing that onto the user is going to be problematic when one considers that multiple users may be interacting with a given AM. If I'm the ops person that needs to take down a bad acting AM, why should I have to cycle through a bunch of different commands to figure out which one works when the RM should be able to provide a hint? To expand on that: bq. Similarly, for service framework, it's a special AM. It has its own semantics and use cases. E.g. flex the component count, upgrade the component. The component is the concept only specific to service, not to the yarn generic app. If we merge it with generic "application" command, what will the 'component' mean for other apps like MR? Not all command line arguments actually have to work with every AM type. If a user gives a nonsense request, it's ok to throw a helpful message and error out. Not everything needs to succeed. In this particular case, the command should be asking the RM if the given AM was started with the services API and act appropriately. If not, throw an error. bq. it seems more of a larger umbrella effort - expanding "yarn application" to provide a unified support for all disparate apps to roll into it. e.g., what should have been part of this project from the start. To me, this is a showstopper issue for the "native services" API. In it's current incarnation, it definitely feels bolted on rather than real functionality included as part of YARN. Just for completeness: bq. MapReduce is a customized AM on YARN, it has its own mapred command to interact with its own AM, which only makes sense to itself. like "mapred distcp". Will it make sense to merge the 'distcp' sub command to 'yarn application' command? Apples and oranges and completely ignoring the historical context involved. [I'd expand on the history here, but it is sort of an orthogonal discussion. If anyone cares, I can write up though.] I will say that given hindsight, I'm fairly confident that the mapred command wouldn't exist and almost all of the features it provides would either not exist or be merged into the yarn command somewhere. > Merge yarn-native-service branch into trunk > --- > > Key: YARN-7127 > URL: https://issues.apache.org/jira/browse/YARN-7127 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-7127.01.patch, YARN-7127.02.patch, > YARN-7127.03.patch, YARN-7127.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204764#comment-16204764
]
Hudson commented on YARN-7269:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13085 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13085/])
Addendum fix for: YARN-7269. Tracking URL in the app state does not get
(wangda: rev 87ea1dff9c431fb88e064e497b35fc17c65f1d53)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmFilterInitializer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilterInitializer.java
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch, YARN-7269.addendum.001.patch,
> YARN-7269.addendum.002.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204751#comment-16204751
]
Wangda Tan commented on YARN-6927:
--
Thanks [~grepas] for working on this.
I have an idea here regarding to multiple resource types support for
applications:
Can we add a library to get a Resource object from String which include all
resource values, this can be shared by different projects, an example format
could be:
"(memory=x unit, vcore=y unit, gpu=z unit, ...)" unit could be null.
And for applications, they can specify resources like:
{{... -Dmapreduce.mapper/am/reducer.resources=(memory=x, ..)}}
> Add support for individual resource types requests in MapReduce
> ---
>
> Key: YARN-6927
> URL: https://issues.apache.org/jira/browse/YARN-6927
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Attachments: YARN-6927.000.patch, YARN-6927.001.patch
>
>
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource
> profiles don't give users much flexibility in their resource requests. To
> satisfy users' needs, MapReduce should also allow users to specify arbitrary
> resource requests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204735#comment-16204735 ] Botong Huang commented on YARN-7102: Unit test failure is irrelevant and tracked in YARN-6747. > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102.v1.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7013) merge related work for YARN-3926 branch
[ https://issues.apache.org/jira/browse/YARN-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-7013: --- Attachment: YARN-7013.branch-3.0.001.patch Rebased after YARN-7270. > merge related work for YARN-3926 branch > --- > > Key: YARN-7013 > URL: https://issues.apache.org/jira/browse/YARN-7013 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Sunil G >Assignee: Daniel Templeton > Attachments: YARN-7013.001.patch, YARN-7013.002.patch, > YARN-7013.003.patch, YARN-7013.004.patch, YARN-7013.005.patch, > YARN-7013.006.patch, YARN-7013.008.patch, YARN-7013.branch-3.0.000.patch, > YARN-7013.branch-3.0.001.patch > > > To run jenkins for whole branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7013) merge related work for YARN-3926 branch
[
https://issues.apache.org/jira/browse/YARN-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204702#comment-16204702
]
Hadoop QA commented on YARN-7013:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-7013 does not apply to branch-3.0. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7013 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12891384/YARN-7013.branch-3.0.000.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17930/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> merge related work for YARN-3926 branch
> ---
>
> Key: YARN-7013
> URL: https://issues.apache.org/jira/browse/YARN-7013
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Sunil G
>Assignee: Daniel Templeton
> Attachments: YARN-7013.001.patch, YARN-7013.002.patch,
> YARN-7013.003.patch, YARN-7013.004.patch, YARN-7013.005.patch,
> YARN-7013.006.patch, YARN-7013.008.patch, YARN-7013.branch-3.0.000.patch
>
>
> To run jenkins for whole branch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7013) merge related work for YARN-3926 branch
[ https://issues.apache.org/jira/browse/YARN-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204701#comment-16204701 ] Daniel Templeton commented on YARN-7013: Looks like the unit test failures were caused by infra. Retriggering. > merge related work for YARN-3926 branch > --- > > Key: YARN-7013 > URL: https://issues.apache.org/jira/browse/YARN-7013 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Sunil G >Assignee: Daniel Templeton > Attachments: YARN-7013.001.patch, YARN-7013.002.patch, > YARN-7013.003.patch, YARN-7013.004.patch, YARN-7013.005.patch, > YARN-7013.006.patch, YARN-7013.008.patch, YARN-7013.branch-3.0.000.patch > > > To run jenkins for whole branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6927) Add support for individual resource types requests in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6927: -- Target Version/s: 3.1.0 (was: YARN-3926) > Add support for individual resource types requests in MapReduce > --- > > Key: YARN-6927 > URL: https://issues.apache.org/jira/browse/YARN-6927 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Daniel Templeton >Assignee: Gergo Repas > Attachments: YARN-6927.000.patch, YARN-6927.001.patch > > > YARN-6504 adds support for resource profiles in MapReduce jobs, but resource > profiles don't give users much flexibility in their resource requests. To > satisfy users' needs, MapReduce should also allow users to specify arbitrary > resource requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6927) Add support for individual resource types requests in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6927: -- Affects Version/s: (was: YARN-3926) > Add support for individual resource types requests in MapReduce > --- > > Key: YARN-6927 > URL: https://issues.apache.org/jira/browse/YARN-6927 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Daniel Templeton >Assignee: Gergo Repas > Attachments: YARN-6927.000.patch, YARN-6927.001.patch > > > YARN-6504 adds support for resource profiles in MapReduce jobs, but resource > profiles don't give users much flexibility in their resource requests. To > satisfy users' needs, MapReduce should also allow users to specify arbitrary > resource requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6927) Add support for individual resource types requests in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204696#comment-16204696
]
Daniel Templeton commented on YARN-6927:
Thanks, [~grepas]. My one and only quibble is here:
{{MR_AM_PREFIX+"resource.";}}. Can we add some spaces on either side of that
plus? [~vvasudev], [~leftnoteasy], any comments?
> Add support for individual resource types requests in MapReduce
> ---
>
> Key: YARN-6927
> URL: https://issues.apache.org/jira/browse/YARN-6927
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Attachments: YARN-6927.000.patch, YARN-6927.001.patch
>
>
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource
> profiles don't give users much flexibility in their resource requests. To
> satisfy users' needs, MapReduce should also allow users to specify arbitrary
> resource requests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7246) Fix the default docker binary path
[ https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204626#comment-16204626 ] Shane Kumpf commented on YARN-7246: --- Thanks for your help [~jlowe] and [~ebadger]! > Fix the default docker binary path > -- > > Key: YARN-7246 > URL: https://issues.apache.org/jira/browse/YARN-7246 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Shane Kumpf >Assignee: Shane Kumpf >Priority: Blocker > Fix For: 2.8.2 > > Attachments: YARN-7246-branch-2.8.2.001.patch, > YARN-7246-branch-2.8.2.002.patch, YARN-7246-branch-2.8.2.003.patch, > YARN-7246-branch-2.8.2.004.patch, YARN-7246-branch-2.8.2.005.patch, > YARN-7246-branch-2.8.2.006.patch, YARN-7246-branch-2.8.2.007.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204612#comment-16204612
]
Shane Kumpf edited comment on YARN-7066 at 10/14/17 12:51 PM:
--
[~eyang] thanks for the updated patch. I'm still confused as to what this is
providing over YARN-5534. Both support the user supplying mounts via an
environment variable, which then gets added to the {{docker run}} command. The
only difference for that piece is the format of the user supplied environment
variable, but YARN-5534 goes a step further to attempt to validate the mount is
also in the white list. I reread your comment about why you feel the approach
used by this patch is superior, but I'm not following what you mean.
was (Author: shaneku...@gmail.com):
[~eyang] thanks for the updated patch. I'm still confused as to what this is
providing over YARN-5534. Both support the user supplying mounts via an
environment variable, which then gets added to the {{docker run}} command. The
only difference for that piece is the format of the user supplied environment
variable, but YARN-5534 goes a step further to attempt to validate the mount is
also in the white list. I reread your comment about why you feel this approach
is superior, but I'm not following what you mean.
> Add ability to specify volumes to mount for DockerContainerRuntime
> --
>
> Key: YARN-7066
> URL: https://issues.apache.org/jira/browse/YARN-7066
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Affects Versions: 3.0.0-beta1
>Reporter: Eric Yang
> Attachments: YARN-7066.001.patch, YARN-7066.002.patch
>
>
> Yarnfile describes environment, docker image, and configuration template for
> launching docker containers in YARN. It would be nice to have ability to
> specify the volumes to mount. This can be used in combination to
> AMBARI-21748 to mount HDFS as data directories to docker containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204612#comment-16204612
]
Shane Kumpf commented on YARN-7066:
---
[~eyang] thanks for the updated patch. I'm still confused as to what this is
providing over YARN-5534. Both support the user supplying mounts via an
environment variable, which then gets added to the {{docker run}} command. The
only difference for that piece is the format of the user supplied environment
variable, but YARN-5534 goes a step further to attempt to validate the mount is
also in the white list. I reread your comment about why you feel this approach
is similar, but I'm not following what you mean.
> Add ability to specify volumes to mount for DockerContainerRuntime
> --
>
> Key: YARN-7066
> URL: https://issues.apache.org/jira/browse/YARN-7066
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Affects Versions: 3.0.0-beta1
>Reporter: Eric Yang
> Attachments: YARN-7066.001.patch, YARN-7066.002.patch
>
>
> Yarnfile describes environment, docker image, and configuration template for
> launching docker containers in YARN. It would be nice to have ability to
> specify the volumes to mount. This can be used in combination to
> AMBARI-21748 to mount HDFS as data directories to docker containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204612#comment-16204612
]
Shane Kumpf edited comment on YARN-7066 at 10/14/17 12:50 PM:
--
[~eyang] thanks for the updated patch. I'm still confused as to what this is
providing over YARN-5534. Both support the user supplying mounts via an
environment variable, which then gets added to the {{docker run}} command. The
only difference for that piece is the format of the user supplied environment
variable, but YARN-5534 goes a step further to attempt to validate the mount is
also in the white list. I reread your comment about why you feel this approach
is superior, but I'm not following what you mean.
was (Author: shaneku...@gmail.com):
[~eyang] thanks for the updated patch. I'm still confused as to what this is
providing over YARN-5534. Both support the user supplying mounts via an
environment variable, which then gets added to the {{docker run}} command. The
only difference for that piece is the format of the user supplied environment
variable, but YARN-5534 goes a step further to attempt to validate the mount is
also in the white list. I reread your comment about why you feel this approach
is similar, but I'm not following what you mean.
> Add ability to specify volumes to mount for DockerContainerRuntime
> --
>
> Key: YARN-7066
> URL: https://issues.apache.org/jira/browse/YARN-7066
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Affects Versions: 3.0.0-beta1
>Reporter: Eric Yang
> Attachments: YARN-7066.001.patch, YARN-7066.002.patch
>
>
> Yarnfile describes environment, docker image, and configuration template for
> launching docker containers in YARN. It would be nice to have ability to
> specify the volumes to mount. This can be used in combination to
> AMBARI-21748 to mount HDFS as data directories to docker containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7329) TestTimelineReaderWebServices times out on YARN-5355_branch2
Varun Saxena created YARN-7329: -- Summary: TestTimelineReaderWebServices times out on YARN-5355_branch2 Key: YARN-7329 URL: https://issues.apache.org/jira/browse/YARN-7329 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Saxena Priority: Critical -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7295) Backport ATSv2 to branch-2
[ https://issues.apache.org/jira/browse/YARN-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-7295: --- Attachment: YARN-7295-branch-2.01.patch Running patch for full ATS code against branch-2. > Backport ATSv2 to branch-2 > -- > > Key: YARN-7295 > URL: https://issues.apache.org/jira/browse/YARN-7295 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Saxena >Assignee: Varun Saxena >Priority: Critical > Attachments: YARN-7295-branch-2.01.patch > > > This JIRA is to run QA on patch with ATSv2 changes on branch-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7254) UI and metrics changes related to absolute resource configuration
[ https://issues.apache.org/jira/browse/YARN-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204528#comment-16204528 ] Sunil G commented on YARN-7254: --- These test case are tracked in YARN-7270 and is already committed. This branch will be rebased again after this commit [~leftnoteasy] please help to check the same. > UI and metrics changes related to absolute resource configuration > - > > Key: YARN-7254 > URL: https://issues.apache.org/jira/browse/YARN-7254 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-7254.001.patch, YARN-7254.002.patch, > YARN-7254.YARN-5881.002.patch, YARN-7254.YARN-5881.003.patch, > YARN-7254.YARN-5881.004.patch, YARN-7254.YARN-5881.005.patch, > YARN-7254.YARN-5881.006.patch, YARN-7254.YARN-5881.007.patch, > YARN-7254.YARN-5881.008.patch, YARN-7254.YARN-5881.009.patch > > > Impact on UI and metrics related to absolute resource configuration on CS -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204508#comment-16204508
]
Varun Saxena commented on YARN-7190:
Yes, I think so. I will put up a patch to fix shellcheck warnings.
> Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user
> classpath
>
>
> Key: YARN-7190
> URL: https://issues.apache.org/jira/browse/YARN-7190
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineclient, timelinereader, timelineserver
>Reporter: Vrushali C
>Assignee: Varun Saxena
> Attachments: YARN-7190-YARN-5355_branch2.01.patch,
> YARN-7190-YARN-5355_branch2.02.patch
>
>
> [~jlowe] had a good observation about the user classpath getting extra jars
> in hadoop 2.x brought in with TSv2. If users start picking up Hadoop 2,x's
> version of HBase jars instead of the ones they shipped with their job, it
> could be a problem.
> So when TSv2 is to be used in 2,x, the hbase related jars should come into
> only the NM classpath not the user classpath.
> Here is a list of some jars
> {code}
> commons-csv-1.0.jar
> commons-el-1.0.jar
> commons-httpclient-3.1.jar
> disruptor-3.3.0.jar
> findbugs-annotations-1.3.9-1.jar
> hbase-annotations-1.2.6.jar
> hbase-client-1.2.6.jar
> hbase-common-1.2.6.jar
> hbase-hadoop2-compat-1.2.6.jar
> hbase-hadoop-compat-1.2.6.jar
> hbase-prefix-tree-1.2.6.jar
> hbase-procedure-1.2.6.jar
> hbase-protocol-1.2.6.jar
> hbase-server-1.2.6.jar
> htrace-core-3.1.0-incubating.jar
> jamon-runtime-2.4.1.jar
> jasper-compiler-5.5.23.jar
> jasper-runtime-5.5.23.jar
> jcodings-1.0.8.jar
> joni-2.1.2.jar
> jsp-2.1-6.1.14.jar
> jsp-api-2.1-6.1.14.jar
> jsr311-api-1.1.1.jar
> metrics-core-2.2.0.jar
> servlet-api-2.5-6.1.14.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
