[jira] [Commented] (YARN-4476) Matcher for complex node label expresions
[
https://issues.apache.org/jira/browse/YARN-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002256#comment-16002256
]
Hadoop QA commented on YARN-4476:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
9s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m
23s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| |
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.IntervalSolver.match(Set)
makes inefficient use of keySet iterator instead of entrySet iterator At
IntervalSolver.java:keySet iterator instead of entrySet iterator At
IntervalSolver.java:[line 98] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-4476 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835853/YARN-4476.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 772c3529ae66 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 54fd0e4 |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15869/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15869/artifact/patchprocess/new-findbugs-hadoop-y
[jira] [Commented] (YARN-5606) Support multi-label merge into one node label
[
https://issues.apache.org/jira/browse/YARN-5606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002379#comment-16002379
]
jialei weng commented on YARN-5606:
---
Agree with Bin, [~Naganarasimha] Can you consider Bin's suggestion?
> Support multi-label merge into one node label
> -
>
> Key: YARN-5606
> URL: https://issues.apache.org/jira/browse/YARN-5606
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: resourcemanager
>Reporter: jialei weng
> Labels: oct16-hard
> Attachments: YARN-5606.001.patch, YARN-5606.002.patch
>
>
> Support multi-label merge into one node label
> 1. we want to support multo-label like SSD,GPU,FPGA label merged into single
> machine, joined by &. like SSD,GPU,FPGA-> SSD&GPU&FPGA
> 2. we add wildcard match to extend the job request. we define the wildcard
> like *GPU*, it will math all the node labels with GPU as part of multi-label
> merged label. For example, *GPU* will match SSD&GPU&FPGA, GPU&FPGA. We
> define SSD&GPU&FPGA={SSD,GPU,FPGA}, and GPU is one of {SSD,GPU,FPGA}, so the
> job can run on SSD&GPU&FPGA node.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6280) Add a query parameter in ResourceManager Cluster Applications REST API to control whether or not returns ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002402#comment-16002402
]
Sunil G commented on YARN-6280:
---
Thanks [~cltlfcjin]
Few comments
# {{DeSelectQuery.java}} could be inside *webapp* package itself. I do not
think its a common api class yet.
# {{DeSelectQuery}}. I think *setDeSelect* need not have to be invoked from
constructor. This method cud throw some exceptions, so it may be better as a
separate public method which could be invoked from RMWebService api.
# Java doc for {{DeSelectParam}} could be more descriptive and could explain
abt possible params which are supported now.
# In {{setDeSelect}} exception block, currently LOG.info is used. It could be
warn.
# in below code
{code}
43 if (deSelect != null && !deSelect.isEmpty()) {
44for (String query : deSelect) {
45 if (query != null && !query.trim().isEmpty()) {
{code}
I think {{query}} wont be null here. u may skip that. in first if condition, i
think {{deSelect.isEmpty()}} could be avoided. for loop will skip if
{{deSelect}} is empty. What you need is just check for null against deSelect.
Its better a simple null check could be added and return directly.
# Currently you have mentioned {{resourceRequests}} could be set in
{{deSelect}}. Thinking out loud, is it more error prone or tougher for user to
give exact *resourceRequests*. Could we name it *resource-requests* ? thoughts
> Add a query parameter in ResourceManager Cluster Applications REST API to
> control whether or not returns ResourceRequest
>
>
> Key: YARN-6280
> URL: https://issues.apache.org/jira/browse/YARN-6280
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, restapi
>Affects Versions: 2.7.3
>Reporter: Lantao Jin
>Assignee: Lantao Jin
> Attachments: YARN-6280.001.patch, YARN-6280.002.patch,
> YARN-6280.003.patch, YARN-6280.004.patch, YARN-6280.005.patch,
> YARN-6280.006.patch, YARN-6280.007.patch, YARN-6280.008.patch
>
>
> Begin from v2.7, the ResourceManager Cluster Applications REST API returns
> ResourceRequest list. It's a very large construction in AppInfo.
> As a test, we use below URI to query only 2 results:
> http:// address:port>/ws/v1/cluster/apps?states=running,accepted&limit=2
> The results are very different:
> ||Hadoop version|Total Character|Total Word|Total Lines|Size||
> |2.4.1|1192| 42| 42| 1.2 KB|
> |2.7.1|1222179| 48740| 48735| 1.21 MB|
> Most RESTful API requesters don't know about this after upgraded and their
> old queries may cause ResourceManager more GC consuming and slower. Even if
> they know this but have no idea to reduce the impact of ResourceManager
> except slow down their query frequency.
> The patch adding a query parameter "showResourceRequests" to help requesters
> who don't need this information to reduce the overhead. In consideration of
> compatibility of interface, the default value is true if they don't set the
> parameter, so the behaviour is the same as now.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5606) Support multi-label merge into one node label
[
https://issues.apache.org/jira/browse/YARN-5606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002453#comment-16002453
]
Naganarasimha G R commented on YARN-5606:
-
[~wjlei] & [~binw205],
Agree this is the need and it was suggested earlier in other jira too. But IMO
its not required to mention a partition expression for a resource request at
all. if the partition is available under a queue, it should be used for
allocation. So its more like node constraint/attribute to be used to identify
the node for a particular type. And the accessibility of a pool/partition to a
queue should govern which apps can make use of a particular type. I wanted to
bring in this functionality once the Node Attributes are in. hence had not made
any progress on this.
And i apologize for the delay in YARN-3409, will kick start that work shortly.
> Support multi-label merge into one node label
> -
>
> Key: YARN-5606
> URL: https://issues.apache.org/jira/browse/YARN-5606
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: resourcemanager
>Reporter: jialei weng
> Labels: oct16-hard
> Attachments: YARN-5606.001.patch, YARN-5606.002.patch
>
>
> Support multi-label merge into one node label
> 1. we want to support multo-label like SSD,GPU,FPGA label merged into single
> machine, joined by &. like SSD,GPU,FPGA-> SSD&GPU&FPGA
> 2. we add wildcard match to extend the job request. we define the wildcard
> like *GPU*, it will math all the node labels with GPU as part of multi-label
> merged label. For example, *GPU* will match SSD&GPU&FPGA, GPU&FPGA. We
> define SSD&GPU&FPGA={SSD,GPU,FPGA}, and GPU is one of {SSD,GPU,FPGA}, so the
> job can run on SSD&GPU&FPGA node.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5287) LinuxContainerExecutor fails to set proper permission
[ https://issues.apache.org/jira/browse/YARN-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5287: Fix Version/s: (was: 2.8.0) Removed 2.8.0 from fix version because it was reverted before 2.8.0 release. > LinuxContainerExecutor fails to set proper permission > - > > Key: YARN-5287 > URL: https://issues.apache.org/jira/browse/YARN-5287 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.7.2 >Reporter: Ying Zhang >Assignee: Naganarasimha G R >Priority: Minor > Fix For: 3.0.0-alpha1 > > Attachments: YARN-5287.003.patch, YARN-5287.004.patch, > YARN-5287.005.patch, YARN-5287.branch-2.001.patch, YARN-5287-tmp.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > LinuxContainerExecutor fails to set the proper permissions on the local > directories(i.e., /hadoop/yarn/local/usercache/... by default) if the cluster > has been configured with a restrictive umask, e.g.: umask 077. Job failed due > to the following reason: > Path /hadoop/yarn/local/usercache/ambari-qa/appcache/application_ has > permission 700 but needs permission 750 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6573) Improve CS performance and locality via time-based delay-scheduling and better matching
Carlo Curino created YARN-6573: -- Summary: Improve CS performance and locality via time-based delay-scheduling and better matching Key: YARN-6573 URL: https://issues.apache.org/jira/browse/YARN-6573 Project: Hadoop YARN Issue Type: Improvement Reporter: Carlo Curino This JIRA tracks work to improve the CS matching of containers to nodes. We switch to time-based delay scheduling, which allows for more aggressive locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6573) Improve CS performance and locality via time-based delay-scheduling and better matching
[ https://issues.apache.org/jira/browse/YARN-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlo Curino reassigned YARN-6573: -- Assignee: Hitesh Sharma > Improve CS performance and locality via time-based delay-scheduling and > better matching > --- > > Key: YARN-6573 > URL: https://issues.apache.org/jira/browse/YARN-6573 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Carlo Curino >Assignee: Hitesh Sharma > > This JIRA tracks work to improve the CS matching of containers to nodes. We > switch to time-based delay scheduling, which allows for more aggressive > locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6573) Improve CS performance and locality via time-based delay-scheduling and better matching
[ https://issues.apache.org/jira/browse/YARN-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002793#comment-16002793 ] Arun Suresh commented on YARN-6573: --- [~curino], YARN-6443 tracks the scheduler improvements that match containers to nodes better. Repurposing this for adding support for time based locality. > Improve CS performance and locality via time-based delay-scheduling and > better matching > --- > > Key: YARN-6573 > URL: https://issues.apache.org/jira/browse/YARN-6573 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Carlo Curino >Assignee: Hitesh Sharma > > This JIRA tracks work to improve the CS matching of containers to nodes. We > switch to time-based delay scheduling, which allows for more aggressive > locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6573) Allow for time based delay-scheduling in the Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6573: -- Summary: Allow for time based delay-scheduling in the Capacity Scheduler (was: Improve CS performance and locality via time-based delay-scheduling and better matching) > Allow for time based delay-scheduling in the Capacity Scheduler > --- > > Key: YARN-6573 > URL: https://issues.apache.org/jira/browse/YARN-6573 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Carlo Curino >Assignee: Hitesh Sharma > > This JIRA tracks work to improve the CS matching of containers to nodes. We > switch to time-based delay scheduling, which allows for more aggressive > locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6573) Allow for time based delay-scheduling in the Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002801#comment-16002801 ] Carlo Curino commented on YARN-6573: Works, thanks. > Allow for time based delay-scheduling in the Capacity Scheduler > --- > > Key: YARN-6573 > URL: https://issues.apache.org/jira/browse/YARN-6573 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Carlo Curino >Assignee: Hitesh Sharma > > This JIRA tracks work to improve the CS matching of containers to nodes. We > switch to time-based delay scheduling, which allows for more aggressive > locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6573) Allow for time based delay-scheduling in the Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002804#comment-16002804 ] Arun Suresh commented on YARN-6573: --- Considering that Fair Scheduler has support for this. I would prefer to somehow pull out it's implementation to be generally usable to all Schedulers. > Allow for time based delay-scheduling in the Capacity Scheduler > --- > > Key: YARN-6573 > URL: https://issues.apache.org/jira/browse/YARN-6573 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Carlo Curino >Assignee: Hitesh Sharma > > This JIRA tracks work to improve the CS matching of containers to nodes. We > switch to time-based delay scheduling, which allows for more aggressive > locality filtering during matching. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6544) Add Null check RegistryDNS service while parsing registry records
[ https://issues.apache.org/jira/browse/YARN-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karam Singh updated YARN-6544: -- Attachment: YARN-6544-yarn-native-services.003.patch > Add Null check RegistryDNS service while parsing registry records > - > > Key: YARN-6544 > URL: https://issues.apache.org/jira/browse/YARN-6544 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Affects Versions: yarn-native-services >Reporter: Karam Singh >Assignee: Karam Singh > Fix For: yarn-native-services > > Attachments: YARN-6544-yarn-native-services.001.patch, > YARN-6544-yarn-native-services.002.patch, > YARN-6544-yarn-native-services.002.patch, > YARN-6544-yarn-native-services.003.patch > > > Add Null check RegistryDNS service while parsing registry records for Yarn > persistance attribute. > As of now It assumes that yarn registry record always contain yarn > persistance which is not the case -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002918#comment-16002918 ] Haibo Chen commented on YARN-6566: -- Can you please elaborate on how the web UI layout is damaged (Maybe a screenshot)? We may be able to fix that layout issue. > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002941#comment-16002941 ] liuzhenhua commented on YARN-6566: -- for example,if i set a large HQL script text to job's 'name' property,the layout of web page like attach file applications-page.bmp will be damage > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002942#comment-16002942 ] Haibo Chen commented on YARN-6457: -- [~sanjaypujare] I have backported this to 2.7.4+ and 2.8.1+ as well > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Fix For: 2.9.0, 2.7.4, 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6457: - Fix Version/s: 2.8.1 2.7.4 > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Fix For: 2.9.0, 2.7.4, 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002941#comment-16002941 ] liuzhenhua edited comment on YARN-6566 at 5/9/17 4:06 PM: -- for example,if i set a large HQL script text to job's 'name' property,the layout of web page like attach file applications-page.bmp will be damage,so i choose to show the HQL text in the page like attach file application-page.bmp was (Author: liberty1789): for example,if i set a large HQL script text to job's 'name' property,the layout of web page like attach file applications-page.bmp will be damage > add a property for a hadoop job to identified the full hive HQL script text > in hadoop web view in multi-users environment > - > > Key: YARN-6566 > URL: https://issues.apache.org/jira/browse/YARN-6566 > Project: Hadoop YARN > Issue Type: Improvement > Components: client, resourcemanager, webapp, yarn >Affects Versions: 2.6.0 > Environment: centos 6.4 64bit >Reporter: liuzhenhua > Labels: patch > Fix For: 2.6.0 > > Attachments: application-page.bmp, applications-page.bmp, > YARN-6566.1.patch, YARN-6566.2.patch, YARN-6566.3.patch, YARN-6566.4.patch, > YARN-6566.5.patch, YARN-6566.6.patch, YARN-6566.7.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When I tuning the hive HQL in multi-users environment,I can not get the full > SQL text in hadoop web view,so it is difficult to tuning the SQL.When I try > to set the SQL text in hadoop job's jobname property,I realize it is going to > damage the structure of hadoop applications web view,so I add a property to > hadoop job,which named "jobdescription",When a hive HQL be submitted,the full > HQL text was assigned to the property,so i can identified the HQL coveniently -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6544) Add Null check RegistryDNS service while parsing registry records
[
https://issues.apache.org/jira/browse/YARN-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002960#comment-16002960
]
Hadoop QA commented on YARN-6544:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
31s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6544 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867122/YARN-6544-yarn-native-services.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ebdac3d3c0f2 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 4d4d19a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/15870/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15870/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15870/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add Null check RegistryDNS service while parsing registry records
> -
>
> Key: YARN-6544
> URL: https://issues.apache.org/jira/browse/YARN-6544
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002985#comment-16002985 ] Akira Ajisaka commented on YARN-6457: - Hi [~haibochen], would you backport this to branch-2.8.1 as well? > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Fix For: 2.9.0, 2.7.4, 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6571) Fix JavaDoc issues in SchedulingPolicy
[
https://issues.apache.org/jira/browse/YARN-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned YARN-6571:
--
Assignee: Weiwei Yang
> Fix JavaDoc issues in SchedulingPolicy
> --
>
> Key: YARN-6571
> URL: https://issues.apache.org/jira/browse/YARN-6571
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Weiwei Yang
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-6571.001.patch
>
>
> There are several javadoc issues:
> * Class JavaDoc is missing.
> * {{getInstance()}} is missing {{@return}} and {{@param}} tags.
> * {{parse()}} is missing {{@return}} tag and description for {{@throws}} tag.
> * {{checkIfUsageOverFairShare()}} is missing a period at the end of the first
> sentence.
> * {{getHeadroom()}} should use {code}{@code}{code} instead of {{}} tags.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003003#comment-16003003 ] Haibo Chen commented on YARN-6457: -- Will do. Did not realize that 2.8.1 has been cut. Thanks for the reminder! > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Fix For: 2.9.0, 2.7.4, 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6571) Fix JavaDoc issues in SchedulingPolicy
[
https://issues.apache.org/jira/browse/YARN-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003007#comment-16003007
]
Daniel Templeton commented on YARN-6571:
Thanks for the patch, [~cheersyang]. Wanna add JavaDoc for the class as well?
> Fix JavaDoc issues in SchedulingPolicy
> --
>
> Key: YARN-6571
> URL: https://issues.apache.org/jira/browse/YARN-6571
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Weiwei Yang
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-6571.001.patch
>
>
> There are several javadoc issues:
> * Class JavaDoc is missing.
> * {{getInstance()}} is missing {{@return}} and {{@param}} tags.
> * {{parse()}} is missing {{@return}} tag and description for {{@throws}} tag.
> * {{checkIfUsageOverFairShare()}} is missing a period at the end of the first
> sentence.
> * {{getHeadroom()}} should use {code}{@code}{code} instead of {{}} tags.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003049#comment-16003049
]
Sanjay M Pujare commented on YARN-4562:
---
[~vvasudev] Thanks. There is a branch-2.8.1 where I didn't see a commit for
this https://github.com/apache/hadoop/tree/branch-2.8.1 . Could you backport
there as well? Thanks
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha1, 2.8.1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6302) Fail the node if Linux Container Executor is not configured properly
[
https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003067#comment-16003067
]
Daniel Templeton commented on YARN-6302:
Thanks, [~miklos.szeg...@cloudera.com]. Committed to branch-2.
> Fail the node if Linux Container Executor is not configured properly
>
>
> Key: YARN-6302
> URL: https://issues.apache.org/jira/browse/YARN-6302
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-6302.000.patch, YARN-6302.001.patch,
> YARN-6302.002.patch, YARN-6302.003.patch, YARN-6302.005.branch-2.patch
>
>
> We have a cluster that has one node with misconfigured Linux Container
> Executor. Every time an AM or regular container is launched on the cluster,
> it will fail. The node will still have resources available, so it keeps
> failing apps until the administrator notices the issue and decommissions the
> node. AM Blacklisting only helps, if the application is already running.
> As a possible improvement, when the LCE is used on the cluster and a NM gets
> certain errors back from the LCE, like error 24 configuration not found, we
> should not try to allocate anything on the node anymore or shut down the node
> entirely. That kind of problem normally does not fix itself and it means that
> nothing can really run on that node.
> {code}
> Application application_1488920587909_0010 failed 2 times due to AM Container
> for appattempt_1488920587909_0010_02 exited with exitCode: -1000
> Failing this attempt.Diagnostics: Application application_1488920587909_0010
> initialization failed (exitCode=24) with output:
> For more detailed output, check the application tracking page:
> http://node-1.domain.com:8088/cluster/app/application_1488920587909_0010 Then
> click on links to logs of each attempt.
> . Failing the application.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6302) Fail the node if Linux Container Executor is not configured properly
[
https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated YARN-6302:
---
Fix Version/s: 2.9.0
> Fail the node if Linux Container Executor is not configured properly
>
>
> Key: YARN-6302
> URL: https://issues.apache.org/jira/browse/YARN-6302
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-6302.000.patch, YARN-6302.001.patch,
> YARN-6302.002.patch, YARN-6302.003.patch, YARN-6302.005.branch-2.patch
>
>
> We have a cluster that has one node with misconfigured Linux Container
> Executor. Every time an AM or regular container is launched on the cluster,
> it will fail. The node will still have resources available, so it keeps
> failing apps until the administrator notices the issue and decommissions the
> node. AM Blacklisting only helps, if the application is already running.
> As a possible improvement, when the LCE is used on the cluster and a NM gets
> certain errors back from the LCE, like error 24 configuration not found, we
> should not try to allocate anything on the node anymore or shut down the node
> entirely. That kind of problem normally does not fix itself and it means that
> nothing can really run on that node.
> {code}
> Application application_1488920587909_0010 failed 2 times due to AM Container
> for appattempt_1488920587909_0010_02 exited with exitCode: -1000
> Failing this attempt.Diagnostics: Application application_1488920587909_0010
> initialization failed (exitCode=24) with output:
> For more detailed output, check the application tracking page:
> http://node-1.domain.com:8088/cluster/app/application_1488920587909_0010 Then
> click on links to logs of each attempt.
> . Failing the application.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5531) UnmanagedAM pool manager for federating application across clusters
[ https://issues.apache.org/jira/browse/YARN-5531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-5531: --- Attachment: YARN-5531-YARN-2915.v10.patch Thanks [~subru] for the fast review. All comments addressed in v10 patch. > UnmanagedAM pool manager for federating application across clusters > --- > > Key: YARN-5531 > URL: https://issues.apache.org/jira/browse/YARN-5531 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Botong Huang > Attachments: YARN-5531-YARN-2915.v10.patch, > YARN-5531-YARN-2915.v1.patch, YARN-5531-YARN-2915.v2.patch, > YARN-5531-YARN-2915.v3.patch, YARN-5531-YARN-2915.v4.patch, > YARN-5531-YARN-2915.v5.patch, YARN-5531-YARN-2915.v6.patch, > YARN-5531-YARN-2915.v7.patch, YARN-5531-YARN-2915.v8.patch, > YARN-5531-YARN-2915.v9.patch > > > One of the main tenets the YARN Federation is to *transparently* scale > applications across multiple clusters. This is achieved by running UAMs on > behalf of the application on other clusters. This JIRA tracks the addition of > a UnmanagedAM pool manager for federating application across clusters which > will be used the FederationInterceptor (YARN-3666) which is part of the > AMRMProxy pipeline introduced in YARN-2884. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch
[
https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003127#comment-16003127
]
Vrushali C commented on YARN-6559:
--
Yes, that output is confusing.
Looking at the console output
https://builds.apache.org/job/PreCommit-YARN-Build/15853/console
it says
{code}
| +1 | findbugs | 0m 30s |
| |||
hadoop-yarn-project/hadoop-yarn/hadoop-yar
| |||
n-server/hadoop-yarn-server-timelineservic
| ||| e generated 0 new + 0 unchanged - 1 fixed
| ||| = 0 total (was 1)
{code}
> Findbugs warning in YARN-5355 branch
>
>
> Key: YARN-6559
> URL: https://issues.apache.org/jira/browse/YARN-6559
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Varun Saxena
>Assignee: Vrushali C
>Priority: Minor
> Attachments: FindBugs Report.html, YARN-6559-YARN-5355.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6568) A queue which runs a long time job couldn't acquire any container for long time.
[
https://issues.apache.org/jira/browse/YARN-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003143#comment-16003143
]
Yufei Gu commented on YARN-6568:
According to the code in FairSharePolicy.java, {{minShareRatio}} is
{{resourceUsage / min(minShare, demand)}} if {{minShare}} is big enough. Why
queueA's {{minShareRatio}} is always bigger than queueB's? {{resourceUsage}} of
queueA could be zero or will be zero over time in your case.
> A queue which runs a long time job couldn't acquire any container for long
> time.
>
>
> Key: YARN-6568
> URL: https://issues.apache.org/jira/browse/YARN-6568
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
> Environment: CentOS 7.1
>Reporter: zhengchenyu
> Fix For: 2.7.4
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> In our cluster, we find some applications couldn't acquire any container for
> long time. (Note: we use FairSharePolicy and FairScheduler)
> First, I found some unreasonable configuration, we set minRes=maxRes. So some
> application keep pending for long time, we kill some large applicaiton to
> solve this problem. Then we changed this configuration, this problem
> relieves.
> But this problem is not completely solved. In our cluster, I found
> applications in some queue which request few container keep pending for long
> time.
> I simulate in test cluster. I submit DistributedShell application which run
> many loo applications to queueA, then I submit my own yarn application which
> request container and release container constantly to queueB. At this time,
> any applicaitons which are submmited to queueA keep pending!
> We know this is the problem of FairSharePolicy, it consider the request of
> queue. So after sort the queues, some queues which have few request are
> ordered last all time.
> We know if the AM container is launched, then the request will increase, But
> FairSharePolicy can't distinguish which request is AM request. I think if am
> container is assigned, the problem is solved.
> Our companion discuss this problem. we recommend set a timeout for queue, it
> means the time length of a queue is not assigned. If timeout, we set this
> queue to the first place of queues list.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4476) Matcher for complex node label expresions
[ https://issues.apache.org/jira/browse/YARN-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated YARN-4476: Attachment: YARN-4476.004.patch > Matcher for complex node label expresions > - > > Key: YARN-4476 > URL: https://issues.apache.org/jira/browse/YARN-4476 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Chris Douglas >Assignee: Chris Douglas > Labels: oct16-medium > Attachments: YARN-4476.003.patch, YARN-4476.004.patch, > YARN-4476-0.patch, YARN-4476-1.patch, YARN-4476-2.patch > > > Implementation of a matcher for complex node label expressions based on a > [paper|http://dl.acm.org/citation.cfm?id=1807171] from SIGMOD 2010. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems
[
https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003151#comment-16003151
]
Daniel Templeton commented on YARN-5301:
OK, did one last pass. Not a biggie, but in
{{initializeControllerPathsFromMtab()}} the {{parsedMtab}} param hides the
{{parsedMtab}} member variable. I'd be OK with ignoring that. Also, it looks
like we never finished the conversation about how best to do the String
concatenations. Can we just go with {{StringUtils.join()}}? It makes me cringe
to add back in all those pluses.
> NM mount cpu cgroups failed on some systems
> ---
>
> Key: YARN-5301
> URL: https://issues.apache.org/jira/browse/YARN-5301
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: sandflee
>Assignee: Miklos Szegedi
> Attachments: YARN-5301.000.patch, YARN-5301.001.patch,
> YARN-5301.002.patch, YARN-5301.003.patch, YARN-5301.004.patch,
> YARN-5301.005.patch, YARN-5301.006.patch, YARN-5301.007.patch,
> YARN-5301.008.patch, YARN-5301.009.patch, YARN-5301.010.patch
>
>
> on ubuntu with linux kernel 3.19, , NM start failed if enable auto mount
> cgroup. try command:
> ./bin/container-executor --mount-cgroups yarn-hadoop cpu=/cgroup/cpufail
> ./bin/container-executor --mount-cgroups yarn-hadoop cpu,cpuacct=/cgroup/cpu
> succ
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003187#comment-16003187
]
Varun Vasudev commented on YARN-4562:
-
Thanks for pointing that out [~sanjaypujare]. I've backported it to 2.8.1 as
well.
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha1, 2.8.1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6563) ConcurrentModificationException in TimelineCollectorManager while stopping RM
[
https://issues.apache.org/jira/browse/YARN-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003191#comment-16003191
]
Vrushali C commented on YARN-6563:
--
+ 1 LGTM
Will commit it shortly.
> ConcurrentModificationException in TimelineCollectorManager while stopping RM
> -
>
> Key: YARN-6563
> URL: https://issues.apache.org/jira/browse/YARN-6563
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Rohith Sharma K S
>Assignee: Haibo Chen
> Attachments: YARN-6563.00.patch
>
>
> It is seen that ConcurrentModificationException while stopping RM when ATSv2
> enabled.
> {noformat}
> 2017-05-05 15:04:11,563 WARN org.apache.hadoop.service.CompositeService: When
> stopping the service
> org.apache.hadoop.yarn.server.resourcemanager.timelineservice.RMTimelineCollectorManager
> : java.util.ConcurrentModificationException
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
> at java.util.HashMap$ValueIterator.next(HashMap.java:1466)
> at
> org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorManager.serviceStop(TimelineCollectorManager.java:222)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStop(ResourceManager.java:1285)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003194#comment-16003194 ] Miklos Szegedi commented on YARN-5301: -- Thank you for the review [~templedf]. I think, there is no member field hiding in initializeControllerPathsFromMtab because it is a static function, so it does not relate to any object members. In terms of string concatenation there are two sides of the question. The style side is a theological question, so I do not want to comment on that. Talking about functionality the compiler instructs us to use + instead of StringBuilder because of performance. StringUtils.join uses StringBuilder inside, so using it would be a regression. I think we should use + in all cases with a fixed number of + operators and use StringUtils.join, when we have variable number of items. > NM mount cpu cgroups failed on some systems > --- > > Key: YARN-5301 > URL: https://issues.apache.org/jira/browse/YARN-5301 > Project: Hadoop YARN > Issue Type: Bug >Reporter: sandflee >Assignee: Miklos Szegedi > Attachments: YARN-5301.000.patch, YARN-5301.001.patch, > YARN-5301.002.patch, YARN-5301.003.patch, YARN-5301.004.patch, > YARN-5301.005.patch, YARN-5301.006.patch, YARN-5301.007.patch, > YARN-5301.008.patch, YARN-5301.009.patch, YARN-5301.010.patch > > > on ubuntu with linux kernel 3.19, , NM start failed if enable auto mount > cgroup. try command: > ./bin/container-executor --mount-cgroups yarn-hadoop cpu=/cgroup/cpufail > ./bin/container-executor --mount-cgroups yarn-hadoop cpu,cpuacct=/cgroup/cpu > succ -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[
https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003219#comment-16003219
]
Hadoop QA commented on YARN-6533:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
33s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
58s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
in yarn-native-services has 3 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 19s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 1 new + 146 unchanged - 1 fixed = 147 total (was 147)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
45s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6533 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865165/YARN-6533-yarn-native-services.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0f0f848d20df 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 4d4d19a |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15876/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15876/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core.txt
|
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[
https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003221#comment-16003221
]
Hadoop QA commented on YARN-6533:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
1s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
3s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
in yarn-native-services has 3 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 1 new + 146 unchanged - 1 fixed = 147 total (was 147)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
39s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6533 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865165/YARN-6533-yarn-native-services.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 81f6fc2aa8d6 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 4d4d19a |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15875/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15875/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core.txt
|
[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003224#comment-16003224 ] Daniel Templeton commented on YARN-5301: Interesting. http://stackoverflow.com/questions/1532461/stringbuilder-vs-string-concatenation-in-tostring-in-java Looks like the performance is actual identical between the two approaches because the compiler inserts the StringBuilder under the covers. Things I did not know. OK. +1 > NM mount cpu cgroups failed on some systems > --- > > Key: YARN-5301 > URL: https://issues.apache.org/jira/browse/YARN-5301 > Project: Hadoop YARN > Issue Type: Bug >Reporter: sandflee >Assignee: Miklos Szegedi > Attachments: YARN-5301.000.patch, YARN-5301.001.patch, > YARN-5301.002.patch, YARN-5301.003.patch, YARN-5301.004.patch, > YARN-5301.005.patch, YARN-5301.006.patch, YARN-5301.007.patch, > YARN-5301.008.patch, YARN-5301.009.patch, YARN-5301.010.patch > > > on ubuntu with linux kernel 3.19, , NM start failed if enable auto mount > cgroup. try command: > ./bin/container-executor --mount-cgroups yarn-hadoop cpu=/cgroup/cpufail > ./bin/container-executor --mount-cgroups yarn-hadoop cpu,cpuacct=/cgroup/cpu > succ -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6545) Followup fix for YARN-6405
[
https://issues.apache.org/jira/browse/YARN-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003228#comment-16003228
]
Hadoop QA commented on YARN-6545:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
7s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
23s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
56s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
3s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
in yarn-native-services has 3 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
55s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in yarn-native-services has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} yarn-native-services passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 9m 12s{color}
| {color:red} hadoop-yarn-project_hadoop-yarn generated 6 new + 65 unchanged -
6 fixed = 71 total (was 71) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 2s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 11 new + 675 unchanged - 9 fixed = 686 total (was 684) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
26s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
59s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6545 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12866986/YARN-6545.yarn-native-services.07.patch
|
| Op
[jira] [Updated] (YARN-5413) Create a proxy chain for ResourceManager Admin API in the Router
[ https://issues.apache.org/jira/browse/YARN-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-5413: --- Attachment: YARN-5413-YARN-2915.v3.patch > Create a proxy chain for ResourceManager Admin API in the Router > > > Key: YARN-5413 > URL: https://issues.apache.org/jira/browse/YARN-5413 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5413-YARN-2915.v1.patch, > YARN-5413-YARN-2915.v2.patch, YARN-5413-YARN-2915.v3.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager Admin API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5413) Create a proxy chain for ResourceManager Admin API in the Router
[ https://issues.apache.org/jira/browse/YARN-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-5413: --- Attachment: (was: YARN-5413-YARN-2915.v3.patch) > Create a proxy chain for ResourceManager Admin API in the Router > > > Key: YARN-5413 > URL: https://issues.apache.org/jira/browse/YARN-5413 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5413-YARN-2915.v1.patch, > YARN-5413-YARN-2915.v2.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager Admin API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003254#comment-16003254 ] Eric Payne commented on YARN-2113: -- [~sunilg] / [~leftnoteasy]. It looks like the latest patch has all of the features we agreed on. Thanks [~sunilg] for all of your hard work on this feature and [~leftnoteasy] for all of your reviews and valuable feedback. > Add cross-user preemption within CapacityScheduler's leaf-queue > --- > > Key: YARN-2113 > URL: https://issues.apache.org/jira/browse/YARN-2113 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Vinod Kumar Vavilapalli >Assignee: Sunil G > Attachments: IntraQueue Preemption-Impact Analysis.pdf, > TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt, > YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, > YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch, > YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch, > YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch, > YARN-2113.0013.patch, YARN-2113.0014.patch, > YARN-2113.apply.onto.0012.ericp.patch, YARN-2113 Intra-QueuePreemption > Behavior.pdf, YARN-2113.v0.patch > > > Preemption today only works across queues and moves around resources across > queues per demand and usage. We should also have user-level preemption within > a queue, to balance capacity across users in a predictable manner. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6563) ConcurrentModificationException in TimelineCollectorManager while stopping RM
[
https://issues.apache.org/jira/browse/YARN-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003278#comment-16003278
]
Hudson commented on YARN-6563:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11706 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11706/])
YARN-6563 ConcurrentModificationException in TimelineCollectorManager
(vrushali: rev 7dd258d8f4aef594346e874e5ad4ba22c3171cd1)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollectorManager.java
> ConcurrentModificationException in TimelineCollectorManager while stopping RM
> -
>
> Key: YARN-6563
> URL: https://issues.apache.org/jira/browse/YARN-6563
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Rohith Sharma K S
>Assignee: Haibo Chen
> Fix For: YARN-5355, YARN-5355-branch-2, 3.0.0-alpha3
>
> Attachments: YARN-6563.00.patch
>
>
> It is seen that ConcurrentModificationException while stopping RM when ATSv2
> enabled.
> {noformat}
> 2017-05-05 15:04:11,563 WARN org.apache.hadoop.service.CompositeService: When
> stopping the service
> org.apache.hadoop.yarn.server.resourcemanager.timelineservice.RMTimelineCollectorManager
> : java.util.ConcurrentModificationException
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
> at java.util.HashMap$ValueIterator.next(HashMap.java:1466)
> at
> org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorManager.serviceStop(TimelineCollectorManager.java:222)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStop(ResourceManager.java:1285)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4476) Matcher for complex node label expresions
[
https://issues.apache.org/jira/browse/YARN-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003293#comment-16003293
]
Hadoop QA commented on YARN-4476:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 28 new + 0 unchanged - 0 fixed = 28 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 40m
49s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-4476 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867140/YARN-4476.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux bf12a20e8faa 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 45382bf |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15877/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15877/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15877/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Matcher for complex node label expresions
> -
>
> Key: YARN-4476
> URL: http
[jira] [Updated] (YARN-6504) Add support for resource profiles in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-6504: Attachment: YARN-6504-YARN-3926.001.patch > Add support for resource profiles in MapReduce > -- > > Key: YARN-6504 > URL: https://issues.apache.org/jira/browse/YARN-6504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: YARN-6504-YARN-3926.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5531) UnmanagedAM pool manager for federating application across clusters
[
https://issues.apache.org/jira/browse/YARN-5531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003365#comment-16003365
]
Hadoop QA commented on YARN-5531:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
3s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
17s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
31s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
30s{color} | {color:green} YARN-2915 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
10s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
YARN-2915 has 1 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in YARN-2915 has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
36s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 53s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 48 unchanged - 1 fixed = 49 total (was 49) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
19s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
2s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 40m 4s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}128m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5531 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867138/YARN-5531-YARN-2915.v10.patch
|
| Optional Tests | asflicense compile j
[jira] [Commented] (YARN-3742) YARN RM will shut down if ZKClient creation times out
[
https://issues.apache.org/jira/browse/YARN-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003374#comment-16003374
]
Hadoop QA commented on YARN-3742:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 51s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 132 unchanged - 4 fixed = 133 total (was 136) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 34s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
50s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-3742 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863141/YARN-3742.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3462e2ad67dc 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 45382bf |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15874/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15874/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results
[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003402#comment-16003402 ] Haibo Chen commented on YARN-6559: -- Thanks for pointing that out. +1 > Findbugs warning in YARN-5355 branch > > > Key: YARN-6559 > URL: https://issues.apache.org/jira/browse/YARN-6559 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Vrushali C >Priority: Minor > Attachments: FindBugs Report.html, YARN-6559-YARN-5355.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003410#comment-16003410 ] Haibo Chen commented on YARN-6559: -- Wait, this looks like YARN-6518? Should we cherry-pick that instead? > Findbugs warning in YARN-5355 branch > > > Key: YARN-6559 > URL: https://issues.apache.org/jira/browse/YARN-6559 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Vrushali C >Priority: Minor > Attachments: FindBugs Report.html, YARN-6559-YARN-5355.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6561) Update exception information during timeline collector aux service initialization
[ https://issues.apache.org/jira/browse/YARN-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003418#comment-16003418 ] Haibo Chen commented on YARN-6561: -- +1 on the patch. Will wait until the end of today to commit > Update exception information during timeline collector aux service > initialization > - > > Key: YARN-6561 > URL: https://issues.apache.org/jira/browse/YARN-6561 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Vrushali C >Priority: Minor > Attachments: YARN-6561.001.patch > > > If the NM is started with timeline service v2 turned off AND aux services > setting still containing "timeline_collector", NM will fail to start up since > the PerNodeTimelineCollectorsAuxService#serviceInit is invoked and it throws > an exception. The exception message is a bit misleading and does not indicate > where the actual misconfiguration is. > We should update the exception message so that the right error is conveyed > and helps the cluster admin/ops to correct the relevant yarn config settings. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[ https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlo Curino updated YARN-6484: --- Attachment: YARN-6484-YARN-2915.v0.patch > [Documentation] Documenting the YARN Federation feature > --- > > Key: YARN-6484 > URL: https://issues.apache.org/jira/browse/YARN-6484 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan > Attachments: YARN-6484-YARN-2915.v0.patch > > > We should document the high level design and configuration to enable YARN > Federation -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[ https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003428#comment-16003428 ] Carlo Curino commented on YARN-6484: Initial version of documentation produce from old docs, and current conf. [~botong] can you please take a look? Make sure the docs are aligned with the latest set of changes. > [Documentation] Documenting the YARN Federation feature > --- > > Key: YARN-6484 > URL: https://issues.apache.org/jira/browse/YARN-6484 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan > Attachments: YARN-6484-YARN-2915.v0.patch > > > We should document the high level design and configuration to enable YARN > Federation -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[ https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlo Curino reassigned YARN-6484: -- Assignee: Carlo Curino Affects Version/s: YARN-2915 > [Documentation] Documenting the YARN Federation feature > --- > > Key: YARN-6484 > URL: https://issues.apache.org/jira/browse/YARN-6484 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: YARN-2915 >Reporter: Subru Krishnan >Assignee: Carlo Curino > Attachments: YARN-6484-YARN-2915.v0.patch > > > We should document the high level design and configuration to enable YARN > Federation -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003452#comment-16003452 ] Hudson commented on YARN-5301: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11707 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11707/]) YARN-5301. NM mount cpu cgroups failed on some systems (Contributed by (templedf: rev a2f680493f040704e2b85108e286731ee3860a52) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java > NM mount cpu cgroups failed on some systems > --- > > Key: YARN-5301 > URL: https://issues.apache.org/jira/browse/YARN-5301 > Project: Hadoop YARN > Issue Type: Bug >Reporter: sandflee >Assignee: Miklos Szegedi > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-5301.000.patch, YARN-5301.001.patch, > YARN-5301.002.patch, YARN-5301.003.patch, YARN-5301.004.patch, > YARN-5301.005.patch, YARN-5301.006.patch, YARN-5301.007.patch, > YARN-5301.008.patch, YARN-5301.009.patch, YARN-5301.010.patch > > > on ubuntu with linux kernel 3.19, , NM start failed if enable auto mount > cgroup. try command: > ./bin/container-executor --mount-cgroups yarn-hadoop cpu=/cgroup/cpufail > ./bin/container-executor --mount-cgroups yarn-hadoop cpu,cpuacct=/cgroup/cpu > succ -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6545) Followup fix for YARN-6405
[ https://issues.apache.org/jira/browse/YARN-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003462#comment-16003462 ] Billie Rinaldi commented on YARN-6545: -- +1 for patch 07. > Followup fix for YARN-6405 > -- > > Key: YARN-6545 > URL: https://issues.apache.org/jira/browse/YARN-6545 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-6545.yarn-native-services.01.patch, > YARN-6545.yarn-native-services.02.patch, > YARN-6545.yarn-native-services.03.patch, > YARN-6545.yarn-native-services.04.patch, > YARN-6545.yarn-native-services.05.patch, > YARN-6545.yarn-native-services.06.patch, > YARN-6545.yarn-native-services.07.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[ https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003474#comment-16003474 ] Jian He commented on YARN-6533: --- +1, committing > Race condition in writing service record to registry in yarn native services > > > Key: YARN-6533 > URL: https://issues.apache.org/jira/browse/YARN-6533 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-6533-yarn-native-services.001.patch > > > The ServiceRecord is written twice, once when the container is initially > registered and again in the Docker provider once the IP has been obtained for > the container. These occur asynchronously, so the more important record (the > one with the IP) can be overwritten by the initial record. Only one record > needs to be written, so we can stop writing the initial record when the > Docker provider is being used. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3545) Investigate the concurrency issue with the map of timeline collector
[ https://issues.apache.org/jira/browse/YARN-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003479#comment-16003479 ] Haibo Chen commented on YARN-3545: -- Looking at this as it seems related to YARN-6563. [~vrushalic], still remember the reason why Li and you decided to not proceed? > Investigate the concurrency issue with the map of timeline collector > > > Key: YARN-3545 > URL: https://issues.apache.org/jira/browse/YARN-3545 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Li Lu > Labels: YARN-5355, oct16-medium > Attachments: YARN-3545-YARN-2928.000.patch > > > See the discussion in YARN-3390 for details. Let's continue the discussion > here. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003476#comment-16003476 ] Vrushali C commented on YARN-6559: -- Huh, interesting. Naga does mention that he has committed to YARN-5355 but why don't we see it? Am I missing something. > Findbugs warning in YARN-5355 branch > > > Key: YARN-6559 > URL: https://issues.apache.org/jira/browse/YARN-6559 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Vrushali C >Priority: Minor > Attachments: FindBugs Report.html, YARN-6559-YARN-5355.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[
https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003480#comment-16003480
]
Hadoop QA commented on YARN-6484:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
57s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 50 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 18 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6484 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867172/YARN-6484-YARN-2915.v0.patch
|
| Optional Tests | asflicense mvnsite xml |
| uname | Linux 9df4d9465583 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-2915 / 0a93a32 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/15879/artifact/patchprocess/whitespace-eol.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/15879/artifact/patchprocess/whitespace-tabs.txt
|
| modules | C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
U: . |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15879/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [Documentation] Documenting the YARN Federation feature
> ---
>
> Key: YARN-6484
> URL: https://issues.apache.org/jira/browse/YARN-6484
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Affects Versions: YARN-2915
>Reporter: Subru Krishnan
>Assignee: Carlo Curino
> Attachments: YARN-6484-YARN-2915.v0.patch
>
>
> We should document the high level design and configuration to enable YARN
> Federation
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[ https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003496#comment-16003496 ] Jian He commented on YARN-6533: --- Actually, there's an issue, previously, componentInstanceStarted is called in registerComponent which will put some initial ATS data, then componentInstanceUpdated will be called. Now only componentInstanceUpdated will be called, some ATS data will be missing because of this. we need to merge componentInstanceStarted into componentInstanceUpdated > Race condition in writing service record to registry in yarn native services > > > Key: YARN-6533 > URL: https://issues.apache.org/jira/browse/YARN-6533 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-6533-yarn-native-services.001.patch > > > The ServiceRecord is written twice, once when the container is initially > registered and again in the Docker provider once the IP has been obtained for > the container. These occur asynchronously, so the more important record (the > one with the IP) can be overwritten by the initial record. Only one record > needs to be written, so we can stop writing the initial record when the > Docker provider is being used. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3872) TimelineReader Web UI Implementation
[ https://issues.apache.org/jira/browse/YARN-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003512#comment-16003512 ] Haibo Chen commented on YARN-3872: -- [~varun_impala_149e] Is the reader UI based on YARN web ui 2.0, or just the old-school java servlet? > TimelineReader Web UI Implementation > > > Key: YARN-3872 > URL: https://issues.apache.org/jira/browse/YARN-3872 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: YARN-5355 > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[
https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003537#comment-16003537
]
Haibo Chen commented on YARN-5094:
--
[~gtCarrera9] Do you mind if I take over? I am looking at YARN-5169, which is
the real cause.
> some YARN container events have timestamp of -1 in REST output
> --
>
> Key: YARN-5094
> URL: https://issues.apache.org/jira/browse/YARN-5094
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Li Lu
> Labels: YARN-5355
> Attachments: YARN-5094-YARN-2928.001.patch
>
>
> Some events in the YARN container entities have timestamp of -1. The
> RM-generated container events have proper timestamps. It appears that it's
> the NM-generated events that have -1: YARN_CONTAINER_CREATED,
> YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED,
> YARN_NM_CONTAINER_LOCALIZATION_STARTED.
> In the YARN container page,
> {noformat}
> {
> id: "YARN_CONTAINER_CREATED",
> timestamp: -1,
> info: { }
> },
> {
> id: "YARN_CONTAINER_FINISHED",
> timestamp: -1,
> info: {
> YARN_CONTAINER_EXIT_STATUS: 0,
> YARN_CONTAINER_STATE: "RUNNING",
> YARN_CONTAINER_DIAGNOSTICS_INFO: ""
> }
> },
> {
> id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED",
> timestamp: -1,
> info: { }
> },
> {
> id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED",
> timestamp: -1,
> info: { }
> }
> {noformat}
> I think the data itself is OK, but the values are not being populated in the
> REST output?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6574) Support global configuration mutation in MutableConfProvider
Jonathan Hung created YARN-6574: --- Summary: Support global configuration mutation in MutableConfProvider Key: YARN-6574 URL: https://issues.apache.org/jira/browse/YARN-6574 Project: Hadoop YARN Issue Type: Improvement Reporter: Jonathan Hung Assignee: Jonathan Hung Right now mutating configs assumes they are only queue configs. Support should be added to mutate global scheduler configs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003557#comment-16003557 ] Jonathan Hung commented on YARN-5949: - Discussed this offline, will upload a patch addressing scheduler agnostic configuration mutation. Also created YARN-6574 for supporting global scheduler configuration mutation. > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6575) Support global configuration mutation in MutableConfProvider
Jonathan Hung created YARN-6575: --- Summary: Support global configuration mutation in MutableConfProvider Key: YARN-6575 URL: https://issues.apache.org/jira/browse/YARN-6575 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jonathan Hung Assignee: Jonathan Hung Right now mutating configs assumes they are only queue configs. Support should be added to mutate global scheduler configs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6574) Support global configuration mutation in MutableConfProvider
[ https://issues.apache.org/jira/browse/YARN-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved YARN-6574. - Resolution: Duplicate Dupe of YARN-6575 (seems you cannot create a ticket and retroactively make it a subtask of another ticket...) > Support global configuration mutation in MutableConfProvider > > > Key: YARN-6574 > URL: https://issues.apache.org/jira/browse/YARN-6574 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Jonathan Hung > > Right now mutating configs assumes they are only queue configs. Support > should be added to mutate global scheduler configs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003557#comment-16003557 ] Jonathan Hung edited comment on YARN-5949 at 5/9/17 9:08 PM: - Discussed this offline, will upload a patch addressing scheduler agnostic configuration mutation. Also created YARN-6575 for supporting global scheduler configuration mutation. was (Author: jhung): Discussed this offline, will upload a patch addressing scheduler agnostic configuration mutation. Also created YARN-6574 for supporting global scheduler configuration mutation. > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6533) Race condition in writing service record to registry in yarn native services
[ https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6533: - Attachment: YARN-6533-yarn-native-services.002.patch > Race condition in writing service record to registry in yarn native services > > > Key: YARN-6533 > URL: https://issues.apache.org/jira/browse/YARN-6533 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-6533-yarn-native-services.001.patch, > YARN-6533-yarn-native-services.002.patch > > > The ServiceRecord is written twice, once when the container is initially > registered and again in the Docker provider once the IP has been obtained for > the container. These occur asynchronously, so the more important record (the > one with the IP) can be overwritten by the initial record. Only one record > needs to be written, so we can stop writing the initial record when the > Docker provider is being used. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3545) Investigate the concurrency issue with the map of timeline collector
[ https://issues.apache.org/jira/browse/YARN-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003581#comment-16003581 ] Vrushali C commented on YARN-3545: -- Hmm, I think we thought the patch was stale even then and decided to come back to it later. I think it might be fine to look at it afresh now. > Investigate the concurrency issue with the map of timeline collector > > > Key: YARN-3545 > URL: https://issues.apache.org/jira/browse/YARN-3545 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Li Lu > Labels: YARN-5355, oct16-medium > Attachments: YARN-3545-YARN-2928.000.patch > > > See the discussion in YARN-3390 for details. Let's continue the discussion > here. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[ https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlo Curino updated YARN-6484: --- Attachment: YARN-6484-YARN-2915.v1.patch > [Documentation] Documenting the YARN Federation feature > --- > > Key: YARN-6484 > URL: https://issues.apache.org/jira/browse/YARN-6484 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: YARN-2915 >Reporter: Subru Krishnan >Assignee: Carlo Curino > Attachments: YARN-6484-YARN-2915.v0.patch, > YARN-6484-YARN-2915.v1.patch > > > We should document the high level design and configuration to enable YARN > Federation -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3742) YARN RM will shut down if ZKClient creation times out
[
https://issues.apache.org/jira/browse/YARN-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003614#comment-16003614
]
Karthik Kambatla commented on YARN-3742:
Missed that. Looks good. +1.
> YARN RM will shut down if ZKClient creation times out
> ---
>
> Key: YARN-3742
> URL: https://issues.apache.org/jira/browse/YARN-3742
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Daniel Templeton
> Attachments: YARN-3742.001.patch, YARN-3742.002.patch
>
>
> The RM goes down showing the following stacktrace if the ZK client connection
> fails to be created. We should not exit but transition to StandBy and stop
> doing things and let the other RM take over.
> {code}
> 2015-04-19 01:22:20,513 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> java.io.IOException: Wait for ZKClient creation timed out
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1066)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1090)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.existsWithRetries(ZKRMStateStore.java:996)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationStateInternal(ZKRMStateStore.java:643)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:162)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:147)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:806)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:879)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:874)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6504) Add support for resource profiles in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003617#comment-16003617
]
Daniel Templeton commented on YARN-6504:
A few comments, [~vvasudev]:
{{TaskAttemptImpl}}
* Since you're setting {{resourceProfile}} in the constructor, it would be
better not to set it in the declaration.
* The {{LOG.info()}} in the constructor should probably be {{LOG.debug()}}. I
might also move it into {{getResourceProfile()}}.
{{ContainerRequestEvent}}
* {{Configuration}} import is unused.
* {{resourceProfile}} in the constructor args should probably come right after
{{capability}}.
* Is it useful to overload {{createContainerRequestEventForFailedContainer()}}?
Doesn't look like the 2-arg version is needed anymore. And if you dump the
2-arg version, you can add profile to the 2-arg constructor, making
{{createContainerRequestEventForFailedContainer()}} simpler.
* Missing javadoc for new accessors.
{{RMCommunicator}}
* Missing javadoc for {{getResourceProfilesMap()}}
{{RMContainerAllocator}}
* {{Hamlet}} import is unused.
* Might be cleaner to move the logic about calculating a resource from the
profile and capability into a method you can reuse.
{{RMContainerRequester}}
* The profile arg in the {{ContainerRequest}} constructors should come right
after capability.
{{MRJobConfig}}
* {{DEFAULT_REDUCE_RESOURCE_PROFILE}} appears unused.
{{ProfileCapability}}
* Is it important to fail a null override? I should think it would be
friendlier to treat it as {{Resource.newInstance(0, 0)}}.
* In {{toResource()}} returning the override if the profile map in empty seems
a nonintuitive choice. Why not return the default profile? In any case, the
javadoc should explain the expected return values for all the special cases.
* {{none}} in {{toResource()}} should be a constant.
* In {{toResource()}} the consecutive _if_s in the _for_ loop can be combined.
Considering that there could be a large number of resource types, it probably
makes more sense to scrap the loop for an _if-memory_ and an _if-cpu_.
{{Resource}}
* In {{newInstance()}} the _try-catch_ doesn't cover all cases.
{{ResourcePBImpl.getResourceInformation()}} throws a
{{ResourceNotFoundException}}, which is not a {{YarnException}}.
{{TestResourceProfiles}}
* In {{testConvertProfileToResourceCapability()}}, the _try_ should start right
before the copy.
> Add support for resource profiles in MapReduce
> --
>
> Key: YARN-6504
> URL: https://issues.apache.org/jira/browse/YARN-6504
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Varun Vasudev
>Assignee: Varun Vasudev
> Attachments: YARN-6504-YARN-3926.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3742) YARN RM will shut down if ZKClient creation times out
[
https://issues.apache.org/jira/browse/YARN-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003625#comment-16003625
]
Karthik Kambatla commented on YARN-3742:
Just committed to trunk and branch-2. Thanks for fixing this and making the
logs actionable, Daniel.
> YARN RM will shut down if ZKClient creation times out
> ---
>
> Key: YARN-3742
> URL: https://issues.apache.org/jira/browse/YARN-3742
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Daniel Templeton
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-3742.001.patch, YARN-3742.002.patch
>
>
> The RM goes down showing the following stacktrace if the ZK client connection
> fails to be created. We should not exit but transition to StandBy and stop
> doing things and let the other RM take over.
> {code}
> 2015-04-19 01:22:20,513 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> java.io.IOException: Wait for ZKClient creation timed out
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1066)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1090)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.existsWithRetries(ZKRMStateStore.java:996)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationStateInternal(ZKRMStateStore.java:643)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:162)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:147)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:806)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:879)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:874)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6435) [ATSv2] Can't retrieve more than 1000 versions of metrics in time series
[
https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003634#comment-16003634
]
Haibo Chen commented on YARN-6435:
--
Thanks for the patch, [~rohithsharma]. The max version is still being set to
DEFAULT_METRICS_MAX_VERSIONS directly, rather than being read from hbaseConf
(i.e. it is still not configurable).
> [ATSv2] Can't retrieve more than 1000 versions of metrics in time series
>
>
> Key: YARN-6435
> URL: https://issues.apache.org/jira/browse/YARN-6435
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Critical
> Attachments: YARN-6435.0001.patch, YARN-6435.YARN-5355.0001.patch
>
>
> It is observed that, even though *metricslimit* is set to 1500, maximum
> number of metrics values retrieved is 1000.
> This is due to, while creating EntityTable, metrics column family max version
> is specified as 1000 which is hardcoded in
> {{EntityTable#DEFAULT_METRICS_MAX_VERSIONS}}. So, HBase will return max
> version with following {{MIN(cf max version , user provided max version)}}.
> This behavior is contradicting the documentation which claims that
> {code}
> metricslimit - If specified, defines the number of metrics to return.
> Considered only if fields contains METRICS/ALL or metricstoretrieve is
> specified. Ignored otherwise. The maximum possible value for metricslimit can
> be maximum value of Integer. If it is not specified or has a value less than
> 1, and metrics have to be retrieved, then metricslimit will be considered as
> 1 i.e. latest single value of metric(s) will be returned.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[ https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003636#comment-16003636 ] Billie Rinaldi commented on YARN-6533: -- Thanks for the review, [~jianhe]. I am attaching a new patch that allows the registerComponent method to be called, but removes the initial registration of the service record. (After thinking about it some more, I don't think any providers will want this partial record to be written.) I also found a potentially bad bug where the YARN_ID was not being encoded when the service record was updated. > Race condition in writing service record to registry in yarn native services > > > Key: YARN-6533 > URL: https://issues.apache.org/jira/browse/YARN-6533 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Attachments: YARN-6533-yarn-native-services.001.patch, > YARN-6533-yarn-native-services.002.patch > > > The ServiceRecord is written twice, once when the container is initially > registered and again in the Docker provider once the IP has been obtained for > the container. These occur asynchronously, so the more important record (the > one with the IP) can be overwritten by the initial record. Only one record > needs to be written, so we can stop writing the initial record when the > Docker provider is being used. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6380) FSAppAttempt keeps redundant copy of the queue
[
https://issues.apache.org/jira/browse/YARN-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated YARN-6380:
---
Attachment: YARN-6380.004.patch
Updated the patch to clean up the {{FSAppAttempt.getQueue()}} method.
> FSAppAttempt keeps redundant copy of the queue
> --
>
> Key: YARN-6380
> URL: https://issues.apache.org/jira/browse/YARN-6380
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-6380.001.patch, YARN-6380.002.patch,
> YARN-6380.003.patch, YARN-6380.004.patch
>
>
> The {{FSAppAttempt}} class defines its own {{fsQueue}} variable that is a
> second copy of the {{SchedulerApplicationAttempt}}'s {{queue}} variable.
> Aside from being redundant, it's also a bug, because when moving
> applications, we only update the {{SchedulerApplicationAttempt}}'s {{queue}},
> not the {{FSAppAttempt}}'s {{fsQueue}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6380) FSAppAttempt keeps redundant copy of the queue
[
https://issues.apache.org/jira/browse/YARN-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003641#comment-16003641
]
Karthik Kambatla commented on YARN-6380:
+1, pending Jenkins.
> FSAppAttempt keeps redundant copy of the queue
> --
>
> Key: YARN-6380
> URL: https://issues.apache.org/jira/browse/YARN-6380
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-6380.001.patch, YARN-6380.002.patch,
> YARN-6380.003.patch, YARN-6380.004.patch
>
>
> The {{FSAppAttempt}} class defines its own {{fsQueue}} variable that is a
> second copy of the {{SchedulerApplicationAttempt}}'s {{queue}} variable.
> Aside from being redundant, it's also a bug, because when moving
> applications, we only update the {{SchedulerApplicationAttempt}}'s {{queue}},
> not the {{FSAppAttempt}}'s {{fsQueue}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3742) YARN RM will shut down if ZKClient creation times out
[
https://issues.apache.org/jira/browse/YARN-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003652#comment-16003652
]
Hudson commented on YARN-3742:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11708 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11708/])
YARN-3742. YARN RM will shut down if ZKClient creation times out. (kasha: rev
166be0ee95d5ef976f074342656b289b41a11ccd)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMCriticalThreadUncaughtExceptionHandler.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestRMFailover.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMFatalEvent.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ActiveStandbyElectorBasedElectorService.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestMemoryRMStateStore.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMFatalEventType.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
> YARN RM will shut down if ZKClient creation times out
> ---
>
> Key: YARN-3742
> URL: https://issues.apache.org/jira/browse/YARN-3742
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Daniel Templeton
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: YARN-3742.001.patch, YARN-3742.002.patch
>
>
> The RM goes down showing the following stacktrace if the ZK client connection
> fails to be created. We should not exit but transition to StandBy and stop
> doing things and let the other RM take over.
> {code}
> 2015-04-19 01:22:20,513 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> java.io.IOException: Wait for ZKClient creation timed out
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1066)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1090)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.existsWithRetries(ZKRMStateStore.java:996)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationStateInternal(ZKRMStateStore.java:643)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:162)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore.java:147)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:806)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:879)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:874)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.T
[jira] [Created] (YARN-6576) Improve Diagonstic by moving Error stack trace from NM to slider AM
Yesha Vora created YARN-6576: Summary: Improve Diagonstic by moving Error stack trace from NM to slider AM Key: YARN-6576 URL: https://issues.apache.org/jira/browse/YARN-6576 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora Slider Master diagonstics should improve to show root cause of App failures for issues like missing docker image. Currently, Slider Master log does not show proper error message to debug such failure. User have to access Nodemanager logs to find out root cause of such issues where container failed to start. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6533) Race condition in writing service record to registry in yarn native services
[
https://issues.apache.org/jira/browse/YARN-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003665#comment-16003665
]
Hadoop QA commented on YARN-6533:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
27s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 1 new + 145 unchanged - 1 fixed = 146 total (was 146)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6533 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867188/YARN-6533-yarn-native-services.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 538d432deb9e 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 3c9f707 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15880/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-slider_hadoop-yarn-slider-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15880/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
|
| Console output |
https://builds
[jira] [Updated] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-5949: Attachment: YARN-5949-YARN-5734.003.patch > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch, YARN-5949-YARN-5734.003.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6544) Add Null check RegistryDNS service while parsing registry records
[ https://issues.apache.org/jira/browse/YARN-6544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003669#comment-16003669 ] Gour Saha commented on YARN-6544: - [~karams] thanks for adding the UT. The patch looks great now. Just fix the 2 whitespace related issues and I will commit the patch. > Add Null check RegistryDNS service while parsing registry records > - > > Key: YARN-6544 > URL: https://issues.apache.org/jira/browse/YARN-6544 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Affects Versions: yarn-native-services >Reporter: Karam Singh >Assignee: Karam Singh > Fix For: yarn-native-services > > Attachments: YARN-6544-yarn-native-services.001.patch, > YARN-6544-yarn-native-services.002.patch, > YARN-6544-yarn-native-services.002.patch, > YARN-6544-yarn-native-services.003.patch > > > Add Null check RegistryDNS service while parsing registry records for Yarn > persistance attribute. > As of now It assumes that yarn registry record always contain yarn > persistance which is not the case -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003668#comment-16003668 ] Jonathan Hung commented on YARN-5949: - Attached 003 patch which pushes the client request -> key value map transformation down from RMWebServices to MutableCSConfigurationProvider, to avoid weird parsing logic. > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch, YARN-5949-YARN-5734.003.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6473) Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations
[ https://issues.apache.org/jira/browse/YARN-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlo Curino updated YARN-6473: --- Attachment: YARN-6473.v2.patch > Create ReservationInvariantChecker to validate ReservationSystem + Scheduler > operations > --- > > Key: YARN-6473 > URL: https://issues.apache.org/jira/browse/YARN-6473 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6473.v0.patch, YARN-6473.v1.patch, > YARN-6473.v2.patch > > > This JIRA tracks an application of YARN-6451 ideas to the ReservationSystem. > It is in particularly useful to create integration tests, or for test > clusters, where we can continuously (and possibly costly) check the > ReservationSystem + Scheduler are operating as expected. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6473) Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations
[ https://issues.apache.org/jira/browse/YARN-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003675#comment-16003675 ] Carlo Curino commented on YARN-6473: I addressed the checkstyle, the findbugs is already present, and the unit tests is flacky (sometime passes sometime doesn't) and it fails in trunk as well: YARN-5240 > Create ReservationInvariantChecker to validate ReservationSystem + Scheduler > operations > --- > > Key: YARN-6473 > URL: https://issues.apache.org/jira/browse/YARN-6473 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6473.v0.patch, YARN-6473.v1.patch, > YARN-6473.v2.patch > > > This JIRA tracks an application of YARN-6451 ideas to the ReservationSystem. > It is in particularly useful to create integration tests, or for test > clusters, where we can continuously (and possibly costly) check the > ReservationSystem + Scheduler are operating as expected. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6504) Add support for resource profiles in MapReduce
[
https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003679#comment-16003679
]
Hadoop QA commented on YARN-6504:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
2s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
35s{color} | {color:green} YARN-3926 passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 7m
8s{color} | {color:red} root in YARN-3926 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
23s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
19s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
4s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
32s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} YARN-3926 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 7m
21s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 7m 21s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 19s{color} | {color:orange} root: The patch generated 19 new + 1044
unchanged - 4 fixed = 1063 total (was 1048) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
0s{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
32s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api
generated 1 new + 123 unchanged - 0 fixed = 124 total (was 123) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
55s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 50s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
12s{color} | {color:green} hadoop-yarn-applications-distributedshell in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
24s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
35s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
56s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
|
| | Invocation of toString on RMContainerRequestor$ContainerRequest.hosts in
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor$ContainerRequest.toString()
At RMContainerRequestor.java:in
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor$ContainerRequest.t
[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues
[
https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003682#comment-16003682
]
Daniel Templeton commented on YARN-6475:
Thanks for working on the patch, [~soumabrata]. Looks generally good. A few
comments:
* In {{buildContainerRuntimeContext()}}, {{runAsUser}} is only used once, so
you may as well inline it like you did with the rest of the attribute values.
* In {{deriveContainerWorkDir()}}, the {{StringBuilder}} is unnecessary. The
compiler will turn + concatenation into a {{StringBuilder}} under the covers.
If you think + concatenation would be easier to read, feel free to use it. Or
indeed, this change may not be necessary.
* In {{prepareContainer()}}, your code: {code}exec.prepareContainer(new
ContainerPrepareContext.Builder()
.setContainer(container)
.setLocalizedResources(localResources)
.setUser(container.getUser())
.setContainerLocalDirs(containerLocalDirs)
.setCommands(container.getLaunchContext()
.getCommands()).build());{code} might be better formatted as: {code}
exec.prepareContainer(new ContainerPrepareContext.Builder()
.setContainer(container)
.setLocalizedResources(localResources)
.setUser(container.getUser())
.setContainerLocalDirs(containerLocalDirs)
.setCommands(container.getLaunchContext().getCommands())
.build());{code}
* In {{StatusUpdaterRunnable}}, we don't need an empty line after the class
declaration.
* In {{StatusUpdaterRunnable}}, it looks like you have several places with
8-space indentation where 4-space should do.
* In {{StatusUpdaterRunnable.run()}}, you may as well replace {{if
(containersToSignal.size() != 0)}} with {{if (!containersToSignal.isEmpty())}}
* In {{StatusUpdaterRunnable.run()}}, you should also fix the {{catch
(Throwable)}}. Either make it a multi-catch instead or just catch
{{Exception}}.
Also, please post a patch from the PR to this JIRA so that the Jenkins
pre-commit has something to chew on.
> Fix some long function checkstyle issues
>
>
> Key: YARN-6475
> URL: https://issues.apache.org/jira/browse/YARN-6475
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Soumabrata Chakraborty
>Priority: Trivial
> Labels: newbie
>
> I am talking about these two:
> {code}
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java:441:
> @Override:3: Method length is 176 lines (max allowed is 150). [MethodLength]
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java:159:
> @Override:3: Method length is 158 lines (max allowed is 150). [MethodLength]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6484) [Documentation] Documenting the YARN Federation feature
[
https://issues.apache.org/jira/browse/YARN-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003683#comment-16003683
]
Hadoop QA commented on YARN-6484:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
9s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6484 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867193/YARN-6484-YARN-2915.v1.patch
|
| Optional Tests | asflicense mvnsite xml |
| uname | Linux a19729e30401 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-2915 / 0a93a32 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/15881/artifact/patchprocess/whitespace-eol.txt
|
| modules | C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
U: . |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15881/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [Documentation] Documenting the YARN Federation feature
> ---
>
> Key: YARN-6484
> URL: https://issues.apache.org/jira/browse/YARN-6484
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Affects Versions: YARN-2915
>Reporter: Subru Krishnan
>Assignee: Carlo Curino
> Attachments: YARN-6484-YARN-2915.v0.patch,
> YARN-6484-YARN-2915.v1.patch
>
>
> We should document the high level design and configuration to enable YARN
> Federation
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6504) Add support for resource profiles in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003702#comment-16003702 ] Daniel Templeton commented on YARN-6504: Forgot to ask about additional tests, too. > Add support for resource profiles in MapReduce > -- > > Key: YARN-6504 > URL: https://issues.apache.org/jira/browse/YARN-6504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: YARN-6504-YARN-3926.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5006) ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk
[
https://issues.apache.org/jira/browse/YARN-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003712#comment-16003712
]
Daniel Templeton commented on YARN-5006:
Thanks for the patch, [~bibinchundatt]. A few comments:
* {{RM_ZK_NUM_ZNODE_SIZE_LIMIT}} would be clearer as
{{RM_ZK_ZNODE_SIZE_LIMIT_BYTES}}. Even better, could we make it generic?
{{RM_APP_DATA_SIZE_LIMIT_BYTES}}
* I don't think the {{isRejectApp()}} method makes it clearer; just inline the
_instanceof_.
* {{StoreLimitException}} needs javadoc to explain what it should be used for.
* {{super("RMStateStore not limit reached");}} could use a clearer message,
like: "Application exceeds the maximum allowed size for application
data. See yarn.resourcemanager.whatever.max-data-size.bytes." Same goes for
the message in {{storeApplicationStateInternal()}}.
* Please add javadoc for the new methods in {{RMAppEvent}}.
* Please add some additional unit tests to cover the new behavior.
> ResourceManager quit due to ApplicationStateData exceed the limit size of
> znode in zk
> --
>
> Key: YARN-5006
> URL: https://issues.apache.org/jira/browse/YARN-5006
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.6.0, 2.7.2
>Reporter: dongtingting
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: YARN-5006.001.patch
>
>
> Client submit a job, this job add 1 file into DistributedCache. when the
> job is submitted, ResourceManager sotre ApplicationStateData into zk.
> ApplicationStateData is exceed the limit size of znode. RM exit 1.
> The related code in RMStateStore.java :
> {code}
> private static class StoreAppTransition
> implements SingleArcTransition {
> @Override
> public void transition(RMStateStore store, RMStateStoreEvent event) {
> if (!(event instanceof RMStateStoreAppEvent)) {
> // should never happen
> LOG.error("Illegal event type: " + event.getClass());
> return;
> }
> ApplicationState appState = ((RMStateStoreAppEvent)
> event).getAppState();
> ApplicationId appId = appState.getAppId();
> ApplicationStateData appStateData = ApplicationStateData
> .newInstance(appState);
> LOG.info("Storing info for app: " + appId);
> try {
> store.storeApplicationStateInternal(appId, appStateData); //store
> the appStateData
> store.notifyApplication(new RMAppEvent(appId,
>RMAppEventType.APP_NEW_SAVED));
> } catch (Exception e) {
> LOG.error("Error storing app: " + appId, e);
> store.notifyStoreOperationFailed(e); //handle fail event, system
> exit
> }
> };
> }
> {code}
> The Exception log:
> {code}
> ...
> 2016-04-20 11:26:35,732 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
> AsyncDispatcher event handler: Maxed out ZK retries. Giving up!
> 2016-04-20 11:26:35,732 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore
> AsyncDispatcher event handler: Error storing app:
> application_1461061795989_17671
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:931)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:936)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:933)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1075)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1096)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:933)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:947)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.createWithRetries(ZKRMStateStore.java:956)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.storeApplicationStateInternal(ZKRMStateStore.java:626)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(RMStateStore.java:138)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(
[jira] [Commented] (YARN-6380) FSAppAttempt keeps redundant copy of the queue
[
https://issues.apache.org/jira/browse/YARN-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003730#comment-16003730
]
Hadoop QA commented on YARN-6380:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 20s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 1 unchanged - 0 fixed = 3 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m
17s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6380 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867196/YARN-6380.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1b9ab9a7f0ca 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 166be0e |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15882/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15882/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15882/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FSAppAttempt keeps redun
[jira] [Commented] (YARN-5841) Report only local collectors on node upon resync with RM after RM fails over
[ https://issues.apache.org/jira/browse/YARN-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003745#comment-16003745 ] Haibo Chen commented on YARN-5841: -- NMs maintains a set of registered collectors that they send to RM during heartbeat. Tracking the code, a collector is added to the set of registered collectors only if an AM container is launched. Thus, NMs are already reporting only local collectors to RM. Unless I am missing something, I think this jira can be closed as not a problem. > Report only local collectors on node upon resync with RM after RM fails over > > > Key: YARN-5841 > URL: https://issues.apache.org/jira/browse/YARN-5841 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena > > As per discussion on YARN-3359, we can potentially optimize reporting of > collectors to RM after RM fails over. > Currently NM would report all the collectors known to itself in HB after > resync with RM. This would mean many NMs' may pretty much report similar set > of collector infos in first NM HB on reconnection. > This JIRA is to explore how to optimize this flow and if possible, fix it. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5413) Create a proxy chain for ResourceManager Admin API in the Router
[ https://issues.apache.org/jira/browse/YARN-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-5413: --- Attachment: YARN-5413-YARN-2915.v3.patch > Create a proxy chain for ResourceManager Admin API in the Router > > > Key: YARN-5413 > URL: https://issues.apache.org/jira/browse/YARN-5413 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5413-YARN-2915.v1.patch, > YARN-5413-YARN-2915.v2.patch, YARN-5413-YARN-2915.v3.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager Admin API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003792#comment-16003792 ] Wangda Tan commented on YARN-2113: -- Thanks [~sunilg], Few comments: 1) INTRAQUEUE_PREEMPTION_ORDER => _POLICY (Since it is not only ordering, but affect can/cannot do preemption). Some internal fields / naming need to be updated as well. 2) Javadocs issues in calculateIdealAssignedResourcePerApp Beyond the two, +1 to latest patch. > Add cross-user preemption within CapacityScheduler's leaf-queue > --- > > Key: YARN-2113 > URL: https://issues.apache.org/jira/browse/YARN-2113 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Vinod Kumar Vavilapalli >Assignee: Sunil G > Attachments: IntraQueue Preemption-Impact Analysis.pdf, > TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt, > YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, > YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch, > YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch, > YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch, > YARN-2113.0013.patch, YARN-2113.0014.patch, > YARN-2113.apply.onto.0012.ericp.patch, YARN-2113 Intra-QueuePreemption > Behavior.pdf, YARN-2113.v0.patch > > > Preemption today only works across queues and moves around resources across > queues per demand and usage. We should also have user-level preemption within > a queue, to balance capacity across users in a predictable manner. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5413) Create a proxy chain for ResourceManager Admin API in the Router
[ https://issues.apache.org/jira/browse/YARN-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003794#comment-16003794 ] Giovanni Matteo Fumarola commented on YARN-5413: Thanks [~subru] for the feedback. I fixed some of the checkstyle warnings and the missing javadocs. > Create a proxy chain for ResourceManager Admin API in the Router > > > Key: YARN-5413 > URL: https://issues.apache.org/jira/browse/YARN-5413 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5413-YARN-2915.v1.patch, > YARN-5413-YARN-2915.v2.patch, YARN-5413-YARN-2915.v3.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager Admin API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003803#comment-16003803 ] Vrushali C commented on YARN-6323: -- bq. However, in the upgrade path, we could choose to not write anything or have a null writer. So while testing on an NM, I was pleasantly surprised to see this message in the NM logs. We do have defensive code right at the writer end so that we don't end up trying to write nulls to the backend as part of row keys. https://github.com/apache/hadoop/blob/YARN-5355/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineWriterImpl.java#L131 That said, the NPE when trying to read previous app state still needs to be fixed. https://github.com/apache/hadoop/blob/YARN-5355/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java#L387 I have patch on that, will upload it shortly. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6518) Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice
[ https://issues.apache.org/jira/browse/YARN-6518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003822#comment-16003822 ] Haibo Chen commented on YARN-6518: -- [~Naganarasimha] Can you please cherry-pick this into YARN-5355 branch as well? > Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice > > > Key: YARN-6518 > URL: https://issues.apache.org/jira/browse/YARN-6518 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Labels: findbugs > Fix For: 3.0.0-alpha3 > > Attachments: YARN-6518.001.patch > > > There is 1 findbugs warning in hadoop-yarn-server-timelineservice since > switched to spotbugs > # Possible null pointer dereference in > org.apache.hadoop.yarn.server.timelineservice.storage.FileSystemTimelineReaderImpl.getEntities(File, > String, TimelineEntityFilters, TimelineDataToRetrieve) due to return value > of called method > See more in > [https://builds.apache.org/job/PreCommit-HADOOP-Build/12157/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-warnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003823#comment-16003823 ] Haibo Chen commented on YARN-6559: -- I don't see YARN-6518 in YARN-5355 branch either. I have asked Naga in YARN-6518 to cherry-pick. Let's wait and see what he says. > Findbugs warning in YARN-5355 branch > > > Key: YARN-6559 > URL: https://issues.apache.org/jira/browse/YARN-6559 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Vrushali C >Priority: Minor > Attachments: FindBugs Report.html, YARN-6559-YARN-5355.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6380) FSAppAttempt keeps redundant copy of the queue
[
https://issues.apache.org/jira/browse/YARN-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003834#comment-16003834
]
Karthik Kambatla commented on YARN-6380:
At least one of the checkstyle issues looks legit. Mind revving the patch?
> FSAppAttempt keeps redundant copy of the queue
> --
>
> Key: YARN-6380
> URL: https://issues.apache.org/jira/browse/YARN-6380
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-6380.001.patch, YARN-6380.002.patch,
> YARN-6380.003.patch, YARN-6380.004.patch
>
>
> The {{FSAppAttempt}} class defines its own {{fsQueue}} variable that is a
> second copy of the {{SchedulerApplicationAttempt}}'s {{queue}} variable.
> Aside from being redundant, it's also a bug, because when moving
> applications, we only update the {{SchedulerApplicationAttempt}}'s {{queue}},
> not the {{FSAppAttempt}}'s {{fsQueue}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[
https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003837#comment-16003837
]
Hadoop QA commented on YARN-5949:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
18s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
56s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s{color} | {color:green} YARN-5734 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
5s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in YARN-5734 has 8 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} YARN-5734 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 8 new + 325 unchanged - 0 fixed = 333 total (was 325) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
30s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager
generated 1 new + 880 unchanged - 0 fixed = 881 total (was 880) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 49s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-5949 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867200/YARN-5949-YARN-5734.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ae5c60170a47 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/prec
[jira] [Updated] (YARN-6160) Create an agent-less docker-less provider in the native services framework
[
https://issues.apache.org/jira/browse/YARN-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billie Rinaldi updated YARN-6160:
-
Attachment: YARN-6160-yarn-native-services.001.patch
Here is a first attempt at enabling different providers based on the artifact
type of the component. One question I have is whether we should add a new
artifact type such as "simple" or "command" that does not even require a
tarball, or whether we should make that a special case of the "tarball"
artifact type (where the id specified has a particular value such as "none" or
"null"). [~jianhe] [~gsaha]
Example json:
{noformat}
{
"name": "test1",
"lifetime": "3600",
"components" :
[
{
"name": "SLEEP",
"number_of_containers": 2,
"artifact": {
"id": "",
"type": "TARBALL"
},
"launch_command": "sleep 3600",
"resource": {
"cpus": 2,
"memory": "1024"
}
},
{
"name": "SLEEP_DOCKER",
"number_of_containers": 2,
"artifact": {
"id": "",
"type": "DOCKER"
},
"launch_command": "sleep 3600",
"resource": {
"cpus": 2,
"memory": "1024"
}
}
]
}
{noformat}
> Create an agent-less docker-less provider in the native services framework
> --
>
> Key: YARN-6160
> URL: https://issues.apache.org/jira/browse/YARN-6160
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
> Fix For: yarn-native-services
>
> Attachments: YARN-6160-yarn-native-services.001.patch
>
>
> The goal of the agent-less docker-less provider is to be able to use the YARN
> native services framework when Docker is not installed or other methods of
> app resource installation are preferable.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6473) Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations
[
https://issues.apache.org/jira/browse/YARN-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003891#comment-16003891
]
Hadoop QA commented on YARN-6473:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
37s{color} | {color:red} hadoop-tools/hadoop-sls in trunk has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m
5s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 58s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.TestSLSRunner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6473 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867201/YARN-6473.v2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b05069e435d5 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 166be0e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15884/artifact/patchprocess/branch-findbugs-hadoop-tools_hadoop-sls-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15884/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15884/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-serve
