[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438239#comment-16438239 ] Hudson commented on YARN-8156: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13997 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13997/]) YARN-8156. Increase the default value of (rohithsharmaks: rev 669eb7bdea34f26e9b9b8a2260ae4356791622e7) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestPerNodeTimelineCollectorsAuxService.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438233#comment-16438233 ] Rohith Sharma K S commented on YARN-8156: - committing shortly > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[
https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436931#comment-16436931
]
genericqa commented on YARN-8156:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 31m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 56s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
12s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
3s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}113m 51s{color} |
{color:black} {color} |
\\
\\
||
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436890#comment-16436890 ] Rohith Sharma K S commented on YARN-8156: - +lgtm, pending jenkins > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[
https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436887#comment-16436887
]
Rohith Sharma K S commented on YARN-8156:
-
{quote}Does the default setting increase to 60 second in the patch not help?
{quote}
I think its better to set for 1sec because test case might wait for 60 seconds
and gets timeout or takes a minute to complete. Setting to1sec will make test
to pass quickly.
> Increase the default value of
> yarn.timeline-service.app-collector.linger-period.ms
> --
>
> Key: YARN-8156
> URL: https://issues.apache.org/jira/browse/YARN-8156
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Charan Hebri
>Priority: Major
> Attachments: YARN-8156.001.patch, YARN-8156.002.patch
>
>
> The default value of *yarn.timeline-service.app-collector.linger-period.ms*
> is 1 sec which is very low. In production cluster, we observe that most of
> the entity information is not published because of 1sec linger period.
> I would suggest to increase the default value to either 10sec or 30sec or 60
> sec. As far as I have seen, 10sec should be sufficient since node heartbeat
> period is 3 sec by default, all NM gets request to stop the application.
> cc:/ [~haibo.chen] [~vrushalic]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436835#comment-16436835 ] Vrushali C commented on YARN-8156: -- v002 looks good to me. One question, in the test case, do you require (re)setting it to 1 second? Does the default setting increase to 60 second in the patch not help? > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436777#comment-16436777 ] Charan Hebri commented on YARN-8156: [~rohithsharma] attached patch with 60 secs as the default. > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch, YARN-8156.002.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436044#comment-16436044 ] Vrushali C commented on YARN-8156: -- Also, [~rohithsharma] and I discussed this and he is also good with increasing to 60 seconds. > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[ https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436043#comment-16436043 ] Vrushali C commented on YARN-8156: -- Thanks [~charanh] for the patch! It looks good, but I think let's increase the default interval to 60 seconds. It is configurable anyways, so should not be an issue that it might appear too big to some. In the default config case, it will then take care of most situations of race conditions of entities coming in after the collector has been removed. > Increase the default value of > yarn.timeline-service.app-collector.linger-period.ms > -- > > Key: YARN-8156 > URL: https://issues.apache.org/jira/browse/YARN-8156 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Charan Hebri >Priority: Major > Attachments: YARN-8156.001.patch > > > The default value of *yarn.timeline-service.app-collector.linger-period.ms* > is 1 sec which is very low. In production cluster, we observe that most of > the entity information is not published because of 1sec linger period. > I would suggest to increase the default value to either 10sec or 30sec or 60 > sec. As far as I have seen, 10sec should be sufficient since node heartbeat > period is 3 sec by default, all NM gets request to stop the application. > cc:/ [~haibo.chen] [~vrushalic] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8156) Increase the default value of yarn.timeline-service.app-collector.linger-period.ms
[
https://issues.apache.org/jira/browse/YARN-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435330#comment-16435330
]
genericqa commented on YARN-8156:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 10s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 213 unchanged - 0 fixed = 214 total (was 213) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 51s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
21s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
1s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
