[jira] [Updated] (YARN-10485) TimelineConnector swallows InterruptedException
[
https://issues.apache.org/jira/browse/YARN-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Brennan updated YARN-10485:
---
Fix Version/s: 3.2.3
3.4.1
3.1.5
3.3.1
> TimelineConnector swallows InterruptedException
> ---
>
> Key: YARN-10485
> URL: https://issues.apache.org/jira/browse/YARN-10485
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Fix For: 3.3.1, 3.1.5, 3.4.1, 3.2.3
>
>
> Some tests timeout or take excessively long to shutdown because the
> {{TimelineConnector}} will catch InterruptedException and go into a retry
> loop instead of aborting.
> [~daryn] reported that this makes debugging more difficult and he suggests
> the exception to be thrown.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10485) TimelineConnector swallows InterruptedException
[
https://issues.apache.org/jira/browse/YARN-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Brennan resolved YARN-10485.
Resolution: Fixed
> TimelineConnector swallows InterruptedException
> ---
>
> Key: YARN-10485
> URL: https://issues.apache.org/jira/browse/YARN-10485
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Fix For: 3.3.1, 3.1.5, 3.4.1, 3.2.3
>
>
> Some tests timeout or take excessively long to shutdown because the
> {{TimelineConnector}} will catch InterruptedException and go into a retry
> loop instead of aborting.
> [~daryn] reported that this makes debugging more difficult and he suggests
> the exception to be thrown.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10486) FS-CS converter: handle case when weight=0
[
https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231799#comment-17231799
]
Hadoop QA commented on YARN-10486:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | | {color:green} The patch appears to include 3 new or modified
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
4s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 7s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
46s{color} | | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
47s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 24s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} unit {color} | {colo
[jira] [Commented] (YARN-8558) NM recovery level db not cleaned up properly on container finish
[
https://issues.apache.org/jira/browse/YARN-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231778#comment-17231778
]
Jim Brennan commented on YARN-8558:
---
Any objection to pulling this back to branch-2.10? It looks like
{{remove_container()}} is missing
{noformat}
CONTAINER_START_TIME_KEY_SUFFIX
CONTAINER_VERSION_KEY_SUFFIX
CONTAINER_REMAIN_RETRIES_KEY_SUFFIX
CONTAINER_WORK_DIR_KEY_SUFFIX
CONTAINER_LOG_DIR_KEY_SUFFIX
{noformat}
> NM recovery level db not cleaned up properly on container finish
>
>
> Key: YARN-8558
> URL: https://issues.apache.org/jira/browse/YARN-8558
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Bibin Chundatt
>Assignee: Bibin Chundatt
>Priority: Critical
> Fix For: 3.2.0, 3.1.1, 3.0.4
>
> Attachments: YARN-8558-branch-3.0.002.patch,
> YARN-8558-branch-3.0.003.patch, YARN-8558.001.patch, YARN-8558.002.patch
>
>
> {code}
> 2018-07-20 16:49:23,117 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl:
> Application application_1531994217928_0054 transitioned from NEW to INITING
> 2018-07-20 16:49:23,204 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_18 with incomplete
> records
> 2018-07-20 16:49:23,204 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_19 with incomplete
> records
> 2018-07-20 16:49:23,204 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_20 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_21 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_22 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_23 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_24 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_25 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_38 with incomplete
> records
> 2018-07-20 16:49:23,205 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_39 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_41 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_44 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_46 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_49 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_52 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_54 with incomplete
> records
> 2018-07-20 16:49:23,206 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_73 with incomplete
> records
> 2018-07-20 16:49:23,207 WARN
> org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService:
> Remove container container_1531994217928_0001_01_74 with incomplete
> records
> 2018-07-20 16:49:23,207 WARN
> org.apache.hadoop
[jira] [Updated] (YARN-10489) Yarn Resource Manager scheduler page gives 503 with FIFO Yarn Scheduler
[ https://issues.apache.org/jira/browse/YARN-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Animesh Nandanwar updated YARN-10489: - Description: When Fifo Yarn Scheduler is configured, the scheduler page gives out 503 Exception: 2020-11-05 02:55:04,940 WARN org.apache.hadoop.ipc.Server: IPC Server handler 17 on 8032, call Call#347 Retry#0 org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from 10.115.40.4:44072 java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getQueueInfo(ClientRMService.java:993) at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getQueueInfo(ApplicationClientProtocolPBServiceImpl.java:308) at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:527) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:503) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)" was: When Fifo Yarn Scheduler is configured, the scheduler page gives out 404. > Yarn Resource Manager scheduler page gives 503 with FIFO Yarn Scheduler > --- > > Key: YARN-10489 > URL: https://issues.apache.org/jira/browse/YARN-10489 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Animesh Nandanwar >Priority: Major > > When Fifo Yarn Scheduler is configured, the scheduler page gives out 503 > Exception: > 2020-11-05 02:55:04,940 WARN org.apache.hadoop.ipc.Server: IPC Server handler > 17 on 8032, call Call#347 Retry#0 > org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getQueueInfo from > 10.115.40.4:44072 > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getQueueInfo(ClientRMService.java:993) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getQueueInfo(ApplicationClientProtocolPBServiceImpl.java:308) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:527) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:503) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)" > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10489) Yarn Resource Manager scheduler page gives 503 with FIFO Yarn Scheduler
[ https://issues.apache.org/jira/browse/YARN-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Animesh Nandanwar updated YARN-10489: - Summary: Yarn Resource Manager scheduler page gives 503 with FIFO Yarn Scheduler (was: Yarn Resource Manager scheduler page gives 404 with FIFO Yarn Scheduler) > Yarn Resource Manager scheduler page gives 503 with FIFO Yarn Scheduler > --- > > Key: YARN-10489 > URL: https://issues.apache.org/jira/browse/YARN-10489 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Animesh Nandanwar >Priority: Major > > When Fifo Yarn Scheduler is configured, the scheduler page gives out 404. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10489) Yarn Resource Manager scheduler page gives 404 with FIFO Yarn Scheduler
Animesh Nandanwar created YARN-10489: Summary: Yarn Resource Manager scheduler page gives 404 with FIFO Yarn Scheduler Key: YARN-10489 URL: https://issues.apache.org/jira/browse/YARN-10489 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Animesh Nandanwar When Fifo Yarn Scheduler is configured, the scheduler page gives out 404. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10486) FS-CS converter: handle case when weight=0
[ https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10486: Attachment: YARN-10486-004-approach2.patch > FS-CS converter: handle case when weight=0 > -- > > Key: YARN-10486 > URL: https://issues.apache.org/jira/browse/YARN-10486 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: fs2cs > Attachments: YARN-10486-001.patch, YARN-10486-002.patch, > YARN-10486-003-approach2.patch, YARN-10486-004-approach2.patch > > > We can encounter an ArithmeticException if there is a single or multiple > queues under a parent with zero weight. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7200) SLS generates a realtimetrack.json file but that file is missing the closing ']'
[
https://issues.apache.org/jira/browse/YARN-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231658#comment-17231658
]
Szilard Nemeth commented on YARN-7200:
--
Hi [~akshink],
Checked the call hierarchy of
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#tearDown, and to me it
looks good because it is only getting invoked from these 2 methods:
{code:java}
SLSFairScheduler.serviceStop() (org.apache.hadoop.yarn.sls.scheduler)
SLSCapacityScheduler.serviceStop() (org.apache.hadoop.yarn.sls.scheduler)
{code}
So, the array closing character is only placed when the scheduler (instance of
AbstractService) is getting stopped, meaning that the serviceStop method is
invoked.
*Also played around with your patch with the following way:*
1. Modified the parameterized class called TestSLSStreamAMSynth to only
execute tests with CS.
2. Added some printouts to SchedulerMetrics.
3. Launched
org.apache.hadoop.yarn.sls.TestSLSStreamAMSynth#testSimulatorRunning.
See the attached patch file and test logs in the zip for details.
I can only see "***teardown: Closing array in json" in the test log output,
which is coming from
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#tearDown.
I also added a log string to
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#closeMetricsWriter, with
String: "**closeMetricsWriter: Closing array in json", but I can't see it in
the output, so this is suspicious.
Have you been able to reproduce an issue with the original code, so that the
array-closing character is misplaced in the realtimetrack.json?
If you check the attachment called 'realtimetrack-intellij-debugging.json' (in
the zip), you can see that the array-closing char is misplaced, but this only
happened when I was debugging in intellij so I wouldn't count it as an issue.
*Could you please justify the following?*
*1. Why the method called
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#closeMetricsWriter is not
getting called from this testcase?*
What was your testing methodology?
Maybe the testcase is different than regular launch of SLS from the CLI, but I
think it should invoke this method, anyway.
*2. What was the issue with the
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#tearDown method?*
I think it's a good place to close the array in the json file.
Considering the method calls to
org.apache.hadoop.yarn.sls.scheduler.SchedulerMetrics#metricsLogBW in
SchedulerMetrics, I think adding the array-closing should be in one centralized
place, like it has been.
However, there can be situations where someone kills the JVM process and the
regular serviceStop methods won't run for some services, so you could add a
shutdown hook to cover such a case, but I think it would be an overkill.
[~pbacsko], [~shuzirra] Please chime in.
> SLS generates a realtimetrack.json file but that file is missing the closing
> ']'
>
>
> Key: YARN-7200
> URL: https://issues.apache.org/jira/browse/YARN-7200
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Reporter: Grant Sohn
>Assignee: Agshin Kazimli
>Priority: Minor
> Labels: newbie, newbie++
> Attachments: YARN-7200-branch-trunk.patch, realtimetrack.json,
> snemeth-testing-20201113.zip
>
>
> File
> hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
> shows:
> {noformat}
> void tearDown() throws Exception {
> if (metricsLogBW != null) {
> metricsLogBW.write("]");
> metricsLogBW.close();
> }
>
> {noformat}
> So the exit logic is flawed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7200) SLS generates a realtimetrack.json file but that file is missing the closing ']'
[
https://issues.apache.org/jira/browse/YARN-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-7200:
-
Attachment: snemeth-testing-20201113.zip
> SLS generates a realtimetrack.json file but that file is missing the closing
> ']'
>
>
> Key: YARN-7200
> URL: https://issues.apache.org/jira/browse/YARN-7200
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Reporter: Grant Sohn
>Assignee: Agshin Kazimli
>Priority: Minor
> Labels: newbie, newbie++
> Attachments: YARN-7200-branch-trunk.patch, realtimetrack.json,
> snemeth-testing-20201113.zip
>
>
> File
> hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
> shows:
> {noformat}
> void tearDown() throws Exception {
> if (metricsLogBW != null) {
> metricsLogBW.write("]");
> metricsLogBW.close();
> }
>
> {noformat}
> So the exit logic is flawed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7200) SLS generates a realtimetrack.json file but that file is missing the closing ']'
[
https://issues.apache.org/jira/browse/YARN-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231602#comment-17231602
]
Hadoop QA commented on YARN-7200:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | | {color:red} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
39s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 57s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
49s{color} | | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} |
[/results-checkstyle-hadoop-tools_hadoop-sls.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/297/artifact/out/results-checkstyle-hadoop-tools_hadoop-sls.txt]
| {color:orange} hadoop-tools/hadoop-sls: The patch generated 5 new + 12
unchanged - 0 fixed = 17 total (was 12) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 50s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color}
[jira] [Commented] (YARN-10486) FS-CS converter: handle case when weight=0
[
https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231573#comment-17231573
]
Hadoop QA commented on YARN-10486:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
45s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | | {color:green} The patch appears to include 3 new or modified
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
5s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 25s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | | {color:green} trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | | {color:green} trunk passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
48s{color} | | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | | {color:green} the patch passed with JDK Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 31s{color} |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/296/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt]
| {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 78 unchanged - 0 fixed = 81 total (was 78) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 39s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | | {color:green} the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 {color} |
| {color
[jira] [Commented] (YARN-7200) SLS generates a realtimetrack.json file but that file is missing the closing ']'
[
https://issues.apache.org/jira/browse/YARN-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231550#comment-17231550
]
Szilard Nemeth commented on YARN-7200:
--
Hi [~akshink],
Checking your patch soon.
As a general rule of thumb, the filename of patches you attach to the jiras
should look like: "YARN--.patch, so for trunk you don't need to
include the branch name in the filename as it's the default branch.
An example can be checked on YARN-10458.
Another important thing is to set the status of the Jira to Submit patch
(should only be done once), so that Yetus can pick the change up and trigger a
Jenkins build.
Later on, when you upload more patches you don't need to adjust the status of
the jira, just upload the attachments with the new version number (e.g. 002) in
the filename and Yetus will trigger a build again.
This process should be automatic, however sometimes it doesn't work. In this
case, committers can trigger the Jenkins build manually for any jira.
Please also note that the build can only run for the patch file that was
uploaded the latest.
> SLS generates a realtimetrack.json file but that file is missing the closing
> ']'
>
>
> Key: YARN-7200
> URL: https://issues.apache.org/jira/browse/YARN-7200
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Reporter: Grant Sohn
>Assignee: Agshin Kazimli
>Priority: Minor
> Labels: newbie, newbie++
> Attachments: YARN-7200-branch-trunk.patch, realtimetrack.json
>
>
> File
> hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
> shows:
> {noformat}
> void tearDown() throws Exception {
> if (metricsLogBW != null) {
> metricsLogBW.write("]");
> metricsLogBW.close();
> }
>
> {noformat}
> So the exit logic is flawed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10486) FS-CS converter: handle case when weight=0
[ https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10486: Attachment: YARN-10486-003-approach2.patch > FS-CS converter: handle case when weight=0 > -- > > Key: YARN-10486 > URL: https://issues.apache.org/jira/browse/YARN-10486 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: fs2cs > Attachments: YARN-10486-001.patch, YARN-10486-002.patch, > YARN-10486-003-approach2.patch > > > We can encounter an ArithmeticException if there is a single or multiple > queues under a parent with zero weight. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10486) FS-CS converter: handle case when weight=0
[ https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10486: Attachment: (was: YARN-10486-003-approach2.patch) > FS-CS converter: handle case when weight=0 > -- > > Key: YARN-10486 > URL: https://issues.apache.org/jira/browse/YARN-10486 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: fs2cs > Attachments: YARN-10486-001.patch, YARN-10486-002.patch > > > We can encounter an ArithmeticException if there is a single or multiple > queues under a parent with zero weight. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10486) FS-CS converter: handle case when weight=0
[ https://issues.apache.org/jira/browse/YARN-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10486: Attachment: YARN-10486-003-approach2.patch > FS-CS converter: handle case when weight=0 > -- > > Key: YARN-10486 > URL: https://issues.apache.org/jira/browse/YARN-10486 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: fs2cs > Attachments: YARN-10486-001.patch, YARN-10486-002.patch > > > We can encounter an ArithmeticException if there is a single or multiple > queues under a parent with zero weight. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10488) Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair
[ https://issues.apache.org/jira/browse/YARN-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10488: -- Assignee: Ankit Kumar > Several typos in package: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair > -- > > Key: YARN-10488 > URL: https://issues.apache.org/jira/browse/YARN-10488 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Ankit Kumar >Priority: Minor > Labels: newbie, newbie++ > > 1. Typo in field name: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.VisitedResourceRequestTracker.TrackerPerPriorityResource#racksVisted > 2. Typo in method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager#setChildResourceLimits > There's a comment: "... max reource ...", typo in the word 'resource'. > 3. Typo in javadoc of method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#reserve > "bookeeping" -> "bookkeeping" > 4. There's a local variable in this method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#updateAMDiagnosticMsg, > > called diagnosticMessageBldr. It's an abbreviation, but could be changed to > something more meaningful. > 5. Typo in javadoc of method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.MaxRunningAppsEnforcer#updateRunnabilityOnReload > "reinitilized" --> "reinitialized" > 6. And last but not least, a funny typo in the method name of: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy.DominantResourceFairnessComparator#compareAttribrutes -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10488) Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair
Szilard Nemeth created YARN-10488: - Summary: Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair Key: YARN-10488 URL: https://issues.apache.org/jira/browse/YARN-10488 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth 1. Typo in field name: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.VisitedResourceRequestTracker.TrackerPerPriorityResource#racksVisted 2. Typo in method: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager#setChildResourceLimits There's a comment: "... max reource ...", typo in the word 'resource'. 3. Typo in javadoc of method: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#reserve "bookeeping" -> "bookkeeping" 4. There's a local variable in this method: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#updateAMDiagnosticMsg, called diagnosticMessageBldr. It's an abbreviation, but could be changed to something more meaningful. 5. Typo in javadoc of method: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.MaxRunningAppsEnforcer#updateRunnabilityOnReload "reinitilized" --> "reinitialized" 6. And last but not least, a funny typo in the method name of: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy.DominantResourceFairnessComparator#compareAttribrutes -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
