[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320677#comment-17320677 ] Qi Zhu commented on YARN-8418: -- [~bibinchundatt] [~rohithsharma] Could you help backport to hadoop 2. Thanks. > App local logs could leaked if log aggregation fails to initialize for the app > -- > > Key: YARN-8418 > URL: https://issues.apache.org/jira/browse/YARN-8418 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin Chundatt >Assignee: Bibin Chundatt >Priority: Critical > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8418.001.patch, YARN-8418.002.patch, > YARN-8418.003.patch, YARN-8418.004.patch, YARN-8418.005.patch, > YARN-8418.006.patch, YARN-8418.007.patch, YARN-8418.008.patch, > YARN-8418.009.patch > > > If log aggregation fails init createApp directory container logs could get > leaked in NM directory > For log running application restart of NM after token renewal this case is > possible/ Application submission with invalid delegation token -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320665#comment-17320665 ] Qi Zhu commented on YARN-10734: --- I find it duplicated with YARN-8418 , so i close it now. Thanks. > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=1121,height=246! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu resolved YARN-10734. --- Resolution: Duplicate > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=1121,height=246! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=1121,height=246! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] was: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=1121,height=246! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7769) FS QueueManager should not create default queue at init
[ https://issues.apache.org/jira/browse/YARN-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320581#comment-17320581 ] Wilfred Spiegelenburg commented on YARN-7769: - These test failures are all related to the change. Please check, the tests assumed a default queue to be available and it no longer is. > FS QueueManager should not create default queue at init > --- > > Key: YARN-7769 > URL: https://issues.apache.org/jira/browse/YARN-7769 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 3.1.0 >Reporter: Wilfred Spiegelenburg >Assignee: Benjamin Teke >Priority: Major > Attachments: YARN-7769.001.patch > > > Currently the FairScheduler QueueManager automatically creates the default > queue. However the default queue does not need to exist. We have two possible > cases which we should handle: > * Based on the placement rule "Default" the name for the default queue might > not be default and it should be created with a different name > * There might not be a "Default" placement rule at all which removes the need > to create the queue. > We should leave the creation of the default queue to the point in time that > we can assess if it is needed or not. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7769) FS QueueManager should not create default queue at init
[
https://issues.apache.org/jira/browse/YARN-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320479#comment-17320479
]
Hadoop QA commented on YARN-7769:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 27m
26s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 6
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
7s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 49s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 19m
53s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
48s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 56s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green}{color} | {color:green} the patch passed with JDK
[jira] [Commented] (YARN-10733) TimelineService Hbase tests are failing with timeout error on branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320278#comment-17320278
]
Jim Brennan commented on YARN-10733:
It looks like the fix is to change hbase-compatible-hadoop.version to 2.7.0
(from 2.5.1).
Seems ok, but I am not sure if it will break anything?
> TimelineService Hbase tests are failing with timeout error on branch-2.10
> -
>
> Key: YARN-10733
> URL: https://issues.apache.org/jira/browse/YARN-10733
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, timelineserver, yarn
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Attachments: 2021-04-12T12-40-21_403-jvmRun1.dump,
> 2021-04-12T12-40-58_857.dumpstream,
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction-output.txt.zip
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:bash}
> 03:54:41 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on
> project hadoop-yarn-server-timelineservice-hbase-tests: There was a timeout
> or other error in the fork -> [Help 1]
> 03:54:41 [ERROR]
> 03:54:41 [ERROR] To see the full stack trace of the errors, re-run Maven with
> the -e switch.
> 03:54:41 [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> 03:54:41 [ERROR]
> 03:54:41 [ERROR] For more information about the errors and possible
> solutions, please read the following articles:
> 03:54:41 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> 03:54:41 [ERROR]
> 03:54:41 [ERROR] After correcting the problems, you can resume the build with
> the command
> 03:54:41 [ERROR] mvn -rf
> :hadoop-yarn-server-timelineservice-hbase-tests
> {code}
> Failure of the tests is due to test unit
> {{TestHBaseStorageFlowRunCompaction}} getting stuck.
> Upon checking the surefire reports, I found several Class no Found Exceptions.
> {code:bash}
> Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/CanUnbuffer
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:66)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:698)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1895)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1009)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2638)
> ... 33 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.CanUnbuffer
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 51 more
> {code}
> and
> {code:bash}
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.regionserver.StoreFileInfo
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:698)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1895)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1009)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2638)
> ... 10 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To
[jira] [Updated] (YARN-7769) FS QueueManager should not create default queue at init
[ https://issues.apache.org/jira/browse/YARN-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Teke updated YARN-7769: Attachment: YARN-7769.001.patch > FS QueueManager should not create default queue at init > --- > > Key: YARN-7769 > URL: https://issues.apache.org/jira/browse/YARN-7769 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 3.1.0 >Reporter: Wilfred Spiegelenburg >Assignee: Benjamin Teke >Priority: Major > Attachments: YARN-7769.001.patch > > > Currently the FairScheduler QueueManager automatically creates the default > queue. However the default queue does not need to exist. We have two possible > cases which we should handle: > * Based on the placement rule "Default" the name for the default queue might > not be default and it should be created with a different name > * There might not be a "Default" placement rule at all which removes the need > to create the queue. > We should leave the creation of the default queue to the point in time that > we can assess if it is needed or not. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9927) RM multi-thread event processing mechanism
[
https://issues.apache.org/jira/browse/YARN-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320065#comment-17320065
]
Hadoop QA commented on YARN-9927:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 29m
1s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 2
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
8s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
2s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
52s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
22m 29s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
3s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 34m
26s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 6m
4s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
14s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
14s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
16s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
16s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
32s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 1s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
|
[jira] [Commented] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320022#comment-17320022
]
Qi Zhu commented on YARN-10648:
---
Thanks [~dmmkr] for good finding.
LGTM +1.
[~brahmareddy]
If we need a unit test also, and it seems a critical problem, we should fix it
soon.
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-10648.001.patch
>
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320016#comment-17320016 ] Qi Zhu edited comment on YARN-10734 at 4/13/21, 8:14 AM: - cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] What's your opinion about this? Thanks. was (Author: zhuqi): cc [~pbacsko] [~gandras] [~ebadger] [~epayne] What's your opinion about this? Thanks. > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] was: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! cc [~pbacsko] [~gandras] [~ebadger] [~epayne] > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~snemeth] [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320016#comment-17320016 ] Qi Zhu commented on YARN-10734: --- cc [~pbacsko] [~gandras] [~ebadger] [~epayne] What's your opinion about this? Thanks. > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! cc [~pbacsko] [~gandras] [~ebadger] [~epayne] was: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > cc [~pbacsko] [~gandras] [~ebadger] [~epayne] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Issue Type: Bug (was: Improvement) > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): !image-2021-04-13-15-39-27-446.png|width=783,height=49! was: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > !image-2021-04-13-15-39-27-446.png|width=783,height=49! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Attachment: image-2021-04-13-15-39-27-446.png > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png, image-2021-04-13-15-39-27-446.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: As follows log aggregation create dir throw failed to setup application log directory : !image-2021-04-13-15-34-22-732.png|width=756,height=166! But actually the log not deleted, i will cause a lot undeleted dirs in large cluster, it will be heavy when the cluster is long running app cluster (such as flink): was: !image-2021-04-13-15-34-22-732.png|width=756,height=166! > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png > > > As follows log aggregation create dir throw failed to setup application log > directory : > !image-2021-04-13-15-34-22-732.png|width=756,height=166! > But actually the log not deleted, i will cause a lot undeleted dirs in large > cluster, it will be heavy when the cluster is long running app cluster (such > as flink): > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: !image-2021-04-13-15-34-22-732.png|width=756,height=166! was:!image-2021-04-13-15-34-22-732.png|width=756,height=166! > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png > > > > !image-2021-04-13-15-34-22-732.png|width=756,height=166! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Summary: Log aggregation create dir throw failed to setup application log directory, but the log dir not deleted. (was: Log aggregation create dir throw failed to setup application log directory, but the dir existed.) > Log aggregation create dir throw failed to setup application log directory, > but the log dir not deleted. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png > > > !image-2021-04-13-15-34-22-732.png|width=756,height=166! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Description: !image-2021-04-13-15-34-22-732.png|width=756,height=166! > Log aggregation create dir throw failed to setup application log directory, > but the dir existed. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png > > > !image-2021-04-13-15-34-22-732.png|width=756,height=166! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Attachment: image-2021-04-13-15-34-22-732.png > Log aggregation create dir throw failed to setup application log directory, > but the dir existed. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png, > image-2021-04-13-15-34-22-732.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.
[ https://issues.apache.org/jira/browse/YARN-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10734: -- Attachment: image-2021-04-13-15-33-06-387.png > Log aggregation create dir throw failed to setup application log directory, > but the dir existed. > > > Key: YARN-10734 > URL: https://issues.apache.org/jira/browse/YARN-10734 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: image-2021-04-13-15-33-06-387.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10734) Log aggregation create dir throw failed to setup application log directory, but the dir existed.
Qi Zhu created YARN-10734: - Summary: Log aggregation create dir throw failed to setup application log directory, but the dir existed. Key: YARN-10734 URL: https://issues.apache.org/jira/browse/YARN-10734 Project: Hadoop YARN Issue Type: Improvement Reporter: Qi Zhu Assignee: Qi Zhu -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10559) Fair sharing intra-queue preemption support in Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319929#comment-17319929 ] VADAGA ANANYO RAO commented on YARN-10559: -- [~epayne], sorry for seeing your comment so late. We have tested this feature with following configs: Queue Properties: 'yarn.scheduler.capacity..ordering-policy': 'fair' Scheduler configurations: 'yarn.resourcemanager.scheduler.monitor.enable': 'true', 'yarn.resourcemanager.scheduler.monitor.policies' : 'org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy', 'yarn.resourcemanager.monitor.capacity.preemption.intra-queue-preemption.enabled': 'true' Post this, we submit job1 from user1 to a leaf queue. When job1 completely uses up the queue capacity, we trigger job2 from user1 to the same leaf queue. We can observe preemption kicking in for job2 from job1. I am not sure of the exact error you are facing. If you can provide some more details of the problems you are facing, I can try and help out with it. Thank you :) > Fair sharing intra-queue preemption support in Capacity Scheduler > - > > Key: YARN-10559 > URL: https://issues.apache.org/jira/browse/YARN-10559 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.1.4 >Reporter: VADAGA ANANYO RAO >Assignee: VADAGA ANANYO RAO >Priority: Major > Attachments: FairOP_preemption-design_doc_v1.pdf, > FairOP_preemption-design_doc_v2.pdf, YARN-10559.0001.patch, > YARN-10559.0002.patch, YARN-10559.0003.patch, YARN-10559.0004.patch, > YARN-10559.0005.patch, YARN-10559.0006.patch, YARN-10559.0007.patch, > YARN-10559.0008.patch, YARN-10559.0009.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Usecase: > Due to the way Capacity Scheduler preemption works, If a single user submits > a large application to a queue (using 100% of resources), that job will not > be preempted by future applications from the same user within the same queue. > This implies that the later applications will be forced to wait for > completion of the long running application. This prevents multiple long > running, large, applications from running concurrently. > Support fair sharing among apps while preempting applications from same queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
