[jira] [Commented] (FLINK-23564) Make taskmanager.out and taskmanager.err rollable
[ https://issues.apache.org/jira/browse/FLINK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393778#comment-17393778 ] zlzhang0122 commented on FLINK-23564: - [~wangyang0918] Maybe using log4j2-iostreams is a better choice to implemention, but it doesn't flush the buffer and would cause OutOfMemory error when using print, println didn't have this problem. I can't solve it and my implemention is mostly like yours. So far I didn't find any problem what other devs are concerned about. > Make taskmanager.out and taskmanager.err rollable > - > > Key: FLINK-23564 > URL: https://issues.apache.org/jira/browse/FLINK-23564 > Project: Flink > Issue Type: Improvement >Affects Versions: 1.13.1 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > Now users can use > System.out.print/System.out.println/System.err.print/System.err.println/e.printStackTrace > to taskmanager.out and taskmanager.err as much as they want and this may use > large space of disk cause the disk problem and influence the checkpoint of > the flink and even the stability of the flink or other application on the > same node. I proposed that we can make the taskmanager.out and > taskmanager.err rollable just like taskmanager.log.By doing this, the disk > consume of the taskmanager.out and taskmanager.err can be control. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-23564) Make taskmanager.out and taskmanager.err rollable
[ https://issues.apache.org/jira/browse/FLINK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392258#comment-17392258 ] Yang Wang commented on FLINK-23564: --- I used to propose a similar solution in this PR[1]. But other devs are quite concerned about stream redirections[2]. [1]. https://github.com/apache/flink/commit/2c1df7529825b6a0af7e57d29febeade36a758b1#diff-d91a03b520137b4a6b44993a960eb4cd4a7e0a92eba9de8aa3863ece9217fa8f [2]. https://github.com/apache/flink/pull/11839#pullrequestreview-399769862 > Make taskmanager.out and taskmanager.err rollable > - > > Key: FLINK-23564 > URL: https://issues.apache.org/jira/browse/FLINK-23564 > Project: Flink > Issue Type: Improvement >Affects Versions: 1.13.1 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > Now users can use > System.out.print/System.out.println/System.err.print/System.err.println/e.printStackTrace > to taskmanager.out and taskmanager.err as much as they want and this may use > large space of disk cause the disk problem and influence the checkpoint of > the flink and even the stability of the flink or other application on the > same node. I proposed that we can make the taskmanager.out and > taskmanager.err rollable just like taskmanager.log.By doing this, the disk > consume of the taskmanager.out and taskmanager.err can be control. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-23564) Make taskmanager.out and taskmanager.err rollable
[ https://issues.apache.org/jira/browse/FLINK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391883#comment-17391883 ] zlzhang0122 commented on FLINK-23564: - [~Jiangang] It's easy to implemention if we redirect the .out and .err to the log and make the log rollable, we have implemention this in our production environment, and it's running well. > Make taskmanager.out and taskmanager.err rollable > - > > Key: FLINK-23564 > URL: https://issues.apache.org/jira/browse/FLINK-23564 > Project: Flink > Issue Type: Improvement >Affects Versions: 1.13.1 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > Now users can use > System.out.print/System.out.println/System.err.print/System.err.println/e.printStackTrace > to taskmanager.out and taskmanager.err as much as they want and this may use > large space of disk cause the disk problem and influence the checkpoint of > the flink and even the stability of the flink or other application on the > same node. I proposed that we can make the taskmanager.out and > taskmanager.err rollable just like taskmanager.log.By doing this, the disk > consume of the taskmanager.out and taskmanager.err can be control. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-23564) Make taskmanager.out and taskmanager.err rollable
[ https://issues.apache.org/jira/browse/FLINK-23564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391414#comment-17391414 ] Liu commented on FLINK-23564: - The log can be rolled by log4j config. I am wondering how the .out and .err can be rolled? > Make taskmanager.out and taskmanager.err rollable > - > > Key: FLINK-23564 > URL: https://issues.apache.org/jira/browse/FLINK-23564 > Project: Flink > Issue Type: Improvement >Affects Versions: 1.13.1 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > Now users can use > System.out.print/System.out.println/System.err.print/System.err.println/e.printStackTraceto > taskmanager.out and taskmanager.err as much as they want and this may use > large space of disk cause the disk problem and influence the checkpoint of > the flink and even the stability of the flink or other application on the > same node. I proposed that we can make the taskmanager.out and > taskmanager.err rollable just like taskmanager.log.By doing this, the disk > consume of the taskmanager.out and taskmanager.err can be control. -- This message was sent by Atlassian Jira (v8.3.4#803005)
