Hi Rui,
Thanks for the proposal.
The proposed solution of supporting System out and err to be redirected to LOG
or discarded and introducing an enum and two options to manage this, seems
reasonable.
+1
Thanks,
Archit Goyal
From: Piotr Nowojski <pnowoj...@apache.org>
Date: Wednesday, November 8, 2023 at 7:38 AM
To: dev@flink.apache.org <dev@flink.apache.org>
Subject: Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to
LOG or discarded
Hi Rui,
Thanks for the proposal.
+1 I don't have any major comments :)
One nit. In `SystemOutRedirectToLog` in this code:
System.arraycopy(buf, count - LINE_SEPARATOR_LENGTH, bytes, 0,
LINE_SEPARATOR_LENGTH);
return Arrays.equals(LINE_SEPARATOR_BYTES, bytes)
Is there a reason why you are suggesting to copy out bytes from `buf` to
`bytes`,
instead of using `Arrays.equals(int[] a, int aFromIndex, int aToIndex,
int[] b, int bFromIndex, int bToIndex)`?
Best,
Piotrek
śr., 8 lis 2023 o 11:53 Rui Fan <1996fan...@gmail.com> napisał(a):
> Hi all!
>
> I would like to start a discussion of FLIP-390: Support System out and err
> to be redirected to LOG or discarded[1].
>
> In various production environments, either cloud native or physical
> machines, the disk space that Flink TaskManager can use is limited.
>
> In general, the flink users shouldn't use the `System.out.println` in
> production,
> however this may happen when the number of Flink jobs and job developers
> is very large. Flink job may use System.out to output a large amount of
> data
> to the taskmanager.out file. This file will not roll, it will always
> increment.
> Eventually the upper limit of what the TM can be used for is reached.
>
> We can support System out and err to be redirected to LOG or discarded,
> the LOG can roll and won't increment forever.
>
> This feature is useful for SREs who maintain Flink environments, they can
> redirect System.out to LOG by default. Although the cause of this problem
> is
> that the user's code is not standardized, for SRE, pushing users to modify
> the code one by one is usually a very time-consuming operation. It's also
> useful for job stability where System.out is accidentally used.
>
> Looking forward to your feedback, thanks~
>
> [1]
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fx%2F4guZE&data=05%7C01%7Cargoyal%40linkedin.com%7C937821de7bd846e6b97408dbe070beae%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638350547072823674%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zEv6B0Xiq2SNuU6Fm%2BAXnH%2BRILbm6Q0uDRbN7h6iaPM%3D&reserved=0<https://cwiki.apache.org/confluence/x/4guZE>
>
> Best,
> Rui
>
