Re: [DISCUSS] FLIP-388: Support Dynamic Logger Level Adjustment

[David Morávek]

Hi Yuepeng,

Thanks for the FLIP! There was already quite a discussion on FLIP-210
[1][2], that has proposed more or less the same thing. FLIP was marked as
out of scope for Flink because underlying technologies already address it.

Are you aware of the effort? If yes, what has changed since then?

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-210%3A+Change+logging+level+dynamically+at+runtime
[2] https://lists.apache.org/thread/n8omkpjf1mk9jphx38b8tfrs4h3nxo3z

Best,
D.

On Tue, Jan 16, 2024 at 4:37 PM Yuepeng Pan  wrote:

> Hi all,
>
>
>
>
> I created the FLIP-388[1] to support dynamic logger level
> adjustment.
>
>
>
>
>  Comprehensive and detailed system logs(like debug, trace, all,
> etc.)
>
> could contribute to improved visibility of internal system execution
> information
>
> and also enhance the efficiency of program debugging. Flink currently only
>
> supports static log level configuration(like debug, trace, all, etc.) to
> help application
>
> debugging, which can lead to the following issues when using static log
> configuration:
>
>   1. A sharp increase in log volume, accelerating disk occupancy.
>
>  2. Potential risks of system performance degradation due to a large
> volume of log printing.
>
>  3. The need to simplify log configuration subsequently, which causes
> inevitably cause the program to restart.
>
>
>
>  Therefore, introducing a mechanism to dynamically adjust the online
> log output level
>
> in the event of debugging programs will be meaningful, which can complete
> the switch
>
> of log level configuration without restarting the program.
>
>
>
>
>  I really appreciate Fan Rui(CC'ed), Zhanghao Chen(CC'ed)  for
> providing some valuable help and suggestions.
>
>  Please refer to the FLIP[1] document for more details about the
> proposed design and implementation.
>
> We welcome any feedback and opinions on this proposal.
>
>
>
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-388%3A+Support+Dynamic+Logger+Level+Adjustment
>
> [2] https://issues.apache.org/jira/browse/FLINK-33320
>
>
>
>
> Best,
>
> Yuepeng Pan

[DISCUSS] FLIP-388: Support Dynamic Logger Level Adjustment

[Yuepeng Pan]

Hi all,




I created the FLIP-388[1] to support dynamic logger level adjustment.




 Comprehensive and detailed system logs(like debug, trace, all, etc.) 

could contribute to improved visibility of internal system execution 
information 

and also enhance the efficiency of program debugging. Flink currently only 

supports static log level configuration(like debug, trace, all, etc.) to help 
application 

debugging, which can lead to the following issues when using static log 
configuration:

  1. A sharp increase in log volume, accelerating disk occupancy.

 2. Potential risks of system performance degradation due to a large volume 
of log printing.

 3. The need to simplify log configuration subsequently, which causes 
inevitably cause the program to restart.

 

 Therefore, introducing a mechanism to dynamically adjust the online log 
output level 

in the event of debugging programs will be meaningful, which can complete the 
switch 

of log level configuration without restarting the program. 




 I really appreciate Fan Rui(CC'ed), Zhanghao Chen(CC'ed)  for providing 
some valuable help and suggestions. 

 Please refer to the FLIP[1] document for more details about the proposed 
design and implementation. 

We welcome any feedback and opinions on this proposal.




[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-388%3A+Support+Dynamic+Logger+Level+Adjustment
 

[2] https://issues.apache.org/jira/browse/FLINK-33320




Best,

Yuepeng Pan