Thank you Raymond for sharing the inputs so looks like this is a bottleneck from logback especially with k8s ConfigMap.
Is there any workaround (apart from restarting the java process) to address this issue ? OR I'll have to wait until it gets fixed from logback by considering checksum per below suggestion. Hello Ceki, please share your advice if you have any. Thanks, Kedar. On Fri, Aug 20, 2021 at 7:06 AM <[email protected]> wrote: > Send logback-user mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.qos.ch/mailman/listinfo/logback-user > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of logback-user digest..." > > > Today's Topics: > > 1. Re: logback autoscan is not working intermittently in k8s pod > (Raymond Augé) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 20 Aug 2021 10:05:22 -0400 > From: Raymond Augé <[email protected]> > To: logback users list <[email protected]> > Subject: Re: [logback-user] logback autoscan is not working > intermittently in k8s pod > Message-ID: > < > camm6hcc+fkzu4ux-z-wpdqk+lgwt_shwaevb7otr3j5dje-...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Here's a direct link [1] to the explanation of the issue which I confirmed > myself to be the actual issue I was facing. > > Maybe it will help both the OP and logback :) > > [1] https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/ > > > On Fri, Aug 20, 2021 at 10:01 AM Raymond Augé <[email protected]> > wrote: > > > Hi, a long time lurker here 🙂. > > > > So, I suspect the issue is with k8s and it's volumes for config > > maps/secrets symlinks are a pain point in Java. Even inotify based > > WachService cannot handle them. > > > > The issue is that logback is only using a timestamp to detect > > configuration file changes, which is fine as we know in 99% of Java > usages. > > > > However, this doesn't seem to work with the config map volumes supplied > in > > a k8s env because they use symlinks which don't cause the timestamp to > > change, I agree it's weird and unexpected. > > > > However, supplying configuration files via K8s config maps is going to > > start being a much more common use case as time passes. > > > > Logback should probably start including a checksum to detect if the > > contents of the file have changed and use a logical OR with the timestamp > > check so that either changes causes a reload. > > > > FYI, I had to patch a similar issue in Apache Felix Fileinstall due to > the > > same k8s config maps/secrets issue [1] > > > > My 2 cents, > > Ray > > > > [1] https://issues.apache.org/jira/browse/FELIX-6229 > > > > > > On Fri., Aug. 20, 2021, 5:35 a.m. Ceki Gülcü, <[email protected]> wrote: > > > >> Hello Kedar, > >> > >> Have you tried to reproduce the problem on a stand alone machine without > >> k8? > >> > >> > >> > >> On 20/08/2021 00:41, Kedar Sirshikar wrote: > >> > Hello Ceki, > >> > I tried debugging by reproducing the issue and observed that after 7th > >> > attempt of log change, autoScan stops working. > >> > > >> > Have you came across similar issue in 1.2.3 version where autoScan > does > >> > not pick changes from config xml if it gets changed after a certain > >> count ? > >> > In the meantime, I am trying to see if same issue occurs on 1.1.3 > >> > version because setMainWatchURL method inConfigurationWatchListUtil is > >> > different on both versions. > >> > > >> > > >> > Thanks, > >> > Kedar. > >> > > >> > > >> > On Sat, Aug 14, 2021 at 3:00 AM <[email protected] > >> > <mailto:[email protected]>> wrote: > >> > > >> > Send logback-user mailing list submissions to > >> > [email protected] <mailto:[email protected]> > >> > > >> > To subscribe or unsubscribe via the World Wide Web, visit > >> > http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user> > >> > or, via email, send a message with subject or body 'help' to > >> > [email protected] <mailto:[email protected]> > >> > > >> > You can reach the person managing the list at > >> > [email protected] <mailto:[email protected]> > >> > > >> > When replying, please edit your Subject line so it is more > specific > >> > than "Re: Contents of logback-user digest..." > >> > > >> > > >> > Today's Topics: > >> > > >> > 1. Re: logback autoscan is not working intermittently in k8s > pod > >> > (Ceki Gülcü) > >> > > >> > > >> > > >> ---------------------------------------------------------------------- > >> > > >> > Message: 1 > >> > Date: Fri, 13 Aug 2021 14:51:09 +0200 > >> > From: Ceki Gülcü <[email protected] <mailto:[email protected]>> > >> > To: [email protected] <mailto:[email protected]> > >> > Subject: Re: [logback-user] logback autoscan is not working > >> > intermittently in k8s pod > >> > Message-ID: <[email protected] > >> > <mailto:[email protected]>> > >> > Content-Type: text/plain; charset=utf-8; format=flowed > >> > > >> > Hello Kedar, > >> > > >> > > >> > I see that changes to the logback config file are picked up. There > >> are > >> > some errors reported when processing the configuration file but > >> these > >> > are unrelated to the issue at hand, i.e. auto scan not working. > >> > > >> > I don't have further suggestions. > >> > > >> > --Ceki > >> > > >> > > >> > On 13/08/2021 09:59, Kedar Sirshikar wrote: > >> > > Hello team, > >> > > Just trying to check if anyone got a chance to look at this > case? > >> > > Appreciate if any clue/hint is provided to resolve this issue. > >> > > > >> > > Thanks, > >> > > Kedar. > >> > > > >> > > > >> > > On Wed, Aug 11, 2021 at 12:57 PM Kedar Sirshikar > >> > > <[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] > >> > <mailto:[email protected]>>> wrote: > >> > > > >> > > Resending the attachment by updating the subject. Please > let > >> > me know > >> > > if any additional details are required from my side. > >> > > > >> > > Thanks, > >> > > Kedar. > >> > > > >> > > > >> > > ---------- Forwarded message --------- > >> > > From: *Kedar Sirshikar* <[email protected] > >> > <mailto:[email protected]> > >> > > <mailto:[email protected] > >> > <mailto:[email protected]>>> > >> > > Date: Wed, Aug 11, 2021 at 7:58 AM > >> > > Subject: Re: logback-user Digest, Vol 115, Issue 2 > >> > > To: <[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>>> > >> > > > >> > > > >> > > Hi Ceki, Thank you for the reply. Forgot to send the logs > >> > earlier so > >> > > please refer attachment. > >> > > I have also added Notes at lines 1 and 91 to describe the > >> test. > >> > > > >> > > > >> > > Thanks, > >> > > Kedar. > >> > > > >> > > > >> > > On Wed, Aug 11, 2021 at 3:00 AM < > [email protected] > >> > <mailto:[email protected]> > >> > > <mailto:[email protected] > >> > <mailto:[email protected]>>> wrote: > >> > > > >> > > Send logback-user mailing list submissions to > >> > > [email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>> > >> > > > >> > > To subscribe or unsubscribe via the World Wide Web, > visit > >> > > http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user> > >> > > <http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user>> > >> > > or, via email, send a message with subject or body > >> 'help' to > >> > > [email protected] <mailto: > [email protected]> > >> > <mailto:[email protected] > >> > <mailto:[email protected]>> > >> > > > >> > > You can reach the person managing the list at > >> > > [email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto: > [email protected] > >> >> > >> > > > >> > > When replying, please edit your Subject line so it is > >> > more specific > >> > > than "Re: Contents of logback-user digest..." > >> > > > >> > > > >> > > Today's Topics: > >> > > > >> > > 1. logback autoscan is not working intermittently > in > >> > k8s pod > >> > > (Kedar Sirshikar) > >> > > 2. Re: logback autoscan is not working > intermittently > >> > in k8s pod > >> > > (Ceki Gülcü) > >> > > > >> > > > >> > > > >> > > >> ---------------------------------------------------------------------- > >> > > > >> > > Message: 1 > >> > > Date: Tue, 10 Aug 2021 23:07:51 -0700 > >> > > From: Kedar Sirshikar <[email protected] > >> > <mailto:[email protected]> > >> > > <mailto:[email protected] > >> > <mailto:[email protected]>>> > >> > > To: [email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>> > >> > > Subject: [logback-user] logback autoscan is not working > >> > > intermittently > >> > > in k8s pod > >> > > Message-ID: > >> > > > >> > > > >> > <CAO= > >> [email protected] <mailto: > >> cjl%2bemfcte7en%[email protected]> > >> > > > >> > <mailto: > >> cjl%2bemfcte7en%[email protected] > >> <mailto: > >> cjl%252bemfcte7en%[email protected] > >>> > >> > > Content-Type: text/plain; charset="utf-8" > >> > > > >> > > Hi team, > >> > > I have raised a detailed query here > >> > > > >> > < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> > > >> > > > >> > < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> >>> > >> > > about > >> > > the issue I am facing. > >> > > Can someone please share inputs or any hint for solving > >> the > >> > > above problem? > >> > > Let me know if any details are required from my side. > >> > > > >> > > Thanks, > >> > > Kedar. > >> > > -------------- next part -------------- > >> > > An HTML attachment was scrubbed... > >> > > URL: > >> > > > >> > < > >> > http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html > >> < > >> > http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html > >> > > >> > > > >> > < > >> > http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html > >> < > >> > http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html > >> >>> > >> > > > >> > > ------------------------------ > >> > > > >> > > Message: 2 > >> > > Date: Wed, 11 Aug 2021 08:39:46 +0200 > >> > > From: Ceki Gülcü <[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>>> > >> > > To: [email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>> > >> > > Subject: Re: [logback-user] logback autoscan is not > >> working > >> > > intermittently in k8s pod > >> > > Message-ID: < > [email protected] > >> > <mailto:[email protected]> > >> > > <mailto:[email protected] > >> > <mailto:[email protected]>>> > >> > > Content-Type: text/plain; charset=utf-8; format=flowed > >> > > > >> > > > >> > > Hello Kedar, > >> > > > >> > > Can you also provide the internal status messages > >> > generated by > >> > > logback? > >> > > -- > >> > > Ceki > >> > > > >> > > On 11/08/2021 08:07, Kedar Sirshikar wrote: > >> > > > Hi team, > >> > > > I have raised a detailed query here > >> > > > > >> > > > >> > < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> > > >> > > > >> > < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> < > >> > https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod > >> >>> about > >> > > > >> > > > the issue I am facing. > >> > > > Can someone please share inputs or any hint for > >> > solving the > >> > > above problem? > >> > > > Let me know if any details are required from my > side. > >> > > > > >> > > > Thanks, > >> > > > Kedar. > >> > > > > >> > > > _______________________________________________ > >> > > > logback-user mailing list > >> > > > [email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>> > >> > > > http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user> > >> > > <http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user>> > >> > > > > >> > > > >> > > > >> > > ------------------------------ > >> > > > >> > > >> > > >> > ------------------------------ > >> > > >> > Subject: Digest Footer > >> > > >> > _______________________________________________ > >> > logback-user mailing list > >> > [email protected] <mailto:[email protected]> > >> > http://mailman.qos.ch/mailman/listinfo/logback-user > >> > <http://mailman.qos.ch/mailman/listinfo/logback-user> > >> > > >> > ------------------------------ > >> > > >> > End of logback-user Digest, Vol 115, Issue 6 > >> > ******************************************** > >> > > >> > > >> > _______________________________________________ > >> > logback-user mailing list > >> > [email protected] > >> > http://mailman.qos.ch/mailman/listinfo/logback-user > >> > > >> _______________________________________________ > >> logback-user mailing list > >> [email protected] > >> http://mailman.qos.ch/mailman/listinfo/logback-user > > > > > > -- > *Raymond Augé* (@rotty3000) > Senior Software Architect *Liferay, Inc.* (@Liferay) > OSGi Fellow, Java Champion > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mailman.qos.ch/pipermail/logback-user/attachments/20210820/fcb5073f/attachment.html > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > logback-user mailing list > [email protected] > http://mailman.qos.ch/mailman/listinfo/logback-user > > ------------------------------ > > End of logback-user Digest, Vol 115, Issue 12 > ********************************************* >
_______________________________________________ logback-user mailing list [email protected] http://mailman.qos.ch/mailman/listinfo/logback-user
