Thank you Raymond for sharing the inputs so looks like this is a
bottleneck from logback especially with k8s ConfigMap.
Is there any workaround (apart from restarting the java process) to
address this issue ? OR I'll have to wait until it gets fixed from
logback by considering checksum per below suggestion.
Hello Ceki, please share your advice if you have any.
Thanks,
Kedar.
On Fri, Aug 20, 2021 at 7:06 AM <[email protected]
<mailto:[email protected]>> wrote:
Send logback-user mailing list submissions to
[email protected] <mailto:[email protected]>
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
or, via email, send a message with subject or body 'help' to
[email protected] <mailto:[email protected]>
You can reach the person managing the list at
[email protected] <mailto:[email protected]>
When replying, please edit your Subject line so it is more specific
than "Re: Contents of logback-user digest..."
Today's Topics:
1. Re: logback autoscan is not working intermittently in k8s pod
(Raymond Augé)
----------------------------------------------------------------------
Message: 1
Date: Fri, 20 Aug 2021 10:05:22 -0400
From: Raymond Augé <[email protected]
<mailto:[email protected]>>
To: logback users list <[email protected]
<mailto:[email protected]>>
Subject: Re: [logback-user] logback autoscan is not working
intermittently in k8s pod
Message-ID:
<camm6hcc+fkzu4ux-z-wpdqk+lgwt_shwaevb7otr3j5dje-...@mail.gmail.com
<mailto:camm6hcc%2bfkzu4ux-z-wpdqk%[email protected]>>
Content-Type: text/plain; charset="utf-8"
Here's a direct link [1] to the explanation of the issue which I
confirmed
myself to be the actual issue I was facing.
Maybe it will help both the OP and logback :)
[1] https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/
<https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/>
On Fri, Aug 20, 2021 at 10:01 AM Raymond Augé
<[email protected] <mailto:[email protected]>>
wrote:
> Hi, a long time lurker here 🙂.
>
> So, I suspect the issue is with k8s and it's volumes for config
> maps/secrets symlinks are a pain point in Java. Even inotify based
> WachService cannot handle them.
>
> The issue is that logback is only using a timestamp to detect
> configuration file changes, which is fine as we know in 99% of
Java usages.
>
> However, this doesn't seem to work with the config map volumes
supplied in
> a k8s env because they use symlinks which don't cause the
timestamp to
> change, I agree it's weird and unexpected.
>
> However, supplying configuration files via K8s config maps is
going to
> start being a much more common use case as time passes.
>
> Logback should probably start including a checksum to detect if the
> contents of the file have changed and use a logical OR with the
timestamp
> check so that either changes causes a reload.
>
> FYI, I had to patch a similar issue in Apache Felix Fileinstall
due to the
> same k8s config maps/secrets issue [1]
>
> My 2 cents,
> Ray
>
> [1] https://issues.apache.org/jira/browse/FELIX-6229
<https://issues.apache.org/jira/browse/FELIX-6229>
>
>
> On Fri., Aug. 20, 2021, 5:35 a.m. Ceki Gülcü, <[email protected]
<mailto:[email protected]>> wrote:
>
>> Hello Kedar,
>>
>> Have you tried to reproduce the problem on a stand alone machine
without
>> k8?
>>
>>
>>
>> On 20/08/2021 00:41, Kedar Sirshikar wrote:
>> > Hello Ceki,
>> > I tried debugging by reproducing the issue and observed that
after 7th
>> > attempt of log change, autoScan stops working.
>> >
>> > Have you came across similar issue in 1.2.3 version where
autoScan does
>> > not pick changes from config xml if it gets changed after a
certain
>> count ?
>> > In the meantime, I am trying to see if same issue occurs on 1.1.3
>> > version because setMainWatchURL method
inConfigurationWatchListUtil is
>> > different on both versions.
>> >
>> >
>> > Thanks,
>> > Kedar.
>> >
>> >
>> > On Sat, Aug 14, 2021 at 3:00 AM <[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>> wrote:
>> >
>> > Send logback-user mailing list submissions to
>> > [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> > http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>
>> > or, via email, send a message with subject or body 'help' to
>> > [email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
>> >
>> > You can reach the person managing the list at
>> > [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> >
>> > When replying, please edit your Subject line so it is more
specific
>> > than "Re: Contents of logback-user digest..."
>> >
>> >
>> > Today's Topics:
>> >
>> > 1. Re: logback autoscan is not working intermittently
in k8s pod
>> > (Ceki Gülcü)
>> >
>> >
>> >
>>
----------------------------------------------------------------------
>> >
>> > Message: 1
>> > Date: Fri, 13 Aug 2021 14:51:09 +0200
>> > From: Ceki Gülcü <[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>
>> > To: [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> > Subject: Re: [logback-user] logback autoscan is not working
>> > intermittently in k8s pod
>> > Message-ID: <[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>
>> > Content-Type: text/plain; charset=utf-8; format=flowed
>> >
>> > Hello Kedar,
>> >
>> >
>> > I see that changes to the logback config file are picked
up. There
>> are
>> > some errors reported when processing the configuration
file but
>> these
>> > are unrelated to the issue at hand, i.e. auto scan not
working.
>> >
>> > I don't have further suggestions.
>> >
>> > --Ceki
>> >
>> >
>> > On 13/08/2021 09:59, Kedar Sirshikar wrote:
>> > > Hello team,
>> > > Just trying to check if anyone got a chance to look at
this case?
>> > > Appreciate if any clue/hint is provided to resolve this
issue.
>> > >
>> > > Thanks,
>> > > Kedar.
>> > >
>> > >
>> > > On Wed, Aug 11, 2021 at 12:57 PM Kedar Sirshikar
>> > > <[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>> wrote:
>> > >
>> > > Resending the attachment by updating the subject.
Please let
>> > me know
>> > > if any additional details are required from my side.
>> > >
>> > > Thanks,
>> > > Kedar.
>> > >
>> > >
>> > > ---------- Forwarded message ---------
>> > > From: *Kedar Sirshikar* <[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>
>> > > <mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>>
>> > > Date: Wed, Aug 11, 2021 at 7:58 AM
>> > > Subject: Re: logback-user Digest, Vol 115, Issue 2
>> > > To: <[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>>
>> > >
>> > >
>> > > Hi Ceki, Thank you for the reply. Forgot to send
the logs
>> > earlier so
>> > > please refer attachment.
>> > > I have also added Notes at lines 1 and 91 to
describe the
>> test.
>> > >
>> > >
>> > > Thanks,
>> > > Kedar.
>> > >
>> > >
>> > > On Wed, Aug 11, 2021 at 3:00 AM
<[email protected] <mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>
>> > > <mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>> wrote:
>> > >
>> > > Send logback-user mailing list submissions to
>> > > [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>
>> > >
>> > > To subscribe or unsubscribe via the World Wide
Web, visit
>> > > http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>
>> > >
<http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>>
>> > > or, via email, send a message with subject or body
>> 'help' to
>> > > [email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>
>> > >
>> > > You can reach the person managing the list at
>> > > [email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>
>> >>
>> > >
>> > > When replying, please edit your Subject line so
it is
>> > more specific
>> > > than "Re: Contents of logback-user digest..."
>> > >
>> > >
>> > > Today's Topics:
>> > >
>> > > 1. logback autoscan is not working
intermittently in
>> > k8s pod
>> > > (Kedar Sirshikar)
>> > > 2. Re: logback autoscan is not working
intermittently
>> > in k8s pod
>> > > (Ceki Gülcü)
>> > >
>> > >
>> > >
>> >
>>
----------------------------------------------------------------------
>> > >
>> > > Message: 1
>> > > Date: Tue, 10 Aug 2021 23:07:51 -0700
>> > > From: Kedar Sirshikar
<[email protected] <mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>
>> > > <mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>>
>> > > To: [email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>
>> > > Subject: [logback-user] logback autoscan is not
working
>> > > intermittently
>> > > in k8s pod
>> > > Message-ID:
>> > >
>> > >
>> > <CAO=
>> [email protected]
<mailto:cjl%2bemfcte7en%[email protected]>
<mailto:
>>
cjl%2bemfcte7en%[email protected]
<mailto:cjl%252bemfcte7en%[email protected]>>
>> > >
>> > <mailto:
>>
cjl%2bemfcte7en%[email protected]
<mailto:cjl%252bemfcte7en%[email protected]>
>> <mailto:
>>
cjl%252bemfcte7en%[email protected]
<mailto:cjl%25252bemfcte7en%[email protected]>>>>
>> > > Content-Type: text/plain; charset="utf-8"
>> > >
>> > > Hi team,
>> > > I have raised a detailed query here
>> > >
>> > <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> >
>> > >
>> > <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> >>>
>> > > about
>> > > the issue I am facing.
>> > > Can someone please share inputs or any hint for
solving
>> the
>> > > above problem?
>> > > Let me know if any details are required from my
side.
>> > >
>> > > Thanks,
>> > > Kedar.
>> > > -------------- next part --------------
>> > > An HTML attachment was scrubbed...
>> > > URL:
>> > >
>> > <
>>
http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html>
>> <
>>
http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html>
>> >
>> > >
>> > <
>>
http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html>
>> <
>>
http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html>
>> >>>
>> > >
>> > > ------------------------------
>> > >
>> > > Message: 2
>> > > Date: Wed, 11 Aug 2021 08:39:46 +0200
>> > > From: Ceki Gülcü <[email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>>
>> > > To: [email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>
>> > > Subject: Re: [logback-user] logback autoscan is not
>> working
>> > > intermittently in k8s pod
>> > > Message-ID:
<[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>
>> > >
<mailto:[email protected]
<mailto:[email protected]>
>> > <mailto:[email protected]
<mailto:[email protected]>>>>
>> > > Content-Type: text/plain; charset=utf-8;
format=flowed
>> > >
>> > >
>> > > Hello Kedar,
>> > >
>> > > Can you also provide the internal status messages
>> > generated by
>> > > logback?
>> > > --
>> > > Ceki
>> > >
>> > > On 11/08/2021 08:07, Kedar Sirshikar wrote:
>> > > > Hi team,
>> > > > I have raised a detailed query here
>> > > >
>> > >
>> > <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> >
>> > >
>> > <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> <
>>
https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
<https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod>
>> >>> about
>> > >
>> > > > the issue I am facing.
>> > > > Can someone please share inputs or any hint for
>> > solving the
>> > > above problem?
>> > > > Let me know if any details are required from
my side.
>> > > >
>> > > > Thanks,
>> > > > Kedar.
>> > > >
>> > > > _______________________________________________
>> > > > logback-user mailing list
>> > > > [email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>
>> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>
>> > > >
http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>
>> > >
<http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>>
>> > > >
>> > >
>> > >
>> > > ------------------------------
>> > >
>> >
>> >
>> > ------------------------------
>> >
>> > Subject: Digest Footer
>> >
>> > _______________________________________________
>> > logback-user mailing list
>> > [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> > http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> > <http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>>
>> >
>> > ------------------------------
>> >
>> > End of logback-user Digest, Vol 115, Issue 6
>> > ********************************************
>> >
>> >
>> > _______________________________________________
>> > logback-user mailing list
>> > [email protected] <mailto:[email protected]>
>> > http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>> >
>> _______________________________________________
>> logback-user mailing list
>> [email protected] <mailto:[email protected]>
>> http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
>
>
--
*Raymond Augé* (@rotty3000)
Senior Software Architect *Liferay, Inc.* (@Liferay)
OSGi Fellow, Java Champion
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210820/fcb5073f/attachment.html
<http://mailman.qos.ch/pipermail/logback-user/attachments/20210820/fcb5073f/attachment.html>>
------------------------------
Subject: Digest Footer
_______________________________________________
logback-user mailing list
[email protected] <mailto:[email protected]>
http://mailman.qos.ch/mailman/listinfo/logback-user
<http://mailman.qos.ch/mailman/listinfo/logback-user>
------------------------------
End of logback-user Digest, Vol 115, Issue 12
*********************************************
_______________________________________________
logback-user mailing list
[email protected]
http://mailman.qos.ch/mailman/listinfo/logback-user
