Hi again Matt, I was able to repro the same with 0.12 nightly (ii prosody-0.12 1nightly194-1~focal). I added the following code to a module that loads under the main virtual host used with websockets and smacks module. With config `smacks_hibernation_time = 60;`:
local function handle_pre_resource_unbind(event)
local error = event.error
local participant_jid = event.session.full_jid
if error then
if participant_jid then
module:log("warn", "Participant %s disconnected abnormally
error %s", participant_jid, error)
end
end
end
module:hook("pre-resource-unbind", handle_pre_resource_unbind, 10);
I'm attaching the 3 debug logs I captured, with 0.12.2, with nightly and
with nightly after applying the patch that is mentioned in the previous
emails.
We have been running with the patch for more than a month now on meet.jit.si
.
Thank you
damencho
On Sat, Feb 18, 2023 at 8:01 AM Damian Minkov <[email protected]> wrote:
>
>
> On Saturday, February 18, 2023 at 7:50:56 AM UTC-6 Damian Minkov wrote:
> Hi,
>
>
> On Sat, Feb 18, 2023, 04:59 Matthew Wild <[email protected]> wrote:
> Hi Damian,
>
> On Fri, 17 Feb 2023 at 13:58, Damian Minkov <[email protected]> wrote:
> >
> > We have been running this patch for more than a month without noticing
> any major issues ... one thing I noticed today is this error, have seen it
> only once:
>
> Sorry for the delayed response on this thread. Past few months have been
> busy.
>
> It sounds like the original issue is easy to reproduce for you. Any
> chance you can try it with the latest 0.12 code (e.g. the
> 'prosody-0.12' package from our repo)? We fixed a bug recently related
> to smacks hibernation, and I'm not sure whether this is related or
> not.
>
> I will check it out next week and let you know. This is different than
> 0.12.2?
> I see now the various session destruction commits in the 0.12 branch after
> 0.12.2.
> I will test it first thing on Monday and will let you know.
>
> Thanks
>
>
>
> Skimming your patch, I'm not clear on exactly what the root cause is.
> A hibernating session wouldn't actually be destroyed (just
> session.conn removed), so I would expect things like the outgoing
> stanza queue to remain.
>
> Actually, I'm also not entirely clear on what the issue you're
> experiencing is. The "hibernating too long" message is not necessarily
> an error - it's expected behaviour if the client disconnects uncleanly
> and does not return before the configured hibernation timeout, which
> can be very common. In the case of a clean disconnect, the hibernation
> code should not kick in at all. In the case of hibernation, the
> session should not be destroyed until after the timeout period. Debug
> logs during the disconnect of a client would help identify which of
> these is happening.
>
> Yep, we saw the problem as we added some logs to the unbind event (
> https://github.com/jitsi/infra-configuration/blob/ae1b579bce46cfbe686ec2cfc4b44b6e2b0309d2/ansible/roles/prosody/files/mod_muc_webhooks.lua#L706).
> To detect those sessions that actually lost connection. And then we noticed
> that we got the print for hibernation for all sessions even for the cleanly
> disconnected, the print is of course a minute after the clean disconnect
> (this is our setting for hibernating a connection duration).
>
> The order of calls is wrong and on clean disconnect, sessions are not
> cleared correctly and all go into hibernation.
>
> Add the unbind message and do a clean connect and disconnect via the
> websocket and you will able to repro.
>
> I hope this makes more sense.
>
> Thanks
> damencho
>
>
>
>
> Regards,
> Matthew
>
> --
> You received this message because you are subscribed to the Google Groups
> "prosody-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prosody-dev/CAJt9-x7t2nMjYBopCTRAG%3DiVMt2RZqsr_Pm6MhBnO-ZrhRb_rQ%40mail.gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "prosody-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prosody-dev/269d0c66-5690-4646-8d6d-5319d0815a94n%40googlegroups.com
> <https://groups.google.com/d/msgid/prosody-dev/269d0c66-5690-4646-8d6d-5319d0815a94n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"prosody-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prosody-dev/CAE07f0JvsvrpRobz1reBfCAQ%2BvOcAuDxCQwsXn-BaXeyzJn_LQ%40mail.gmail.com.
0.12-nightly.log
Description: Binary data
0.12-nigtly-patched.log
Description: Binary data
0.12.2.log
Description: Binary data
