Hi Nick, This is very helpful by narrowing this down to a memory allocation problem. I have the "top" application open so I'll keep an eye on it. We gave 2GB of memory to this particular virtual machine. We never have more than 5 users connecting at a time and all they use it for is RDP. Looks like there's a guacd process of about 150MB resident memory for each connection but also an extra guacd process. I assume that's a parent process. I'm also going to keep an eye on Tomcat though. It's currently referencing 2GB of memory but with 236MB resident. If Tomcat has a memory leak maybe that ends up preventing new guacd connections.
Thank you for your help! I'll see how these numbers look on Monday. Brian On Fri, Aug 16, 2024 at 10:49 AM Nick Couchman <vn...@apache.org> wrote: > On Fri, Aug 16, 2024 at 10:08 AM Brian Filipek > <bfili...@betterworldbooks.com.invalid> wrote: > >> Hi Nick, >> >> This problem continues in FreeBSD with guacd randomly signal 6 aborting >> approximately once a week.When this happens I need to log in and restart >> the guacd service. We had a crash last night and here are the last few >> lines of /var/log/messages: >> >> Aug 15 11:53:14 guacamole guacd[17699]: Client did not terminate in a >> timely manner. Forcibly terminating client and any child processes. >> Aug 15 15:35:14 guacamole guacd[17752]: Client did not terminate in a >> timely manner. Forcibly terminating client and any child processes. >> Aug 15 15:59:19 guacamole guacd[32874]: User is not responding. >> Aug 16 00:09:22 guacamole guacd[65139]: User is not responding. >> Aug 16 03:05:28 guacamole guacd[79034]: User is not responding. >> Aug 16 03:25:22 guacamole kernel: pid 75166 (guacd), jid 0, uid 899: >> exited on signal 6 (no core dump - other error) >> Aug 16 03:25:22 guacamole kernel: pid 1758 (guacd), jid 0, uid 899: >> exited on signal 6 (no core dump - other error) >> Aug 16 03:25:22 guacamole guacd[67047]: Unable to shutdown internal >> socket for connection $2bbbb77e-d090-441a-b2d5-c337bf55b13c. Corresponding >> process may remain running but inactive. >> Aug 16 08:20:40 guacamole login[1783]: ROOT LOGIN (root) ON ttyv0 >> >> > I did a quick Google search on "signal 6" and found that the general > consensus seems to be that this is a "voluntary" termination by the process > (guacd), usually triggered by a call to the "abort()" function. A quick > search through the guacd source code shows that the only times we every > call this function are in the libguac memory allocation functions ( > https://github.com/apache/guacamole-server/blob/main/src/libguac/mem.c), > during memory arithmetic processes (add, subtract, multiple), and during > the realloc process. So, my _guess_ here is that it is failing to allocate > required memory for some operation and is aborting. > > It's possible there's a memory leak somewhere in guacd that is manifesting > itself in your particular case on a semi-regular basis, but hard to say. > The interesting things to try to track this down further would be: > * Any information you have on resources utilization - particularly memory > - that would show any sort of a trend of memory utilization growing up to > the point where the process crashes, resetting, then growing again, etc. > * Any information on resource constraints for the process - that is, are > you running it in such a way that you've limited the available memory to > that specific process (a FreeBSD jail, or whatever the FreeBSD equivalent > might be of the Linux "limits" - any limits on heap space, threads, etc. > * If you can collect a stack trace during the abort, it would be really > useful to drill down and figure where in the code this is being triggered - > if it's in one of the four places in the mem.c file that I mentioned, or if > something else is causing it. > > -Nick > >> -- Regards, Brian Filipek Technical Support Specialist 55740 Currant Road Mishawaka, IN 46545 (574) 855-5745 bfili...@betterworldbooks.com betterworldbooks.com Better World Books is a for-profit social enterprise that collects and sells books online with each sale generating funds for literacy initiatives around the world.
