Understood, and we don’t see this. However we have modified the responders, by making guacamole aware if a VM is up and running.
We wrote an executable that runs on the guest VM system that notifies our platform (StormCloud) it is up and running and awaiting a connection, once the connection is established the program then sets a flag in the database as in use. Once a user is done our system will do 1 of 2 things based on initial parameters being set during the template build, hold session VM state (persistent state), or reset state of VM to zero (Ondemand). We also place a timer on failed attempts to 3 minutes, if an attempt moves in to failed connection state we are notified, then our system will auto remediate, by resetting connection tunnels, then reboot the VM, generally this corrects the issue. Now if you’re in a high VM saturated environment (or Starved resources) you will have connections issues due to lack of resources and/or CPU scheduling. This works for all the connection methods. One main difference is we do not use Docker, we use a full VDI on Ubuntu 20/22.04LTS for greater security. Specs on we use on Guacamole, vCPU 4x8 (32 cores) RAM 64G NIC 10G Drive 25GB This will handle about 1000 connections at once, we run anywhere from 5 to 8 guacamole servers which our platform load balances the connection to the Guacamole systems. This should help clear up on why we don’t see this issue. Thank You Sean Hulbert <http://www.securitycentric.net/> www.securitycentric.net A Cybersecurity Enablement Company We don't just run you through the motions, Our labs teach you how to think! System Award Management CAGE: 8AUV4 AFCEA San Francisco Chapter V.P. If you have heard of a hacker by name, he/she has failed, fear the hacker you haven’t heard of! CONFIDENTIALITY NOTICE: This communication with its contents may contain confidential and/or legally privileged information. It is solely for the use of the intended recipient(s). Unauthorized interception, review, use or disclosure is prohibited and may violate applicable laws including the Electronic Communications Privacy Act. If you are not the intended recipient, please contact the sender and destroy all copies of the communication. Content within this email communication is not legally binding as a contract and no promises are guaranteed unless in a formal contract outside this email communication. igitur qui desiderat pacem, praeparet bellum!!! Epitoma Rei Militaris From: Lee Doughty [mailto:l...@virginiacyberrange.org] Sent: Wednesday, June 8, 2022 7:43 AM To: user@guacamole.apache.org Subject: Re: Guacamole re-connection attempts never stop... they should? Sean, I don't think you're seeing the problem area correctly on this issue.. The issue is that guacamole-client -- the javascript for the front-end that the server gives to users -- has an infinite loop on retrying connections. You can modify guacamole yourself to get away from this problem (that's what GUACAMOLE-1126 shows), but if you are using stock guacamole-client, you have the possibility for this infinite loop In order to trigger this, the target machine must disconnect the user forcefully... Such as the target experiencing a hardware failure, or the target machine otherwise no longer working over the designated protocol (firewall rule, or disabling the protocol's service), or the machine "bouncing" due to a power failure (and possibly now being on a different IP address) -- These are actions/events/issues on the target VM, which might be performed by the user utilizing the connection, or by some other out-of-band process/error. Once the machine is effectively offline/unreachable by guacamole, the user's browser will continue to poll the guacamole server every 15 seconds with an attempt to restart the session to the target. The connection attempt usually takes some time to fail, which adds some delay on the loop (typically ~60 seconds, I think). The server getting hassled by this process is simply the result of being the middle-man between an open browser tab on the user side trying to connect to a VM behind guacamole that was, but is no longer, available. My use case heavily involves education of students around firewalls, server security, etc., so we see dozens of different ways the target machine can fail. USUALLY the students realize the connection is not recoverable and stop the loop themselves after a few minutes (and recovery of the target VM is outside the scope of guacamole), but sometimes they walk (or tab) away, and guacamole retries indefinitely (which some people view as undesirable behaviour). The default for guacamole works very well if you connect users to always-on machines who are utilizing unprivileged user accounts or are otherwise skilled administrators... but starts to have the above issue for ephemeral/on-demand targets, or when users can cause their machines/connections to become unstable -- which admittedly are not guacamole problems -- but I believe the fact guacamole assumes machines will always be available or recoverable and never stops trying IS a guacamole issue. -Lee On Tue, Jun 7, 2022 at 7:18 PM Sean Hulbert <shulb...@securitycentric.net.invalid <mailto:shulb...@securitycentric.net.invalid> > wrote: This has to be a Docker image issue, We run a few guacamole servers and never experienced this issue, however we did built them from scratch and modified the kernels. Thank You Sean Hulbert Founder / CEO Work Ph: 925.292.4309 <http://www.securitycentric.net/> www.securitycentric.net A Cybersecurity Enablement Company We don't just run you through the motions, Our labs teach you how to think! System Award Management CAGE: 8AUV4 AFCEA San Francisco Chapter V.P. If you have heard of a hacker by name, he/she has failed, fear the hacker you haven’t heard of! CONFIDENTIALITY NOTICE: This communication with its contents may contain confidential and/or legally privileged information. It is solely for the use of the intended recipient(s). Unauthorized interception, review, use or disclosure is prohibited and may violate applicable laws including the Electronic Communications Privacy Act. If you are not the intended recipient, please contact the sender and destroy all copies of the communication. Content within this email communication is not legally binding as a contract and no promises are guaranteed unless in a formal contract outside this email communication. igitur qui desiderat pacem, praeparet bellum!!! Epitoma Rei Militaris From: Lee Doughty [mailto:l...@virginiacyberrange.org <mailto:l...@virginiacyberrange.org> ] Sent: Tuesday, June 7, 2022 2:46 PM To: user@guacamole.apache.org <mailto:user@guacamole.apache.org> Subject: Re: Guacamole re-connection attempts never stop... they should? Nick, I agree https://issues.apache.org/jira/browse/ <https://issues.apache.org/jira/browse/GUACAMOLE-1126> GUACAMOLE-1126 looks in the ballpark. I have no personal investment in the implementation, I would simply like the endless server pings to stop so we can more easily monitor application logs, have less "errors" in our logs, and reduce the baseline load on the servers. We do somewhat inflict this problem on ourselves (I think), because we shut down the VMs people are using, but we do so because we reasonably believe the user is idle and wasting resources... I suppose if there was a "guacamole session time limit" that could be set when the user initially authenticates, and they get kicked out and forced to re-authenticate at that point (or at least click a button to extend their session).... that would resolve my issue too... as I understand the auth process now, once they have a guacamole-guacd session successfully established, that seems to be almost entirely independent of their user account or the connection's properties... I have not tried it, but I suspect you could delete either of them, and guacamole would still let the user continue their current session. -Lee On Wed, Jun 1, 2022 at 5:18 AM Jürgen Kuri <juergen.k...@ionos.com <mailto:juergen.k...@ionos.com> > wrote: Suggestion: 1) keep endless reconnect as default 2) have two connection specific and / or global parameter: a. number of retries b. retry interval 3) if we have these parameters as global and on connection level, global is overwritten for specific connection if defined El 31.05.22 a las 22:44, Lee Doughty escribió: > Hello Guacamole Community, > > I tried asking this a few weeks ago, but it looks like there was not a lot of > traction on this idea.. but I wanted to try one more time before I gave up on > it. > > I think it would be a great feature to stop auto-reconnect attempts that are > simply not connecting after several dozen attempts. I've seen in our logs > that some users hit the "Reconnect" button or otherwise get into a reconnect > loop, then leave the tab open for hours *or days*. This results in our > guacamole server getting a ping every minute or so from a user trying to > connect to a VM that is not available, and they just leave it retrying over > and over again. > > It would be nice to at least require user interaction to resume the > connection attempts... So users have to return to the tab every N attempts to > restart the countdown, instead of the current never-ending loop... I'm not > suggesting any value for N... because any reasonable value would be nice over > infinite. My record was somewhere in the ballpark of 7,000 attempts (5 days) > before the user was kind enough to close the tab and stop poking our > Guacamole server. > > Is this something that can make it into an upcoming Guacamole release? > > -Lee > -- Jürgen --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@guacamole.apache.org <mailto:user-unsubscr...@guacamole.apache.org> For additional commands, e-mail: user-h...@guacamole.apache.org <mailto:user-h...@guacamole.apache.org>
