Thank you for your reply. I looked further. libvnc does seem to support
X509 certificate-based TLS for VNC. What is missing is support for
authentication in the guacamole server itself.
The interaction between guacamole server and libvncclient is as follows.
guacd calls libvncclient to manage the VNC connection. During the initial
VNC packet exchanges, the VNC server and libvncclient agree on
encryption/authentication schemes for the VNC connection. My requirement
for the connection is to:
1) Use TLS for encryption.
2) Avoid using anonymous TLS.
So I configure my VNC server accordingly. I can choose to configure VLS
authentication type that the server offers to be either rfbVeNCryptX509None
(260), rfbVeNCryptX509VNC (261) or rfbVeNCryptX509Plain (262), all of which
are based on X509 certificates. All such attempts result in guacamole
server dropping the connection and producing output like this one.
NOTE: This output comes from libvncclient library performing the initial
exchange of parameters with the server:
Aug 05 17:47:11 vbox guacd[4460]: VNC server supports protocol version 3.8
(viewer 3.8)
Aug 05 17:47:11 vbox guacd[4460]: We have 1 security types to read
Aug 05 17:47:11 vbox guacd[4460]: 0) Received security type 19
Aug 05 17:47:11 vbox guacd[4460]: Selecting security type 19 (0/1 in the
list)
Aug 05 17:47:11 vbox guacd[4460]: Selected Security Scheme 19
Aug 05 17:47:11 vbox guacd[4460]: Got VeNCrypt version 0.2 from server.
Aug 05 17:47:11 vbox guacd[4460]: We have 1 security types to read
Aug 05 17:47:11 vbox guacd[4460]: 0) Received security type 260
Aug 05 17:47:11 vbox guacd[4460]: Selecting security type 260
Aug 05 17:47:11 vbox guacd[4460]: GnuTLS version 3.8.10 initialized.
NOTE: Here, libvncclient calls the callback for credentials in guacd. This
is guacd output. Note the requested type of credential (1 means X509).
Aug 05 17:47:11 vbox guacd[4460]: guacd[4460]: ERROR: Unsupported
credential type requested.
Aug 05 17:47:11 vbox guacd[4460]: guacd[4460]: DEBUG: Unable to
provide requested type of credential: 1.
NOTE: libvncclient again.
Aug 05 17:47:11 vbox guacd[4460]: Unsupported credential type requested.
Aug 05 17:47:11 vbox guacd[4460]: Unable to provide requested type of
credential: 1.
Aug 05 17:47:11 vbox guacd[4460]: Reading credential failed
So guacd fails even before attempting any TLS handshake. This is how this
works on the source code level.
1. Libvncclient provides the rfbCredential structure. The structure is used
by the main program (guacd in this case) to convey VNC credentials to the
library. Two types of credentials sets are supported by libvnclient:
username/password and X509 credentials such as CA certificate, client
certificate/key, etc.
libvncclient:
/usr/include/rfb/rfbclient.h:
/** For GetCredentialProc callback function to return */
typedef union _rfbCredential
{
/** X509 (VeNCrypt) */
struct
{
char *x509CACertFile;
char *x509CACrlFile;
char *x509ClientCertFile;
char *x509ClientKeyFile;
uint8_t x509CrlVerifyMode; /* Only required for OpenSSL - see meanings
below */
} x509Credential;
/** Plain (VeNCrypt), MSLogon (UltraVNC) */
struct
{
char *username;
char *password;
} userCredential;
} rfbCredential;
2. This is how libvncclient handles TLS and credentials. For non-anonymous
VNC+TLS, it calls the callback function GetCredential() and expects to get
credentials for the authentication scheme it has just negotiated with the
server. The credential type supplied to the callback is
rfbCredentialTypeX509 (1).
libvncclient:
src/libvncclient/tls_gnutls.c:
rfbBool
HandleVeNCryptAuth(rfbClient* client) {
....
switch (authScheme)
{
/* Unencrypted types do not require additional actions */
case rfbNoAuth:
case rfbVncAuth:
case rfbVeNCryptPlain:
return TRUE;
break;
/* Some VeNCrypt security types are anonymous TLS, others are X509 */
case rfbVeNCryptTLSNone:
case rfbVeNCryptTLSVNC:
case rfbVeNCryptTLSPlain:
#ifdef LIBVNCSERVER_HAVE_SASL
case rfbVeNCryptTLSSASL:
#endif /* LIBVNCSERVER_HAVE_SASL */
anonTLS = TRUE;
break;
default:
anonTLS = FALSE;
break;
}
/* Get X509 Credentials if it's not anonymous */
if (!anonTLS)
{
rfbCredential *cred;
if (!client->GetCredential)
{
rfbClientLog("GetCredential callback is not set.\n");
return FALSE;
}
cred = client->GetCredential(client, rfbCredentialTypeX509);
if (!cred)
{
rfbClientLog("Reading credential failed\n");
return FALSE;
}
x509_cred = CreateX509CertCredential(cred);
FreeX509Credential(cred);
if (!x509_cred) return FALSE;
}
3. Finally, this is the guacd callback (guac_vnc_get_credentials()) code
(omitted irrelevant lines). It is clear that it supports only
username/password credentials (rfbCredentialTypeUser == 2).
guacamole-server:
src/protocols/vnc/vnc.c:
#ifdef ENABLE_VNC_GENERIC_CREDENTIALS
/* Authentication */
rfb_client->GetCredential = guac_vnc_get_credentials;
#endif
#ifdef ENABLE_VNC_GENERIC_CREDENTIALS
rfbCredential* guac_vnc_get_credentials(rfbClient* client, int
credentialType) {
....
/* Handle request for Username/Password credentials */
if (credentialType == rfbCredentialTypeUser) {
rfbCredential *creds = malloc(sizeof(rfbCredential));
.... Get username/password credentials for this connection.
/* Copy the values and return the credential set. */
creds->userCredential.username = guac_strdup(settings->username);
creds->userCredential.password = guac_strdup(settings->password);
return creds;
}
guac_client_abort(gc, GUAC_PROTOCOL_STATUS_SERVER_ERROR,
"Unsupported credential type requested.");
guac_client_log(gc, GUAC_LOG_DEBUG,
"Unable to provide requested type of credential: %d.",
credentialType);
return NULL;
}
#endif
As far as I understand, in the simplest case, username/password are taken
from user-mapping.xml file. But there are no similar settings in guacamole
client/server configuration for the server CA to trust, or for its client
certificate/key.
So non-anonymous TLS is not supported by guacamole server at this point
(August 2025). My question is, are there any plans to add support for TLS
credentials? At the very least, the server CA certificate support would be
helpful.
Oleg
On Mon, Aug 4, 2025 at 9:48 PM Nick Couchman <[email protected]> wrote:
> On Thu, Jul 31, 2025 at 6:50 AM Oleg Rosowiecki <[email protected]>
> wrote:
>
> > Hello,
> >
> > I am struggling to make Guacamole use TLS with VNC. Anonymous TLS works,
> > but my requirement is not to use anonymous TLS. If I configure a
> > certificate on the server side of the VNC connection (I'm using x11vnc as
> > the server), Guacamole is unable to establish communication to the
> server.
> > I am running Guacamole version 1.6.0.
> >
> >
> Can you provide more specific details as to how you're setting this up on
> the server side? Depending on what you mean by "TLS with VNC", Guacamole
> may not support this, but it's a bit hard to say. It _should_ support TLS +
> normal VNC credentials (username + password and possibly even just
> password). I haven't looked at the libvncserver library for this
> specifically, but I would not be shocked if it doesn't support
> certificate-based authentication (client certificates). Anyway, more detail
> on how you're configuring the server should help determine that.
>
>
> > guacd displays the following in debug mode (reducing the log to relevant
> > entries):
> >
> > Received security type 10 (0/1 in the list)
> > Selected Security Scheme 10
> > Got VeNCrypt version 0.2. from server.
> > We have 1 security types to read.
> > 0) Received security type 260
> > Selecting security type 260
> > GnuTLS version 3.8.10 initialized.
> > guacd[4139]: ERROR: Unsupported credential type requested.
> > guacd[4139]: DEBUG: Unable to provide requested type of credential: 1.
> >
> > As far as I can see, the error messages are produced by
> > guac_vnc_get_credentials() function in the guacd source code. The
> function
> > only expects rfbCredentialTypeUser (2, defined in libvncserver). All
> other
> > types are rejected, including the numeric value 1 reported in the log.
> This
> > value 1 corresponds to rfbCredentialTypeX509 in libvncserver.
> >
> >
> The fact that it's triggering a rfbCredentialTypeX509 makes me think that
> it's trying to do certificate-based authentication. Again, I'm uncertain as
> to the level of support libvncserver has for this - if it does support it,
> then we just need to implement it in Guacamole. If it does not, then you're
> out of luck.
>
> -Nick
>
