Here is my bugfix for the problem: The idea is to set a flag when the authentication result was sent and to inhibit the problematic server message funktions sending disturbing bytes. The presented method might be quite stupid but so I didn't had to dive in that deep of the code. I hope in coming distibutions the bug will be fixed (even more sophisticated ?!)
I compiled the patched version of the unix code (3.3.7) on a linux debian-woody distribution and it looks ok. The patched source files reside in the directory: "~/vnc-3.3.7-unixsrc/Xvnc/programs/Xserver/hw/vnc" I introduced a new flag in the "rfb.h" that will be set if the authentication result was sent: typedef struct rfbClientRec { int sock; char *host; /* Possible client states: */ enum { RFB_PROTOCOL_VERSION, /* establishing protocol version */ RFB_AUTHENTICATION, /* authenticating */ RFB_INITIALISATION, /* sending initialisation messages */ RFB_NORMAL /* normal protocol messages */ } state; -> Bool rfb_authentication_sent; /* new flag for bugfix */ Bool reverseConnection; ......... } rfbClientRec, *rfbClientPtr; So in the following two files the flag is handled: 1. "auth.c": Here the flag will be set after a successful authentication: /* * rfbAuthProcessClientMessage is called when the client sends its * authentication response. */ void rfbAuthProcessClientMessage (cl) rfbClientPtr cl; { ........... -> /* authentication OK was sent so set the flag */ -> cl->rfb_authentication_sent = TRUE; cl->state = RFB_INITIALISATION; } 2. "rfbserver.c": Here the flag will be initialised by the following funktion /* * rfbNewClient is called when a new connection has been made by whatever * means. */ static rfbClientPtr rfbNewClient(sock) int sock; { rfbProtocolVersionMsg pv; rfbClientPtr cl; BoxRec box; struct sockaddr_in addr; unsigned int addrlen = sizeof(struct sockaddr_in); .............. cl->host = strdup(inet_ntoa(addr.sin_addr)); cl->state = RFB_PROTOCOL_VERSION; --> /* bugfix to prevent server messages */ --> cl->rfb_authentication_sent=FALSE; ............. } And here are the problematic server message functions: /* * rfbSendBell sends a Bell message to all the clients. */ void rfbSendBell() { rfbClientPtr cl, nextCl; rfbBellMsg b; for (cl = rfbClientHead; cl; cl = nextCl) { nextCl = cl->next; b.type = rfbBell; -> /* bugfix preventing server messages to client before authentication result was sent*/ -> if (cl->rfb_authentication_sent==TRUE){ if (WriteExact(cl->sock, (char *)&b, sz_rfbBellMsg) < 0) { rfbLogPerror("rfbSendBell: write"); rfbCloseSock(cl->sock); } -> } } } /* * rfbSendServerCutText sends a ServerCutText message to all the clients. */ void rfbSendServerCutText(char *str, int len) { rfbClientPtr cl, nextCl; rfbServerCutTextMsg sct; for (cl = rfbClientHead; cl; cl = nextCl) { -> /* bugfix preventing server messages to client before authentication result was sent*/ -> if (cl->rfb_authentication_sent==TRUE){ nextCl = cl->next; sct.type = rfbServerCutText; sct.length = Swap32IfLE(len); if (WriteExact(cl->sock, (char *)&sct, sz_rfbServerCutTextMsg) < 0) { rfbLogPerror("rfbSendServerCutText: write"); rfbCloseSock(cl->sock); continue; } if (WriteExact(cl->sock, str, len) < 0) { rfbLogPerror("rfbSendServerCutText: write"); rfbCloseSock(cl->sock); } -> } } } Good luck ! Bernd Mueller *:o) [EMAIL PROTECTED]@realvnc.com am 27.05.2004 13:18:50 Gesendet von: [EMAIL PROTECTED] An: [EMAIL PROTECTED] Kopie: Fax: Thema: RE: Unknown VNC authentication result - BUG! Reason found! The problem of disturbing server messages only occurs while entering the password. Messages befor the connection is opened are lost anyway. So these messages should not be buffered at my opinion. I don't know if there could be an other message than the bell. (The only others I could imagine are messages during a shared session). But in my application I have "ringing" alarm bells in a xterm annoying me while the login! Are there bell messages from a WinVNC-server? (I dont think so, because I've never heard anything from a Windows VNC-server or thes might be realised in the new Verion V4.X) Bernd M|ller "Seak, Teng-Fong" <[EMAIL PROTECTED]>@realvnc.com am 27.05.2004 11:23:15 Gesendet von: [EMAIL PROTECTED] An: <[EMAIL PROTECTED]> Kopie: Fax: Thema: RE: Unknown VNC authentication result - BUG! Reason found! Impressive diagnostics! For those server messages to client before authentification: isn't it better stock these messages in a buffer? In this way, we could avoid loss of possibly important messages, no? > -----Message d'origine----- > De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Envoyi : mercredi 26 mai 2004 16:30 > @ : [EMAIL PROTECTED] > Objet : Unknown VNC authentication result - BUG! Reason found! > > After the detection (with ethereal) of the troubling 0x02-byte while the > authentication procedure, I had a closer look at the RFB protocol. > (I should have looked there much earIyer!) > In the documentation of the server to client messages the disturbing one > byte-message was shortly discribed as BELL! > > Now the reason of the "Unknown VNC authentication result" -error is > obvious: > At my linux vncserver I wrote a simple application (surveying the server > logfile) that launches a Xdisplay process with a bell (!), everytime the > server is connected. > > So when the bell message was send during the authentication procedure, > while the client was wating for the authentication result, teh client read > the bell message as first byte (0x02) and the rest three bytes (0x000000) > from of the correct authentication result so that the result was > interpreted as 0x02000000 (dec. 33554432 ). > If there were further bell messages during the authentication procedure, > the return value varied like dercribed in my posts before. > > I think this effect is only at the unix-server (Xserver). > I wrote a very simple test program to make the error obvious (bell.c): > > #include <stdlib.h> > #include <stdio.h> > > void main(){ > while (1){ > fprintf(stdout,"\a--bell--\n"); > fflush(stdout); > sleep(1); > } > } > > If you compile this program (eg "cc bell.c -o bell") and execute it in a > xterm of the Xvnc-Screen and disconnect, > you should have problems to reconnect with report of the "Unknown VNC > authentication result" > > So the bug will be simply fixed: the server messages to the new client > must be suppressed until the authentication result aof the server was > sent. > > In my application of the vncserver there is a process with a alarm bell. > It would be nice if the bug is fixed soon. > So long I have to quiet my bells on the server (hi!) > > Bernd M|ller *:o) > _______________________________________________ > VNC-List mailing list > [EMAIL PROTECTED] > To remove yourself from the list visit: > http://www.realvnc.com/mailman/listinfo/vnc-list _______________________________________________ VNC-List mailing list [EMAIL PROTECTED] To remove yourself from the list visit: http://www.realvnc.com/mailman/listinfo/vnc-list _______________________________________________ VNC-List mailing list [EMAIL PROTECTED] To remove yourself from the list visit: http://www.realvnc.com/mailman/listinfo/vnc-list _______________________________________________ VNC-List mailing list [EMAIL PROTECTED] To remove yourself from the list visit: http://www.realvnc.com/mailman/listinfo/vnc-list