Hi Yonathan, On Friday 13 February 2015 15:22:55 Yonathan wrote: > We would like to contribute this patch for bug > https://www.virtualbox.org/ticket/12611 under the MIT license. It > fixes a crash in the VNC Extension of VirtualBoxVM when a VNC client > connects while VirtualBox is still restoring saved state. The fix is > to allocate a frame buffer of arbitrary size within > VRDEEnableConnections that the VNC thread will use until VRDEResize. > > Background: This is the standard sequence of events at VM startup: > > 1. VirtualBox calls extension entry point VRDECreateServer. > > 2. VirtualBox calls extension entry point VRDEEnableConnections, which > listens for connections and starts VNC server thread. > > 3. VirtualBox calls extension entry point VRDEResize. Extension calls > the callback VRDECallbackFramebufferQuery to obtain the new size and > screen buffer, and allocates a frame buffer according to this size. > > 4. VirtualBox calls extension entry point VRDEUpdate whenever the > pixels in the screen buffer change, and extension will copy the pixels > from screen buffer to frame buffer. > > There can be a significant time between steps 2 and 3 (e.g., when > VirtualBox has to restore saved state). If a VNC client happens to > connect during that gap, then the VNC server tries to copy pixel data > from the frame buffer to the client. Since the frame buffer was NULL > at this point, this resulted in SEGV in the VNC event loop thread. > Sample stack trace: > > Exception Type: EXC_BAD_ACCESS (SIGSEGV) > Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000 > > Thread 41 Crashed: > 0 libsystem_c.dylib 0x00007fff8bc9c9e7 memmove$VARIANT$sse42 + 31 > 1 libvncserver.0.dylib 0x000000010d3b5c73 rfbTranslateNone + 59 > 2 libvncserver.0.dylib 0x000000010d3b497d rfbSendRectEncodingHextile + 4630 > 3 libvncserver.0.dylib 0x000000010d3ae07e rfbSendFramebufferUpdate + 3314 > 4 libvncserver.0.dylib 0x000000010d3a92fb clientOutput + 320 > 5 libsystem_c.dylib 0x00007fff8bcaf742 _pthread_start + 327 > 6 libsystem_c.dylib 0x00007fff8bc9c181 thread_start + 13
thank you for this patch. I've just applied a slightly modified version of your patch to the repository. The next 4.3.x maintenance release will contain the fix as well. Kind regards, Frank -- Dr.-Ing. Frank Mehnert | Software Development Director, VirtualBox ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt, Germany Hauptverwaltung: Riesstr. 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603 Geschäftsführer: Jürgen Kunz Komplementärin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher _______________________________________________ vbox-dev mailing list [email protected] https://www.virtualbox.org/mailman/listinfo/vbox-dev
