thank you for your info. I will adopt your feedback.

about PORT, seems VBox support port range. is there any way that VBox
will manage the port for different VM? Or we have to pick in my code?

I attached WIP which is good enough to have a try.
== Instructions ==
1. Put the attached files into source tree at /src/VBox/ExtPacks/VNC/
2. kmk under the folder. You will get a VBoxVRDP.so. install it to the
folder where your VBoxRT.so files are.
3. run command to enable this: VBoxManage setproperty vrdeextpack
Built-in-VBoxVRDP
4. enable vrde of your vm. or VBoxHeadless --startvm <name> --vrde on
5. use vnc client to connect to 5900 port

== Features==
1. All features of current VNC code in VBoxHeadless
2. Support mouse cursor. you can get your native Windows Cursor instead of X
3. implement DrawSolidRect order which may give very little
performance improvement.

==todo==
1. rework keyboard translate function.
2. add more order support if we can. ( i suspend VNC can only support
very little since no client side cache.)
3. authtication

Please report bug or give suggestions to me.

Again, thank the dev team's support.

On Thu, Feb 10, 2011 at 6:48 PM, Vitali Pelenjow
<[email protected]> wrote:
> Good questions.
>
> 1. You are right, the HVRDESERVER is a pointer. It is supposed to be an
> address of some internal data or object, which is used by the VRDE server.
> For VBox it is an opaque pointer value.
>
> 2. VRDE_SP_NETWORK_BIND_PORT is needed to tell VBox, which port is actually
> used by the server. BTW, this is a 32 value, so you need to replace
> sizeof(int) with sizeof (uint32_t).
> Here is how it is expected to work:
> * VRDECreateServer must start the server and create the listening socket. If
> this fails, call VRDE_SP_NETWORK_BIND_PORT with port value == 0. Note that
> at this point the listening socket exists but the server must refuse all
> connection attempts.
> * If the server created successfully, then VBox will call
> VRDEEnableConnections. The server must call VRDE_SP_NETWORK_BIND_PORT with
> the actual port value.
> * VRDEDestroy must call VRDE_SP_NETWORK_BIND_PORT with port value == ~0
> (0xFFFFFFFF).
>
> 3. ClientLogon is called by the server when the client username/password is
> received, and allow the server to ask VBox if the client should be accepted.
> If ClientLogon returns an error, then the server must drop the client
> connection. ClientConnected is called after the connection with the client
> is completely established. So ClientConnected is actually called after
> ClientLogon.
> For example in RDP:
> * initial connection and encryption;
> * logon packet -> the server calls ClientLogon;
> * license exchange and demand active;
> * now the client is ready and the server calls ClientConnected.
>
> 4. Lock and Unlock were designed to allow exclusive access to the
> framebuffer bitmap (they use a mutex).
> But currently the server does not need the exclusive access, because VBox
> makes sure that the buffer is available for read access when a VRDEUpdate is
> called.
> I think that we may change this is the future probably will remove these
> callbacks. This still needs some thinking.
>
> Note: The server must access the framebuffer bitmap only when the VRDEUpdate
> is called. If for example the server has a second thread which access the
> framebuffer this can lead to crashes. For example, to avoid this problem the
> VRDP server uses an intermediate buffer and copies the framebuffer bitmap
> data to the intermediate buffer in VRDEUpdate. The intermediate buffer is
> then used to actually compress and send data to the client.
>
> Note 2:  VRDECallbackFramebufferQuery can return 'false' if the framebuffer
> is not available. I see that VNCServerImpl::VRDEResize does not check the
> return value. The server should send a black image to the client if the
> framebuffer is not available.
>
> Vitali.
>
> Howard Su wrote:
>>
>> Thank you for the support. I need several answers about VRDE from you:
>> 1. what's type of HVRDESERVER? I use it as a void* in my code if you read
>> it. I am not sure if this is a right way.
>> 2. When should I call queryproperty with VRDE_SP_NETWORK_BIND_PORT? Is
>> this something important as part of state transition.
>> 3. when should i call ClientLogon? Is it mandatory? Should it happen right
>> after ClientConnected?
>> 4. What sort of operations require lock framebuffer? It is not clear how
>> to use Lock/Unlock framebuffer callback.
>>
>>
>>
>> On Wed, Feb 9, 2011 at 10:10 PM, Vitali Pelenjow
>> <[email protected] <mailto:[email protected]>> wrote:
>>
>>    Good work. That was the goal of VRDE interface to allow others to
>>    write a remote desktop server.
>>
>>    What kind of state transition you would like to know about?
>>    Is something not working as expected?
>>
>>
>>    Thanks,
>>    Vitali
>>
>>    Howard Su wrote:
>>
>>        I am experimenting a feature to implement VNC protocol server
>>        based on VRDE interface.
>>
>>        Due to lack of document, i am not sure if my understanding to
>>        the interface is correct. Especially, i am not sure the state
>>        transition is understood correctly. If someone in this list
>>        can give me a review, it will help me moving forward.
>>
>>        Current, the patch can work with OSE version as a
>>        "Built-in-VBoxVRDP". I can successfully connect to VRDP with
>>        mouse support.
>>
>>
>>        --         -Howard
>>
>>
>>        _______________________________________________
>>        vbox-dev mailing list
>>        [email protected] <mailto:[email protected]>
>>        http://vbox.innotek.de/mailman/listinfo/vbox-dev
>>
>>
>>
>>
>>
>> --
>> -Howard
>
>



-- 
-Howard
/*
 * Copyright (C) 2011 Howard Su
 *
 */
#define LOG_GROUP LOG_GROUP_VRDP
#include <iprt/log.h>

#include <iprt/asm.h>
#include <iprt/alloca.h>
#include <iprt/ldr.h>
#include <iprt/param.h>
#include <iprt/path.h>
#include <iprt/mem.h>
#include <iprt/stream.h>
#include <iprt/string.h>
#include <iprt/thread.h>
#include <iprt/cpp/utils.h>

#include <VBox/err.h>
#include <VBox/RemoteDesktop/VRDEOrders.h>
#include <VBox/RemoteDesktop/VRDE.h>

#include <rfb/rfb.h>

class VNCServerImpl
{
public:
    VNCServerImpl() 
    {
        RT_ZERO(mCursor);
        mBytesPerLine = 0;
        mFrameBuffer = 0;
        mScreenBuffer = 0;
    }

    int Init(const VRDEINTERFACEHDR *pCallbacks, void *pvCallback);

    VRDEINTERFACEHDR* GetInterface() { return &Entries.header; }

private:
    // VNC Related variables
    RTTHREAD mVncThread;
    rfbScreenInfoPtr mVncServer;
    void *mCallback;
    rfbCursor mCursor;
    unsigned int mBytesPerLine;
    unsigned char *mScreenBuffer;
    unsigned char *mFrameBuffer;
    static DECLCALLBACK(enum rfbNewClientAction) rfbNewClientEvent(struct _rfbClientRec* cl);
    static DECLCALLBACK(void) vncMouseEvent(int buttonMask, int x, int y, rfbClientPtr cl);
    static DECLCALLBACK(int) vncThreadFn(RTTHREAD hThreadSelf, void *pvUser);
    static void vncKeyboardEvent(rfbBool down, rfbKeySym keySym, rfbClientPtr cl);

    static uint32_t RGB2BGR(uint32_t c)
    {
        c = ((c >> 0) & 0xff) << 16 |
            ((c >> 8) & 0xff) << 8 |
            ((c >> 16) & 0xff) << 0;

        return c;
    }

    static VRDEENTRYPOINTS_3 Entries;
    VRDECALLBACKS_3 *mCallbacks;


    static DECLCALLBACK(void) VRDEDestroy(HVRDESERVER hServer);
    static DECLCALLBACK(int) VRDEEnableConnections(HVRDESERVER hServer, bool fEnable);
    static DECLCALLBACK(void) VRDEDisconnect(HVRDESERVER hServer, uint32_t u32ClientId, 
        bool fReconnect);
    static DECLCALLBACK(void) VRDEResize(HVRDESERVER hServer);
    static DECLCALLBACK(void) VRDEUpdate(HVRDESERVER hServer, unsigned uScreenId,
        void *pvUpdate,uint32_t cbUpdate);
    static DECLCALLBACK(void) VRDEColorPointer(HVRDESERVER hServer,
        const VRDECOLORPOINTER *pPointer);
    static DECLCALLBACK(void) VRDEHidePointer(HVRDESERVER hServer);
    static DECLCALLBACK(void) VRDEAudioSamples(HVRDESERVER hServer,
        const void *pvSamples,
        uint32_t cSamples,
        VRDEAUDIOFORMAT format);
    static DECLCALLBACK(void) VRDEAudioVolume(HVRDESERVER hServer,
        uint16_t u16Left,
        uint16_t u16Right);
    static DECLCALLBACK(void) VRDEUSBRequest(HVRDESERVER hServer,
        uint32_t u32ClientId,
        void *pvParm,
        uint32_t cbParm);
    static DECLCALLBACK(void) VRDEClipboard(HVRDESERVER hServer,
        uint32_t u32Function,
        uint32_t u32Format,
        void *pvData,
        uint32_t cbData,
        uint32_t *pcbActualRead);
    static DECLCALLBACK(void) VRDEQueryInfo(HVRDESERVER hServer,
        uint32_t index,
        void *pvBuffer,
        uint32_t cbBuffer,
        uint32_t *pcbOut);
    static DECLCALLBACK(void) VRDERedirect(HVRDESERVER hServer,
        uint32_t u32ClientId,
        const char *pszServer,
        const char *pszUser,
        const char *pszDomain,
        const char *pszPassword,
        uint32_t u32SessionId,
        const char *pszCookie);
    static DECLCALLBACK(void) VRDEAudioInOpen(HVRDESERVER hServer,
        void *pvCtx,
        uint32_t u32ClientId,
        VRDEAUDIOFORMAT audioFormat,
        uint32_t u32SamplesPerBlock);
    static DECLCALLBACK(void) VRDEAudioInClose(HVRDESERVER hServer,
        uint32_t u32ClientId);
};

VRDEENTRYPOINTS_3 VNCServerImpl::Entries = {
    { VRDE_INTERFACE_VERSION_3, sizeof(VRDEENTRYPOINTS_3) },
    VNCServerImpl::VRDEDestroy,
    VNCServerImpl::VRDEEnableConnections,
    VNCServerImpl::VRDEDisconnect,
    VNCServerImpl::VRDEResize,
    VNCServerImpl::VRDEUpdate,
    VNCServerImpl::VRDEColorPointer,
    VNCServerImpl::VRDEHidePointer,
    VNCServerImpl::VRDEAudioSamples,
    VNCServerImpl::VRDEAudioVolume,
    VNCServerImpl::VRDEUSBRequest,
    VNCServerImpl::VRDEClipboard,
    VNCServerImpl::VRDEQueryInfo,
    VNCServerImpl::VRDERedirect,
    VNCServerImpl::VRDEAudioInOpen,
    VNCServerImpl::VRDEAudioInClose
};


/** Destroy the server instance.
*
* @param hServer The server instance handle.
*
* @return IPRT status code.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEDestroy(HVRDESERVER hServer)
{
    uint32_t port = ~0;
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);
    rfbShutdownServer (instance->mVncServer, TRUE);
    instance->mCallbacks->VRDECallbackProperty(
        instance->mCallback, VRDE_SP_NETWORK_BIND_PORT, &(port), sizeof(uint32_t), NULL);
    return;
}


/** The server should start to accept clients connections.
*
* @param hServer The server instance handle.
* @param fEnable Whether to enable or disable client connections.
*                When is false, all existing clients are disconnected.
*
* @return IPRT status code.
*/
DECLCALLBACK(int) VNCServerImpl::VRDEEnableConnections(HVRDESERVER hServer, bool fEnable)
{
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);

    RTStrmPrintf(g_pStdOut, "VNCServerImpl::VRDEEnableConnections\n");

    RTThreadCreate(&instance->mVncThread, VNCServerImpl::vncThreadFn, instance,
        0 /*cbStack*/, RTTHREADTYPE_GUI, 0 /*fFlags*/, "VNC");
    instance->mCallbacks->VRDECallbackProperty(
        instance->mCallback, VRDE_SP_NETWORK_BIND_PORT, &(instance->mVncServer->port), sizeof(uint32_t), NULL);

    return VINF_SUCCESS;
}

/** The server should disconnect the client.
*
* @param hServer     The server instance handle.
* @param u32ClientId The client identifier.
* @param fReconnect  Whether to send a "REDIRECT to the same server" packet to the
*                    client before disconnecting.
*
* @return IPRT status code.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEDisconnect(HVRDESERVER hServer, uint32_t u32ClientId, 
                                                 bool fReconnect)
{
    RTStrmPrintf(g_pStdOut, "VNCServerImpl::VRDEDisconnect\n");
}

/**
* Inform the server that the display was resized.
* The server will query information about display
* from the application via callbacks.
*
* @param hServer Handle of VRDE server instance.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEResize(HVRDESERVER hServer)
{
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);
    VRDEFRAMEBUFFERINFO info;
    int rc = instance->mCallbacks->VRDECallbackFramebufferQuery(instance->mCallback, 0, &info);
    if (!RT_SUCCESS(rc))
    {
        return;
    }

    RTStrmPrintf(g_pStdOut, "VNCServerImpl::VRDEResize to %dx%dx%dbpp\n", info.cWidth, info.cHeight, info.cBitsPerPixel);

    if (info.cBitsPerPixel == 32)
    {
        instance->mBytesPerLine = info.cWidth * 4;
        // Convert RGB (windows/vbox) to BGR(vnc)
        unsigned char* FrameBuffer = (unsigned char*)RTMemAlloc(instance->mBytesPerLine * info.cHeight); // 4 byte per pixel
        for (uint32_t i = 0; i < instance->mBytesPerLine * info.cHeight; i += 4)
        {
            FrameBuffer[i]   = info.pu8Bits[i+2];
            FrameBuffer[i+1] = info.pu8Bits[i+1];
            FrameBuffer[i+2] = info.pu8Bits[i];
        }
        rfbNewFramebuffer(instance->mVncServer, (char*)(FrameBuffer), info.cWidth, info.cHeight, 8, 3, 4);

        void *temp = instance->mFrameBuffer;
        instance->mFrameBuffer = FrameBuffer;
        instance->mScreenBuffer = (unsigned char*)info.pu8Bits;

        if (temp) RTMemFree(temp);
    }
}

/**
* Send a update.
*
* @param hServer   Handle of VRDE server instance.
* @param uScreenId The screen index.
* @param pvUpdate  Pointer to VBoxGuest.h::VRDEORDERHDR structure with extra data.
* @param cbUpdate  Size of the update data.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEUpdate(HVRDESERVER hServer, unsigned uScreenId,
                                             void *pvUpdate,uint32_t cbUpdate)
{
    char* ptr = (char*)pvUpdate;
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);
    VRDEORDERHDR* order = (VRDEORDERHDR*)(ptr);
    ptr += sizeof(VRDEORDERHDR);
    if (order == NULL)
    {
        /* Inform the VRDP server that the current display update sequence is
        * completed. At this moment the framebuffer memory contains a definite
        * image, that is synchronized with the orders already sent to VRDP client.
        * The server can now process redraw requests from clients or initial
        * fullscreen updates for new clients.
        */

    }
    else
    {
        if (sizeof(VRDEORDERHDR) != cbUpdate)
        {
            VRDEORDERCODE *code = (VRDEORDERCODE*)(ptr);
            ptr += sizeof(VRDEORDERCODE);

            switch(code->u32Code)
            {
            case VRDE_ORDER_SOLIDRECT:
                {
                    VRDEORDERSOLIDRECT *solidrect = (VRDEORDERSOLIDRECT*)ptr;
                    rfbFillRect(instance->mVncServer, solidrect->x, solidrect->y, 
                        solidrect->x+solidrect->w, solidrect->y+solidrect->h, RGB2BGR(solidrect->rgb));
                    return;
                }
                //@todo: more order
            }
        }

        uint32_t joff = order->y * instance->mBytesPerLine + order->x * 4;
        for (uint32_t j = joff; j < joff + order->h * instance->mBytesPerLine; j += instance->mBytesPerLine)
        {
            for (uint32_t i = j; i < j + order->w * 4; i += 4)
            {
                instance->mFrameBuffer[i]   = instance->mScreenBuffer[i+2];
                instance->mFrameBuffer[i+1] = instance->mScreenBuffer[i+1];
                instance->mFrameBuffer[i+2] = instance->mScreenBuffer[i];
            }
        }
        rfbMarkRectAsModified(instance->mVncServer, order->x, order->y, order->x+order->w, order->y+order->h);

    }
}


/**
* Set the mouse pointer shape.
*
* @param hServer  Handle of VRDE server instance.
* @param pPointer The pointer shape information.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEColorPointer(HVRDESERVER hServer,
                                                   const VRDECOLORPOINTER *pPointer)
{
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);

    instance->mCursor.width = pPointer->u16Width;
    instance->mCursor.height = pPointer->u16Height;

    unsigned char* mem = (unsigned char*)RTMemAlloc(pPointer->u16Width * pPointer->u16Height * 4);
    unsigned char* mem0 = mem;

    unsigned char* maskmem = (unsigned char*)RTMemAlloc(pPointer->u16Width * pPointer->u16Height);
    unsigned char* maskmem0 = maskmem;

    unsigned char* mask = (unsigned char*)pPointer + sizeof(VRDECOLORPOINTER);

    for(int i = pPointer->u16Height - 1; i >= 0 ; i--)
    {
        for(uint16_t j = 0; j < pPointer->u16Width/8; j ++)
        {
            *maskmem = ~(*(mask+ i * (pPointer->u16Width / 8) + j));
            *maskmem++;
        }
    }
    unsigned char* color = (unsigned char*)pPointer + sizeof(VRDECOLORPOINTER) + pPointer->u16MaskLen;
    for(int i = pPointer->u16Height - 1; i >= 0 ; i--)
    {
        for(uint16_t j = 0; j < pPointer->u16Width; j ++)
        {
            // put the color value;
            *(mem++) = *(color + (i * pPointer->u16Width *3 + j * 3 + 2));
            *(mem++) = *(color + (i * pPointer->u16Width *3 + j * 3 + 1));
            *(mem++) = *(color + (i * pPointer->u16Width *3 + j * 3));
            *(mem++) = 0xff;
        }
    }

    instance->mCursor.xhot = pPointer->u16HotX;
    instance->mCursor.yhot = pPointer->u16HotY;

    unsigned char* oldMem = instance->mCursor.richSource;
    unsigned char* oldMask = instance->mCursor.mask;
    instance->mCursor.richSource = mem0;
    instance->mCursor.mask = maskmem0;

    rfbSetCursor(instance->mVncServer, &instance->mCursor);

    if (oldMem) RTMemFree(oldMem);
    if (oldMask) RTMemFree(oldMask);
}

/**
* Hide the mouse pointer.
*
* @param hServer Handle of VRDE server instance.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEHidePointer(HVRDESERVER hServer)
{
    VNCServerImpl *instance = (VNCServerImpl*)(hServer);

    rfbSetCursor(instance->mVncServer, NULL);
}

/**
* Queues the samples to be sent to clients.
*
* @param hServer    Handle of VRDE server instance.
* @param pvSamples  Address of samples to be sent.
* @param cSamples   Number of samples.
* @param format     Encoded audio format for these samples.
*
* @note Initialized to NULL when the application audio callbacks are NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEAudioSamples(HVRDESERVER hServer,
                                                   const void *pvSamples,
                                                   uint32_t cSamples,
                                                   VRDEAUDIOFORMAT format)
{
}

/**
* Sets the sound volume on clients.
*
* @param hServer    Handle of VRDE server instance.
* @param left       0..0xFFFF volume level for left channel.
* @param right      0..0xFFFF volume level for right channel.
*
* @note Initialized to NULL when the application audio callbacks are NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEAudioVolume(HVRDESERVER hServer,
                                                  uint16_t u16Left,
                                                  uint16_t u16Right)
{
}

/**
* Sends a USB request.
*
* @param hServer      Handle of VRDE server instance.
* @param u32ClientId  An identifier that allows the server to find the corresponding client.
*                     The identifier is always passed by the server as a parameter
*                     of the FNVRDEUSBCALLBACK. Note that the value is the same as
*                     in the VRDESERVERCALLBACK functions.
* @param pvParm       Function specific parameters buffer.
* @param cbParm       Size of the buffer.
*
* @note Initialized to NULL when the application USB callbacks are NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEUSBRequest(HVRDESERVER hServer,
                                                 uint32_t u32ClientId,
                                                 void *pvParm,
                                                 uint32_t cbParm)
{
}

/**
* Called by the application when (VRDE_CLIPBOARD_FUNCTION_*):
*   - (0) guest announces available clipboard formats;
*   - (1) guest requests clipboard data;
*   - (2) guest responds to the client's request for clipboard data.
*
* @param hServer     The VRDE server handle.
* @param u32Function The cause of the call.
* @param u32Format   Bitmask of announced formats or the format of data.
* @param pvData      Points to: (1) buffer to be filled with clients data;
*                               (2) data from the host.
* @param cbData      Size of 'pvData' buffer in bytes.
* @param pcbActualRead Size of the copied data in bytes.
*
* @note Initialized to NULL when the application clipboard callbacks are NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEClipboard(HVRDESERVER hServer,
                                                uint32_t u32Function,
                                                uint32_t u32Format,
                                                void *pvData,
                                                uint32_t cbData,
                                                uint32_t *pcbActualRead)
{
}

/**
* Query various information from the VRDE server.
*
* @param hServer   The VRDE server handle.
* @param index     VRDE_QI_* identifier of information to be returned.
* @param pvBuffer  Address of memory buffer to which the information must be written.
* @param cbBuffer  Size of the memory buffer in bytes.
* @param pcbOut    Size in bytes of returned information value.
*
* @remark The caller must check the *pcbOut. 0 there means no information was returned.
*         A value greater than cbBuffer means that information is too big to fit in the
*         buffer, in that case no information was placed to the buffer.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEQueryInfo(HVRDESERVER hServer,
                                                uint32_t index,
                                                void *pvBuffer,
                                                uint32_t cbBuffer,
                                                uint32_t *pcbOut)
{

    RTStrmPrintf(g_pStdOut, "VNCServerImpl::VRDEQueryInfo\n");

}


/**
* The server should redirect the client to the specified server.
*
* @param hServer       The server instance handle.
* @param u32ClientId   The client identifier.
* @param pszServer     The server to redirect the client to.
* @param pszUser       The username to use for the redirection.
*                      Can be NULL.
* @param pszDomain     The domain. Can be NULL.
* @param pszPassword   The password. Can be NULL.
* @param u32SessionId  The ID of the session to redirect to.
* @param pszCookie     The routing token used by a load balancer to
*                      route the redirection. Can be NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDERedirect(HVRDESERVER hServer,
                                               uint32_t u32ClientId,
                                               const char *pszServer,
                                               const char *pszUser,
                                               const char *pszDomain,
                                               const char *pszPassword,
                                               uint32_t u32SessionId,
                                               const char *pszCookie)
{
}

/**
* Audio input open request.
*
* @param hServer      Handle of VRDE server instance.
* @param pvCtx        To be used in VRDECallbackAudioIn.
* @param u32ClientId  An identifier that allows the server to find the corresponding client.
* @param audioFormat  Preferred format of audio data.
* @param u32SamplesPerBlock Preferred number of samples in one block of audio input data.
*
* @note Initialized to NULL when the VRDECallbackAudioIn callback is NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEAudioInOpen(HVRDESERVER hServer,
                                                  void *pvCtx,
                                                  uint32_t u32ClientId,
                                                  VRDEAUDIOFORMAT audioFormat,
                                                  uint32_t u32SamplesPerBlock)
{
}

/**
* Audio input close request.
*
* @param hServer      Handle of VRDE server instance.
* @param u32ClientId  An identifier that allows the server to find the corresponding client.
*
* @note Initialized to NULL when the VRDECallbackAudioIn callback is NULL.
*/
DECLCALLBACK(void) VNCServerImpl::VRDEAudioInClose(HVRDESERVER hServer,
                                                   uint32_t u32ClientId)
{
}



int VNCServerImpl::Init(const VRDEINTERFACEHDR *pCallbacks,
                        void *pvCallback)
{
    if (pCallbacks->u64Version == VRDE_INTERFACE_VERSION_3)
    {
        mCallbacks = (VRDECALLBACKS_3*)pCallbacks;
        mCallback = pvCallback;
    }
    else if (pCallbacks->u64Version == VRDE_INTERFACE_VERSION_1)
    {
        // @todo: this is incorrect and it will cause crash if client call unsupport func.
        mCallbacks = (VRDECALLBACKS_3*)pCallbacks;
        mCallback = pvCallback;

        // since they are same in order, let's just change header
        Entries.header.u64Version = VRDE_INTERFACE_VERSION_1;
        Entries.header.u64Size = sizeof(VRDEENTRYPOINTS_1);
    }
    else
        return VERR_VERSION_MISMATCH;

    // query server for the framebuffer
    VRDEFRAMEBUFFERINFO info;
    int rc = mCallbacks->VRDECallbackFramebufferQuery(mCallback, 0, &info);

    mVncServer = rfbGetScreen(0, NULL, info.cWidth, info.cHeight, 8, 3, 4);
    mVncServer->serverFormat.redShift = 16;
    mVncServer->serverFormat.greenShift = 8;
    mVncServer->serverFormat.blueShift = 0;
    mVncServer->screenData = (void*)this;
    mVncServer->port = 5900;
    mVncServer->desktopName = "VBOXVNC";
    rfbInitServer(mVncServer);
    mVncServer->newClientHook = rfbNewClientEvent;
    mVncServer->kbdAddEvent = vncKeyboardEvent;
    //  mVncServer->kbdReleaseAllKeys = vncReleaseKeysEvent;
    mVncServer->ptrAddEvent = vncMouseEvent;

    return VINF_SUCCESS;
}


void VNCServerImpl::vncKeyboardEvent(rfbBool down, rfbKeySym keycode, rfbClientPtr cl)
{
    VNCServerImpl *instance = static_cast<VNCServerImpl*>(cl->screen->screenData);
    VRDEINPUTSCANCODE point;

    /* Conversion table for key code range 32-127 (which happen to equal the ASCII codes)
    * The values in the table differ slightly from the actual scancode values that will be sent,
    * values 0xe0?? indicate that a 0xe0 scancode will be sent first (extended keys), then code ?? is sent
    * values 0x01?? indicate that the shift key must be 'down', then ?? is sent
    * values 0x00?? or 0x?? indicate that the shift key must be 'up', then ?? is sent
    * values 0x02?? indicate that the shift key can be ignored, and scancode ?? is sent
    * This is necessary because the VNC protocol sends a shift key sequence, but also
    * sends the 'shifted' version of the characters. */
    static unsigned codes_low[] =
    {
        //Conversion table for VNC key code range 32-127
        0x39, 0x02, 0x28, 0x04, 0x05, 0x06, 0x08, 0x28, 0x0a, 0x0b, 0x09, 0x0d, 0x29, 0x0c, 0x34, 0x35, //space, !"#$%&'()*+`-./
        0x0b, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x27, 0x27, 0x33, 0x0d, 0x34, 0x35, //0123456789:;<=>?@
        0x03, 0x1e, 0x30, 0x2e, 0x20, 0x12, 0x21, 0x22, 0x23, 0x17, 0x24, 0x25, 0x26, 0x32, 0x31, 0x18, 
        0x19, 0x10, 0x13, 0x1f, 0x14, 0x16, 0x2f, 0x111, 0x2d, 0x15, 0x2c, //A-Z
        0x1a, 0x2b, 0x1b, 0x07, 0x0c, 0x29, //[\]^_`
        0x1e, 0x30, 0x2e, 0x20, 0x12, 0x21, 0x22, 0x23, 0x17, 0x24, 0x25, 0x26, 0x32, 0x31, 0x18, 0x19, 0x10, 0x13, 0x1f, 0x14, 0x16, 0x2f, 0x11, 0x2d, 0x15, 0x2c, //a-z
        0x1a, 0x2b, 0x1b, 0x29 //{|}~
    };

    int code = -1;
    if (keycode < 32)
    {
        //ASCII control codes.. unused..
    }
    else if (keycode < 127)
    {
        //DEL is in high area
        code = codes_low[keycode - 32];
    }
    else if ((keycode & 0xFF00) != 0xFF00)
    {
    }
    else
    {
        switch(keycode)
        {
            /*Numpad keys - these have to be implemented yet
            Todo: numpad arrows, home, pageup, pagedown, end, insert, delete
            65421 Numpad return

            65450 Numpad *
            65451 Numpad +
            65453 Numpad -
            65454 Numpad .
            65455 Numpad /
            65457 Numpad 1
            65458 Numpad 2
            65459 Numpad 3

            65460 Numpad 4
            65461 Numpad 5
            65462 Numpad 6
            65463 Numpad 7
            65464 Numpad 8
            65465 Numpad 9
            65456 Numpad 0
            */
        case 65288: code =   0x0e; break; //Backspace
        case 65289: code =   0x0f; break; //Tab

        case 65293: code =   0x1c; break; //Return
            //case 65299: break; Pause/break
        case 65307: code =   0x01; break; //Escape

        case 65360: code = 0xe047; break; //Home
        case 65361: code = 0xe04b; break; //Left
        case 65362: code = 0xe048; break; //Up
        case 65363: code = 0xe04d; break; //Right
        case 65364: code = 0xe050; break; //Down
        case 65365: code = 0xe049; break; //Page up
        case 65366: code = 0xe051; break; //Page down
        case 65367: code = 0xe04f; break; //End

            //case 65377: break; //Print screen
        case 65379: code = 0xe052; break; //Insert

        case 65383: code = 0xe05d; break; //Menu

        case 65470: code =   0x3b; break; //F1
        case 65471: code =   0x3c; break; //F2
        case 65472: code =   0x3d; break; //F3
        case 65473: code =   0x3e; break; //F4
        case 65474: code =   0x3f; break; //F5
        case 65475: code =   0x40; break; //F6
        case 65476: code =   0x41; break; //F7
        case 65477: code =   0x42; break; //F8
        case 65478: code =   0x43; break; //F9
        case 65479: code =   0x44; break; //F10
        case 65480: code =   0x57; break; //F11
        case 65481: code =   0x58; break; //F12

        case 65505: code = 0x2a; break; //Shift (left + right)
        case 65507: code =   0x1d; break; //Left ctrl
        case 65508: code = 0xe01d; break; //Right ctrl
        case 65513: code =   0x38; break; //Left Alt
        case 65514: code = 0xe038; break; //Right Alt
        case 65515: code = 0xe05b; break; //Left windows key
        case 65516: code = 0xe05c; break; //Right windows key
        case 65535: code = 0xe053; break; //Delete
        }
    }

    if (code == -1)
    {
        RTPrintf("VNC unhandled keyboard code: down=%d code=%d\n", down, keycode);
        return;
    }
    if (code > 0xff)
    {
        point.uScancode = (code >> 8) & 0xff;
        instance->mCallbacks->VRDECallbackInput(instance->mCallback, VRDE_INPUT_SCANCODE, &point, sizeof(point));   
    }

    point.uScancode = (code & 0xff) | (down ? 0 : 0x80);
    instance->mCallbacks->VRDECallbackInput(instance->mCallback, VRDE_INPUT_SCANCODE, &point, sizeof(point));
}

void VNCServerImpl::vncMouseEvent(int buttonMask, int x, int y, rfbClientPtr cl)
{
    VNCServerImpl *instance = static_cast<VNCServerImpl*>(cl->screen->screenData);

    VRDEINPUTPOINT point;
    point.uButtons = buttonMask;
    point.x = x;
    point.y = y;
    instance->mCallbacks->VRDECallbackInput(instance->mCallback, VRDE_INPUT_POINT, &point, sizeof(point));
    rfbDefaultPtrAddEvent(buttonMask, x, y, cl);
}

enum rfbNewClientAction VNCServerImpl::rfbNewClientEvent(struct _rfbClientRec* cl)
{
    RTStrmPrintf(g_pStdOut, "VNCServerImpl::rfbNewClientEvent\n");

    VNCServerImpl *instance = static_cast<VNCServerImpl*>(cl->screen->screenData);

    //@todo: we need auth user here

    instance->mCallbacks->VRDECallbackClientConnect(instance->mCallback,  (int)cl->sock);
}


DECLCALLBACK(int) VNCServerImpl::vncThreadFn(RTTHREAD hThreadSelf, void *pvUser)
{
    VNCServerImpl *instance = static_cast<VNCServerImpl*>(pvUser);

    rfbRunEventLoop(instance->mVncServer, -1, FALSE);

    return VINF_SUCCESS;
}


VNCServerImpl *g_RDPServer = 0;

DECLEXPORT(int) VRDECreateServer (const VRDEINTERFACEHDR *pCallbacks,
                                  void *pvCallback,
                                  VRDEINTERFACEHDR **ppEntryPoints,
                                  HVRDESERVER *phServer)
{
    RTStrmPrintf(g_pStdOut, "VRDECreateServer In\n");

    if (!g_RDPServer)
    {
        g_RDPServer = new VNCServerImpl();
    }

    int rc = g_RDPServer->Init(pCallbacks, pvCallback);

    if (RT_SUCCESS(rc))
    {
        *ppEntryPoints = g_RDPServer->GetInterface();
        *phServer = (HVRDESERVER)g_RDPServer;
    }
    RTStrmPrintf(g_pStdOut, "VRDECreateServer Out\n");

    return rc;
}

Attachment: Makefile.kmk
Description: Binary data

_______________________________________________
vbox-dev mailing list
[email protected]
http://vbox.innotek.de/mailman/listinfo/vbox-dev

Reply via email to