LGTM. Thanks for satisfy high performance needs like RealSense.
> -----Original Message----- > From: Crosswalk-dev > [mailto:crosswalk-dev-boun...@lists.crosswalk-project.org] On Behalf Of Hu, > Ningxin > Sent: Friday, July 17, 2015 7:06 PM > To: Liang, Zhenyu; 'crosswalk-dev@lists.crosswalk-project.org' > Subject: Re: [Crosswalk-dev] Intent to Implement: Binary Messaging Interface > for Crosswalk External Extension > > > Yes, certainly. > > I assume it means LGTM from you. :) Thanks! > > > > > Since I'm neither architect nor owner, you need to get LGTMs from them > > :-) > > Ping @Halton for extension, @Lin for Android and @Alexis for overall. Please > take a look. > > Thanks, > -ningxin > > > > > On 7/17/15 17:44 , "Hu, Ningxin" <ningxin...@intel.com> wrote: > > > > >Hi Zhenyu, > > > > > >> > > >> My suggestion is to introduce a new version of current messaging > > >>interface instead of a whole new interface. > > >> > > >> struct XW_MessagingInterface_2 { > > >> > > >> void (*Register)(XW_Extension extension, XW_HandleMessageCallback > > >> handle_message); > > >> void (*PostMessage)(XW_Instance instance, const char* message); > > >> > > >> void (*RegisterBinaryMessageCallback)(XW_Extension extension, > > >> XW_HandleBinaryMessageCallback handle_message); > > >> void (*PostBinaryMessage)(XW_Instance instance, const char* > > >> message, const size_t size); }; > > >> > > > > > >Thanks for your suggestion. > > > > > >XW_MessagingInterface_2 looks better than > > >XW_BinaryMessagingInterface_1, as the binary messaging API is an > > >enhancement to the async messaging interface. And it looks consistent > > >to > > Java API proposal. > > > > > >I will integrate this change into my design. > > > > > >With this change, does it look good you? > > > > > >Thanks, > > >-ningxin > > > > > >> Thanks, > > >> > > >> Zhenyu > > >> > > >> On 7/16/15 15:25 , "Hu, Ningxin" <ningxin...@intel.com> wrote: > > >> > > >> >Intent to Implement: Binary Messaging Interface for Crosswalk > > >> >External Extension > > >> > > > >> >Description: > > >> >Today's Crosswalk external extension API only provides text based > > >> >messaging interface. However, coming use cases require the > > >> >external extension native code to exchange binary data with JavaScript > code. > > >> >For example, an external extension wants to transfer the images > > >> >stream from native to JavaScript in real-time (30FPS at least). > > >> >With today's external extension messaging API, the extension has > > >> >to encode the binary image data into text in native first, posts > > >> >to JavaScript and then decodes from the text to binary data in > > >> >JavaScript. The encoding, decoding (in > > >> >JavaScript!) and increased message size leads to low performance. > > >> >Experiment shows by using base64 to transfer VGA (640x480) images, > > >> >this approach can only deliver less 10 FPS on mainstream PC. > > >> >The binary messaging interface is proposed to added into Crosswalk > > >> >external extension API. It allows native code to exchange binary > > >> >data (a byte array) to JavaScript without unnecessary > > encoding/decoding. > > >> >JavaScript receives an ArrayBuffer object and can access the > > >> >binary data via different Typed Array View. Vice versa, JavaScript > > >> >can send the binary data in an ArrayBuffer object to native, > > >> >native code receives a byte array. > > >> >Prototype shows 3X-8X (binary messaging vs. text messaging) > > >> >performance speedup when transferring different size of data on > > >> >both PC and mobile device. Initial data shows with binary > > >> >messaging interface, an external extension is able to transfer VGA > > >> >images at > > 60FPS on PC. > > >> > > > >> >Affected component: Crosswalk External Extension > > >> > > > >> >Related feature: > > >> >https://crosswalk-project.org/jira/browse/XWALK-4615 > > >> > > > >> >Target release: Crosswalk-17 > > >> > > > >> >Implementation details: > > >> >1. Interface change: > > >> >For Crosswalk External Extension C API, a new interface > > >> >XW_BinaryMessagingInterface_1 will be added. To keep API and ABI > > >> >compatibility with existing external extension, a new file > > >> >(extensions/public/XW_Extension_BinaryMessage.h) will be added to > > >> >contain the XW_BinaryMessagingInterface_1 interface. The > > >> >XW_BinaryMessagingInterface_1 looks like: > > >> > > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >///////////////// > > >> >#ifdef __cplusplus > > >> >extern "C" { > > >> >#endif > > >> > > > >> >#define XW_BINARY_MESSAGING_INTERFACE_1 > > "XW_BinaryMessagingInterface_1" > > >> >#define XW_BINARY_MESSAGING_INTERFACE > > >> >XW_BINARY_MESSAGING_INTERFACE_1 > > >> > > > >> >typedef void (*XW_HandleBinaryMessageCallback)(XW_Instance > > >> >instance, > > >> > > > >> > const char* message, > > >> > > > >> > const size_t size); > > >> > > > >> >struct XW_BinaryMessagingInterface_1 { > > >> > void (*Register)(XW_Extension extension, > > >> > > XW_HandleBinaryMessageCallback > > >> >handle_message); > > >> > > > >> > void (*PostMessage)(XW_Instance instance, const char* message, > > >> >const size_t size); }; > > >> > > > >> >typedef struct XW_BinaryMessagingInterface_1 > > >> >XW_BinaryMessagingInterface; > > >> > > > >> >#ifdef __cplusplus > > >> >} // extern "C" > > >> >#endif > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >//////////////// > > >> > > > >> >If the C extension wants to use binary messaging, it needs to > > >> >query the "XW_BinaryMessagingInterface_1" interface and use the > > >> >PostMessage method to post binary data to JavaScript. On other > > >> >hand, it needs to register a XW_HandleBinaryMessageCallback to > > >> >receive binary data from > > >> JavaScript. > > >> > > > >> >For Crosswalk external extension Android Java API, two methods > > >> >will be added into XWalkExtensionClient interface: > > >> > > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >/////////////// > > >> > > > >> >public void onBinaryMessage(int extensionInstanceID, byte[] > > >> >message); > > >> > > > >> >public final void postBinaryMessage(int instanceID, byte[] > > >> >message); > > >> > > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >////////////// > > >> > > > >> >Java extension which inherits XWalkExtensionClient can use > > >> >postBinaryMesssage to post binary data to JavaScript and > > >> >implements onBinaryMessage to receive binary data from JavaScript. > > >> > > > >> >No new APIs will be added into Crosswalk extension JavaScript API. > > >> >When native external extension posts binary message, the > > >> >JavaScript message listener receives an ArrayBuffer object. > > >> >When JavaScript invokes extension.postMessage with an ArrayBuffer > > >> >object as argument, the native binary message listener is invoked. > > >> > > > >> >This proposal doesn't cover binary messaging support in > > >> >SyncMessage, as there is no obvious use case. > > >> > > > >> >Sample code: > > >> >An external extension wants to transfer image data to JavaScript. > > >> >To transfer structured data, the extension can use simple custom > > >> >binary message format as this sample shows. Or it can leverage > > >> >more sophisticated binary format, e.g. Google's Protocol Buffer > > >> >(https://developers.google.com/protocol-buffers/). > > >> >C++: > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >///////////// > > >> >// image binary format > > >> >// uint32_t width; > > >> >// uint32_t height; > > >> >// char data[width * height * 4]; > > >> > > > >> >static const XW_BinaryMessagingInterface* g_binary_messaging = > > >> > get_interface(XW_BINARY_MESSAGING_INTERFACE); > > >> >..... > > >> >size_t buffer_size = sizeof(uint32_t) + sizeof(uint32_t) + > > >> >width*height*4*sizeof(char); > > >> >char* buffer = new char[buffer_size ]; > > >> >uint32_t* uint32_array = reinterpret_cast<uint32_t*>(buffer); > > >> >uint32_array[0] = image.width(); > > >> >uint32_array[1] = image.height(); > > >> >char* data = buffer + 2 * sizeof(uint32_t); > > >> >image.CopyImageData(data); > > >> >g_binary_messaging->postMessage(instance, > > >> >buffer, buffer_size); delete[] buffer; > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >///////////// > > >> > > > >> >JavaScript: > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >///////////// > > >> >extension.setMessageListener(function(msg) { > > >> > if (msg instaceof ArrayBuffer) { > > >> > var uint32_view = new Uint32Array(msg); > > >> > var width = uint32_view[0]; > > >> > var height = uint32_view[1]; > > >> > var data = new Uint8Array(msg, 8, width*height*4); > > >> > .... > > >> > } > > >> >} > > >> >////////////////////////////////////////////////////////////////// > > >> >// > > >> >/// > > >> >/// > > >> >//////////// > > >> > > > >> >Details: > > >> >It doesn't require to modify Crosswalk Core Extension code > > >> >(XWalkExtensionModule), as V8ValueConverter already supports to > > >> >convert a base::BinaryValue object to V8 ArrayBuffer object and > > >> >vice > > versa. > > >> > > > >> >It mainly requires to modify the external extension components, to > > >> >expose the binary interface and convert the binary data to > > >> >base::BinaryValue and vice versa. > > >> > > > >> >For C external extension, files to be modified: > > >> >extensions/common/xwalk_external_adapter.cc > > >> >extensions/common/xwalk_external_adapter.h > > >> >extensions/common/xwalk_external_extension.cc > > >> >extensions/common/xwalk_external_extension.h > > >> >extensions/common/xwalk_external_instance.cc > > >> >extensions/common/xwalk_external_instance.h > > >> >extensions/extensions.gyp > > >> > > > >> >new file: > > >> >extensions/public/XW_Extension_BinaryMessage.h > > >> > > > >> >For Java external extension, files to be modified: > > >> >app/android/runtime_client/src/org/xwalk/app/runtime/extension/XWa > > >> >lk > > >> >Cor > > >> >eEx > > >> >tensionBridge.java > > >> >app/android/runtime_client/src/org/xwalk/app/runtime/extension/XWa > > >> >lk > > >> >Ext > > >> >ens > > >> >ionClient.java > > >> >app/android/runtime_client/src/org/xwalk/app/runtime/extension/XWa > > >> >lk > > >> >Ext > > >> >ens > > >> >ionContextClient.java > > >> >app/android/runtime_client/src/org/xwalk/app/runtime/extension/XWa > > >> >lk > > >> >Run > > >> >tim > > >> >eExtensionBridge.java > > >> >app/android/runtime_client/src/org/xwalk/app/runtime/extension/XWa > > >> >lk > > >> >Run > > >> >tim > > >> >eExtensionManager.java > > >> >extensions/android/java/src/org/xwalk/core/internal/extensions/XWa > > >> >lk > > >> >Ext > > >> >ens > > >> >ionAndroid.java > > >> >extensions/common/android/xwalk_extension_android.cc > > >> >extensions/common/android/xwalk_extension_android.h > > >> >runtime/android/core_internal/src/org/xwalk/core/internal/XWalkExt > > >> >en > > >> >sio > > >> >nIn > > >> >ternal.java > > >> > > > >> > > > >> >Thanks, > > >> >-ningxin > > >> > > > >> >_______________________________________________ > > >> >Crosswalk-dev mailing list > > >> >Crosswalk-dev@lists.crosswalk-project.org > > >> >https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev > > > > > _______________________________________________ > Crosswalk-dev mailing list > Crosswalk-dev@lists.crosswalk-project.org > https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev _______________________________________________ Crosswalk-dev mailing list Crosswalk-dev@lists.crosswalk-project.org https://lists.crosswalk-project.org/mailman/listinfo/crosswalk-dev
