If only UTF-8 messages are being parsed successfully, then that sounds like
a clue to me that you probably need to base64-encode the serialized
message. I have only limited experience with JavaScript, but from what I
understand, JavaScript strings can only represent Unicode and not raw
binary data. So if you need to store raw binary in a JavaScript string then
you probably want to base64-encode it (and then decode it on the other side
before parsing the message).
On Thu, Feb 17, 2022 at 11:28 AM Csaba Szigeti <sziget...@gmail.com> wrote:
> I am really sorry. Posting again with correct formatting this time :
>
> I would like to pass a protobuf message from Javascript to a C++
> WebAssembly module.
>
> I do receive, and can successfully parse the simpler protobuf messages on
> the C++ side, however, protobuf is failing to parse the less-simple
> protobuf messages.
>
> Here is the code that I am using :
>
> ```javascript
> goog.module('protobufMessageGenerator');
> goog.require('proto.my_proto.MyMessage');
>
> function generateMyMessage() {
> var msg = new proto.my_proto.MyMessage()
> var internal = new proto.my_proto.MyMessageInternal()
> internal.setAccentColor("#CC1D24")
>
> internal.setLogo("iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAYHSURBVHgB7ZhrbBRVFMf/s7vdR9ul7ba09CXQoqVFqcHwFhMQEh4GDSF+ggQhIT6CCUaN/YCaRsWo6QfAKAkvJVEILzECBSE8CyG0lfAoBQu0ttvndrfbZZ8zu9czd22FD+Lc4cNK3F8yO5mZvXfO/95zzzl3JEbgMcaAx5ykgESTFJBokgISTVJAovk/C2BgoTB8decRbG6GXmJuDzy1xxDp7oYeTNADlU+D586ja/v3QCiEjBdmwTZ+vFAXUBT0/Lib+rmASGsbitdVwTxqFEQRnwEyvmfrDjg3fYOSTz6GOTcXaRPGqxOimZjfj9aqdQjfakHp+mpElQhSS0ugCyZEjLkOHGSNk6azcGsbU1xu1jBpGou0tfNnmlAUdmfte+za3IUs6vMxz+FadnnqLMZCIaYHoRmQO7vRUf0ZshfNh3l0Mbx1ddyFTFmZ9FTS1Ifn2HG49/+MgjVvwJCWhv5DR2DOdgBmM/SgXQC5SNfmLaBRh2PRAm6w9+RpGK1WSCkal1I0it5t38E0KhcZL84GCwZx72I9DKmp0DoA0CuABYLw0GgZcxxIe7qCGxO8fkPE9RG62wpfw2XYp0zmRoc6nVD6+iBJ+oxX0Swg3N5OL+uHJT8fEr2chSOI9Ll4KGUkRguBphswyBHYSsdCHXG5qxdSNIYozYRQFNAjQBnwUPiIwWhP59dMlhFTjSchUYoqDydunNzv5mfTiAzuMUPtot5B+lGgB80CJIsVjKZaNZxfG6gpHYxCoNzv+cd20YEByJ4BLkKyWPi9mByO92Ey8rM84OWDoaJQQnN+/S29R5sgzQIs+XkwWMyQXf18tFRjDPY0soZRFu35618PuoHS04ur8xejq2aDai5sY0bzQYh09/L/mjKz+HXMH0CMZsF7/CSal6+C2eEYFvdvaM7EppEjYZtQzheuarC5oBC2okL4nZ2QnR3wNzRi8Mw5hNvaaVZkmCjB2WdMQ8bk53Cv8TeyN0YJr5yCQA78l69w4dbCAgqfKUBERseXNQjRmirdWANrWZnmoKQ9jBqNyF2xjPu899RZqBnE+uQ4/qizZiPaqtcjeKcV7n0/IUKi7GS4+3AtAtea+GIHtVPjfvb8eQhcuYbQ7bswUfw3ZWdDouwuDwziqc2bqCQpg0hQEkhkDI6FC5A2Yypce/fzUbOOHcOfpE+bgvK9P6Do/bUwkquF2zsQIffJfmkhLDTKjNZAy5p30LxiFXwUiVS6t27ng6LOoopj8SIY0tMhGo0EijkJEmXLkq8+x83lK+H89AsMnDrN7xtsajJLgbm4CBUH98Bz9AS5UhtCLbdhKS6G//cWPFH1LlIKC7lve44eR+sH62BMS4WPZkNFCQSG3yMEEyQWi7F79Y2svrSCddds4OeW19eweC10/xEn4nSyW6vfVBsO9cAP1649rKFsIvuj6kPWUDSOdW7ZxvQgXo2Sg/qvXod95nTkrV5JkSST54e/R2/oiJOSl4cx1R/ddyv+PNjSgpxXlyCLXIeH5+E+xBDeDzAKeb2796CUXAk2Gy/EDOkPqWXIz1Pych+4FSLjXb8cQcWunXxfAIMEoy0VehCegcEzZ2GhEGkrj29g1NxgKSgQ6qN3x0445s2hivYJGGhdMUqI5ryR0IOwgP5DtRjx/HQ+aipKhGqb8jLN7RktVveJU8h+ZbF6RVlZJgGU5ErGQg9iAsK0B25sROqzlXFj6Foml0qvrITW8OdvvknOxpBaXg7V7cJd3TyhWSiC6UFsQ0N7AcXrHY7dQQqTNnpxSkE+tIY/dfFaqb1kjddFvkv1yJwzm0K0BXoQEqD4vFQHMb4w1b2xa98BZM2dA5HUqXh9GBKr7jHcR39FztIl0ItQFDLZ7ZCozumjKKQWc94LF1Hw9lsiXcCUMQLBpmZ4Dh+B9+x5pD8zAbYKwS8a9yOUNWSFNS97jV0qHscaKyfzhCYK1UnUdgqrLyplTS8vZbTPYI+CpP6ICFbre2/dBdgrJyKFrwW1uVj6V0uMwI2byJg1E8bMDDwKwgL+ayQ/7iaapIBEkxSQaJICEk1SQKL5E+3sNu+yTCFZAAAAAElFTkSuQmCC")
> msg.setInternal(internal)
> return msg
> }
> ```
>
> Using the google-closure-compiler, I compile this protobuf message
> generator to pure Javascript.
> I use this code to pass the message to the C++ WebAssembly module :
>
> ```javascript
> function sendMessage() {
>
> Module.self().setMyMessage(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary())
> }
> sendMessage()
> ```
>
> I receive the message on the C++ side using this code :
>
> ```cpp
> void JavascriptPublicInterface::setMyMessage(const QString &myMessage)
> {
> if (_myMessage != myMessage)
> {
> _myMessage = myMessage;
> emit myMessageChanged();
> }
> }
>
> void JavascriptPublicInterface::setMyMessageEMS(const std::string
> &myMessage)
> {
> setMyMessage(QString::fromStdString(myMessage));
> }
>
> //...
>
> void startListeningToMyMessage(std::function<void(euf_proto::MyMessage
> myMessage)> listener)
> {
> const auto changedListener = [listener]() {
> const auto newMyMessageString =
> JavascriptPublicInterface::self()->myMessage();
> my_proto::MyMessage myMessagePb;
> if
> (myMessagePb.ParseFromString(newMyMessageString.toStdString()))
> {
> qDebug() << "C++ : Successfully parsed new
> MyMessage: " << newMyMessageString;
> }
> else
> {
> qDebug() << "C++ : Failed to parse new MyMessage:
> " << newMyMessageString;
> }
>
> qDebug() << "MyMessage received: " <<
> QString::fromStdString(myMessagePb.DebugString());
>
> listener(myMessagePb);
> };
> if (!QObject::connect(JavascriptPublicInterface::self(),
> &JavascriptPublicInterface::myMessageChanged, changedListener))
> {
> throw std::runtime_error("Failed to connect to
> JavascriptPublicInterface myMessageChanged");
> }
> }
> ```
>
> In the Javascript console I see : "C++ : Failed to parse new MyMessage:
> ..."
>
> I guess that it can be related to the fact that Javascript is using UTF-16
> and C++ is using UTF-8 encoding (because the messages that are UTF-8 only
> can be successfully parsed), so I tried to fix the sending and receiving
> like this:
>
> ```javascript
> //For more info see :
> https://emscripten.org/docs/api_reference/emscripten.h.html?highlight=stringtoutf8
> function convertUtf16ToUtf8(input) {
> var uint8Arr = input
> var jsString = String.fromCharCode.apply(null, uint8Arr)
> var lengthBytes = lengthBytesUTF8(jsString)+1
> var stringOnWasmHeap = _malloc(lengthBytes)
> stringToUTF8(jsString, stringOnWasmHeap, lengthBytes)
> return stringOnWasmHeap
> }
>
> function sendMessage() {
> var myMessage =
> convertUtf16ToUtf8(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary())
> Module.self().setMyMessage(myMessage)
> _free(myMessage)
> }
> sendMessage()
> ```
>
> And I modified the parsing on the C++ side like this :
>
> ```cpp
> void JavascriptPublicInterface::setMyMessageEMS(const int pointer)
> {
> const char* msg = (char*)pointer;
> setMyMessage(QString::fromStdString(msg));
> }
> ```
>
> But I still get this error : "C++ : Failed to parse new MyMessage: ..."
>
> How could I fix this?
>
> (I am using protobuf 3.19.1 (same version on both the Javascript and the
> C++ side).)
>
> On Thursday, February 17, 2022 at 8:25:26 PM UTC+1 Csaba Szigeti wrote:
>
>> I would like to pass a protobuf message from Javascript to a C++
>> WebAssembly module.
>>
>> I do receive, and can successfully parse the simpler protobuf messages on
>> the C++ side, however, protobuf is failing to parse the less-simple
>> protobuf messages.
>>
>> Here is the code that I am using :
>> ```
>> goog.module('protobufMessageGenerator');
>> goog.require('proto.my_proto.MyMessage'); function generateMyMessage() {
>> var msg = new proto.my_proto.MyMessage() var internal = new
>> proto.my_proto.MyMessageInternal() internal.setAccentColor("#CC1D24")
>> internal.setLogo("iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAYHSURBVHgB7ZhrbBRVFMf/s7vdR9ul7ba09CXQoqVFqcHwFhMQEh4GDSF+ggQhIT6CCUaN/YCaRsWo6QfAKAkvJVEILzECBSE8CyG0lfAoBQu0ttvndrfbZZ8zu9czd22FD+Lc4cNK3F8yO5mZvXfO/95zzzl3JEbgMcaAx5ykgESTFJBokgISTVJAovk/C2BgoTB8decRbG6GXmJuDzy1xxDp7oYeTNADlU+D586ja/v3QCiEjBdmwTZ+vFAXUBT0/Lib+rmASGsbitdVwTxqFEQRnwEyvmfrDjg3fYOSTz6GOTcXaRPGqxOimZjfj9aqdQjfakHp+mpElQhSS0ugCyZEjLkOHGSNk6azcGsbU1xu1jBpGou0tfNnmlAUdmfte+za3IUs6vMxz+FadnnqLMZCIaYHoRmQO7vRUf0ZshfNh3l0Mbx1ddyFTFmZ9FTS1Ifn2HG49/+MgjVvwJCWhv5DR2DOdgBmM/SgXQC5SNfmLaBRh2PRAm6w9+RpGK1WSCkal1I0it5t38E0KhcZL84GCwZx72I9DKmp0DoA0CuABYLw0GgZcxxIe7qCGxO8fkPE9RG62wpfw2XYp0zmRoc6nVD6+iBJ+oxX0Swg3N5OL+uHJT8fEr2chSOI9Ll4KGUkRguBphswyBHYSsdCHXG5qxdSNIYozYRQFNAjQBnwUPiIwWhP59dMlhFTjSchUYoqDydunNzv5mfTiAzuMUPtot5B+lGgB80CJIsVjKZaNZxfG6gpHYxCoNzv+cd20YEByJ4BLkKyWPi9mByO92Ey8rM84OWDoaJQQnN+/S29R5sgzQIs+XkwWMyQXf18tFRjDPY0soZRFu35618PuoHS04ur8xejq2aDai5sY0bzQYh09/L/mjKz+HXMH0CMZsF7/CSal6+C2eEYFvdvaM7EppEjYZtQzheuarC5oBC2okL4nZ2QnR3wNzRi8Mw5hNvaaVZkmCjB2WdMQ8bk53Cv8TeyN0YJr5yCQA78l69w4dbCAgqfKUBERseXNQjRmirdWANrWZnmoKQ9jBqNyF2xjPu899RZqBnE+uQ4/qizZiPaqtcjeKcV7n0/IUKi7GS4+3AtAtea+GIHtVPjfvb8eQhcuYbQ7bswUfw3ZWdDouwuDwziqc2bqCQpg0hQEkhkDI6FC5A2Yypce/fzUbOOHcOfpE+bgvK9P6Do/bUwkquF2zsQIffJfmkhLDTKjNZAy5p30LxiFXwUiVS6t27ng6LOoopj8SIY0tMhGo0EijkJEmXLkq8+x83lK+H89AsMnDrN7xtsajJLgbm4CBUH98Bz9AS5UhtCLbdhKS6G//cWPFH1LlIKC7lve44eR+sH62BMS4WPZkNFCQSG3yMEEyQWi7F79Y2svrSCddds4OeW19eweC10/xEn4nSyW6vfVBsO9cAP1649rKFsIvuj6kPWUDSOdW7ZxvQgXo2Sg/qvXod95nTkrV5JkSST54e/R2/oiJOSl4cx1R/ddyv+PNjSgpxXlyCLXIeH5+E+xBDeDzAKeb2796CUXAk2Gy/EDOkPqWXIz1Pych+4FSLjXb8cQcWunXxfAIMEoy0VehCegcEzZ2GhEGkrj29g1NxgKSgQ6qN3x0445s2hivYJGGhdMUqI5ryR0IOwgP5DtRjx/HQ+aipKhGqb8jLN7RktVveJU8h+ZbF6RVlZJgGU5ErGQg9iAsK0B25sROqzlXFj6Foml0qvrITW8OdvvknOxpBaXg7V7cJd3TyhWSiC6UFsQ0N7AcXrHY7dQQqTNnpxSkE+tIY/dfFaqb1kjddFvkv1yJwzm0K0BXoQEqD4vFQHMb4w1b2xa98BZM2dA5HUqXh9GBKr7jHcR39FztIl0ItQFDLZ7ZCozumjKKQWc94LF1Hw9lsiXcCUMQLBpmZ4Dh+B9+x5pD8zAbYKwS8a9yOUNWSFNS97jV0qHscaKyfzhCYK1UnUdgqrLyplTS8vZbTPYI+CpP6ICFbre2/dBdgrJyKFrwW1uVj6V0uMwI2byJg1E8bMDDwKwgL+ayQ/7iaapIBEkxSQaJICEk1SQKL5E+3sNu+yTCFZAAAAAElFTkSuQmCC")
>> msg.setInternal(internal) return msg }
>> ```
>> Using the google-closure-compiler, I compile this protobuf message
>> generator to pure Javascript. I use this code to pass the message to the
>> C++ WebAssembly module :
>> ```
>> function sendMessage() {
>> Module.self().setMyMessage(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary())
>> } sendMessage()
>> ```
>> I receive the message on the C++ side using this code :
>> ```
>> void JavascriptPublicInterface::setMyMessage(const QString &myMessage) {
>> if (_myMessage != myMessage) { _myMessage = myMessage; emit
>> myMessageChanged(); } } void
>> JavascriptPublicInterface::setMyMessageEMS(const std::string &myMessage) {
>> setMyMessage(QString::fromStdString(myMessage)); } //... void
>> startListeningToMyMessage(std::function<void(euf_proto::MyMessage
>> myMessage)> listener) { const auto changedListener = [listener]() { const
>> auto newMyMessageString = JavascriptPublicInterface::self()->myMessage();
>> my_proto::MyMessage myMessagePb; if
>> (myMessagePb.ParseFromString(newMyMessageString.toStdString())) { qDebug()
>> << "C++ : Successfully parsed new MyMessage: " << newMyMessageString; }
>> else { qDebug() << "C++ : Failed to parse new MyMessage: " <<
>> newMyMessageString; } qDebug() << "MyMessage received: " <<
>> QString::fromStdString(myMessagePb.DebugString()); listener(myMessagePb);
>> }; if (!QObject::connect(JavascriptPublicInterface::self(),
>> &JavascriptPublicInterface::myMessageChanged, changedListener)) { throw
>> std::runtime_error("Failed to connect to JavascriptPublicInterface
>> myMessageChanged"); } }
>> ```
>>
>>
>> In the Javascript console I see : "C++ : Failed to parse new MyMessage:
>> ..."
>>
>> I guess that it can be related to the fact that Javascript is using
>> UTF-16 and C++ is using UTF-8 encoding (because the messages that are UTF-8
>> only can be successfully parsed), so I tried to fix the sending and
>> receiving like this:
>>
>> ```
>> //For more info see :
>> https://emscripten.org/docs/api_reference/emscripten.h.html?highlight=stringtoutf8
>> function convertUtf16ToUtf8(input) { var uint8Arr = input var jsString =
>> String.fromCharCode.apply(null, uint8Arr) var lengthBytes =
>> lengthBytesUTF8(jsString)+1 var stringOnWasmHeap = _malloc(lengthBytes)
>> stringToUTF8(jsString, stringOnWasmHeap, lengthBytes) return
>> stringOnWasmHeap } function sendMessage() { var myMessage =
>> convertUtf16ToUtf8(module$contents$protobufMessageGenerator_generateMyMessage().serializeBinary())
>> Module.self().setMyMessage(myMessage) _free(myMessage) } sendMessage()
>> ```
>> And I modified the parsing on the C++ side like this :
>> ```
>> void JavascriptPublicInterface::setMyMessageEMS(const int pointer) {
>> const char* msg = (char*)pointer;
>> setMyMessage(QString::fromStdString(msg)); }
>> ```
>>
>> But I still get this error : "C++ : Failed to parse new MyMessage: ..."
>>
>> How could I fix this?
>>
>> (I am using protobuf 3.19.1 (same version on both the Javascript and the
>> C++ side).)
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/protobuf/9ce5f69e-b6de-4627-b8e1-71b68da574b3n%40googlegroups.com
> <https://groups.google.com/d/msgid/protobuf/9ce5f69e-b6de-4627-b8e1-71b68da574b3n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/CADqAXr7L_Kv%2Bm5Kte5Y%3DHeO9hLzobzr77umbYD64hhXsy4GZ5w%40mail.gmail.com.
