Thanks, I think I will choose the JSON option. Inflated size can be tackled with LZ4 compression.
On Tue, Aug 12, 2014 at 4:18 PM, Sergey Lyubka <[email protected]> wrote: > Yes, base64-encoding binary blobs and wrapping resulting text into JSON is a > common practice. Many popular RESTful services do that, for example GitHub > API encodes file text that way. > > Base64 encoding has exactly 4/3 overhead, i.e. output text is 33% larger > then the input data. That determinism is a nice property of base64. > > Using base64 still means that binary blob is transferred. Representation of > floats in binary form is standartized by IEEE (2 formats for single and > double precision). > > So two basic choices are: > > 1. Send a binary blob (base64-encoded or not) of floats. Also send a hint > (in a HTTP header for example), which float format is used (e.g. "floats are > IEEE 754 doubles, with no holes between them", something like "double > arr[N]" in C language would produce). The receiving side would have all > information to unpack the blob properly. Pro: easy to do for C/C++, no need > to transform if both sides use the same standard for floats. Cons: extra > care about exact binary format. > > 2. Transform floats into text, like to JSON or some other format. Pro: > independence from the binary format, easy to debug by a human. Cons: both > sides need to transform, transfer size could be inflated. > > > > > On Mon, Aug 11, 2014 at 2:00 PM, jeff shanab <[email protected]> wrote: >> >> I use base64 encoding within json for arbitray binary data. Images, >> encrypted data, C++ POD objects (ok that one is rare) >> >> >> On Mon, Aug 11, 2014 at 8:51 AM, Sergey Lyubka <[email protected]> wrote: >>> >>> On Mon, Aug 11, 2014 at 1:18 PM, Jamie Vicary <[email protected]> >>> wrote: >>>> >>>> Hi, thanks for your reply. JSON is a plain-text format, right? I would >>>> worry about the increased size of the data packet. >>>> >>>> Is there a platform-safe way to transmit binary data? >>> >>> >>> There are plenty. >>> All of them transform binary data to the platform independent data using >>> some sort of encoding. >>> JSON is great cause it has relatively small overhead, and is fully human >>> readable. >>> >>> If you're 100% sure that sending and receiving sides are on of the same >>> platform, sending binary blob is an option too -- although it is still a >>> risky play (a software could be built with different compiler options, etc). >>> >>> If you're talking about doubles (which are 8 bytes on common >>> architectures), JSON-encoding might be actually even tighter then the >>> original array. Consider array of zeroes: JSON would be [0,0,0,...], >>> each number taking 2 bytes instead of 8 bytes. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "mongoose-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/mongoose-users. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mongoose-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/mongoose-users. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "mongoose-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/mongoose-users/gKs85nB4b-M/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/mongoose-users. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "mongoose-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/mongoose-users. For more options, visit https://groups.google.com/d/optout.
