That is not true. Java serialization works similarly. You can hook it do that
you send the metadata once during connect, and then encode the data.so no a new
connection only needs the metadata and can decode further stream messages. You
may need framing resets to simplify things and reduce the overhead depending on
the hierarchy of “objects and references”
> On Dec 23, 2020, at 6:09 PM, Artur Vianna <lordhowen...@gmail.com> wrote:
>
>
> Before using gob was using encoding.BinaryMarshaler, but that would mean the
> user of the api would need to implement a MarshalBinary for every type, which
> is kind of cumbersome.
>
> An option might be to let the user choose gob, BinaryMarshaler or Json etc to
> best fit the use case, but that takes the simplicity of only gobs away.
>
> I did try your solution to reset the client too but i'm getting inconsistent
> behaviour, in one server it works and in another it doesn't ("corrupted data
> or unknown type"). I think synching the server and client will be error
> prone, while also increasing the use of network.
>
> The easiest solution now is to label the package for ≤32 players and test
> alternative encodings that keep the API as clean as with gob. I took a look
> at flatbuffers but it will be cumbersome for the user to create the builders,
> and i really wanted the simplest possible API.
>
> Maybe i should try UDP Broadcast too and see what happens, probably chaos :D
>
>
>> On Wed, 23 Dec 2020, 20:36 Axel Wagner, <axel.wagner...@googlemail.com>
>> wrote:
>> No, it wouldn't. Because the encoder keeps state about which
>> type-information it already sent and wouldn't sent it again - causing the
>> client to be unable to decode. So you'd also need a new encoder on the
>> server. And at that point, you're back to the status quo, with one encoder
>> per client and the duplication of encoding effort.
>>
>> A solution would, perhaps, be if the gob API would give you a way to send
>> *only* the type-info (so you could, if the connection breaks, create a new
>> encoder, send all the type info, and *then* multicast the encoded values).
>> But it doesn't.
>>
>> Really, I think it's far less effort to just use a different format (and I
>> would maintain that even json would probably be fine) than trying to make
>> this work with gob :)
>>
>> On Thu, Dec 24, 2020 at 12:20 AM Matthew Zimmerman <mzimmer...@gmail.com>
>> wrote:
>>> If you would "reset" each client with a new decoder each time you make a
>>> new encoder, everything should work fine. Just would take some
>>> coordination.
>>>
>>>> On Wed, Dec 23, 2020, 6:08 PM Artur Vianna <lordhowen...@gmail.com> wrote:
>>>> I will look into other protocols, although for now the performance is not
>>>> an issue in servers with less than 100 players.
>>>>
>>>> The problem with io.MultiWriter is that a player inside the group may
>>>> disconnect or a new player may come in. This means a new io.MultiWriter
>>>> must be created each time you dispatch, since the group may have changed
>>>> in the meantime. This would also need a new encoder and then the
>>>> "duplicate type received" happens.
>>>>
>>>>> On Wed, 23 Dec 2020, 19:58 'Axel Wagner' via golang-nuts,
>>>>> <golang-nuts@googlegroups.com> wrote:
>>>>> The issue with that approach is that gob keeps state about which
>>>>> type-information it still has to send. So if you encode to, say, a
>>>>> bytes.Buffer, it would encode all type-info on every message sent, which
>>>>> is a significant overhead.
>>>>> TBH, I don't understand why `io.MultiWriter` wouldn't work. It would be
>>>>> helpful to see the code that causes the error message OP is seeing.
>>>>>
>>>>> However, really, gob just doesn't provide a good API for this sorta
>>>>> thing, as mentioned. The format itself is fine, but the stateful
>>>>> connection means that if you don't want to write *exactly* the same data
>>>>> in exactly the same order to all connections (which can perform poorly
>>>>> and lead to operational problems with timeouts and intermittently lost
>>>>> connections and the like), you are going to have a bad time.
>>>>> You honestly would fare better with a full-fledged RPC framework such as
>>>>> gRPC. Or, if you don't want the overhead of its IDL, even json. Because
>>>>> at least the "encode once, send to each client" is trivial to solve with
>>>>> that.
>>>>>
>>>>> But, that's just my 2¢ :)
>>>>>
>>>>>> On Wed, Dec 23, 2020 at 11:43 PM Robert Engels <reng...@ix.netcom.com>
>>>>>> wrote:
>>>>>> Yes, that is why you need to create your own protocol. Use the gob to
>>>>>> encode to a buffer then send the buffer on each of the connections using
>>>>>> your protocol.
>>>>>>
>>>>>>>> On Dec 23, 2020, at 4:19 PM, Matthew Zimmerman <mzimmer...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>
>>>>>>> My understanding is that gob streams are unique.
>>>>>>>
>>>>>>> From https://golang.org/pkg/encoding/gob/
>>>>>>> "A stream of gobs is self-describing. Each data item in the stream is
>>>>>>> preceded by a specification of its type, expressed in terms of a small
>>>>>>> set of predefined types."
>>>>>>>
>>>>>>> In my own rudimentary understanding/terms, it sends the struct
>>>>>>> definition once, then uses shorthand for it afterwards. E.g, how many
>>>>>>> bytes and what order. If you mix and match streams that send
>>>>>>> definitions in different orders, then chaos ensues.
>>>>>>>
>>>>>>> I think this is why people use other encoders in the scenario you're
>>>>>>> taking about. For a one to one stream gob works great, but in this
>>>>>>> multi scenario I don't think it does.
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>> On Wed, Dec 23, 2020, 5:07 PM Artur Vianna <lordhowen...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> If i create a bytes.Buffer and a gob.Encoder, each time i write to a
>>>>>>>> group of connections i get "duplicate type received" and if i try and
>>>>>>>> reuse the encoder, i get "corrupted data" and "unknown type".
>>>>>>>> It seems i can't use both net.Conn.Write and gob.Encoder.Encode in the
>>>>>>>> same connection, i will try always encoding to a buffer in both
>>>>>>>> unicast and multicast like you said and report if it works.
>>>>>>>>
>>>>>>>>> On Wed, 23 Dec 2020, 18:49 Robert Engels, <reng...@ix.netcom.com>
>>>>>>>>> wrote:
>>>>>>>>> You need to encode once to a byte array then send the byte array on
>>>>>>>>> each connection.
>>>>>>>>>
>>>>>>>>>>> On Dec 23, 2020, at 3:45 PM, meera <lordhowen...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am trying to create a package for game servers using gob. The
>>>>>>>>>> current approach is an application level multicasting over TCP,
>>>>>>>>>> having a gob encoder and decoder for each player connection, and set
>>>>>>>>>> up a goroutine to receive and another to dispatch for each one. The
>>>>>>>>>> code for the dispatcher is here. But summarized, it simply receives
>>>>>>>>>> data from a channel and encodes it.
>>>>>>>>>>
>>>>>>>>>> The problem is that if i want to transmit a single piece of data to
>>>>>>>>>> all players, this piece of data is encoded again and again for each
>>>>>>>>>> connection, doing duplicate work. With less than 100 players this is
>>>>>>>>>> not a problem, but with 300+ my machine is at almost 100% usage and
>>>>>>>>>> the profiler shows that most of it is spent on encoding. Here's the
>>>>>>>>>> issue on github.
>>>>>>>>>>
>>>>>>>>>> I tryied using a io.MultiWriter but gob complains of duplicate type
>>>>>>>>>> received, and if i try to write the raw bytes from the gob.Encoder i
>>>>>>>>>> get corrupted data. An option is using UDP Broadcasting but since
>>>>>>>>>> gob expects a stream, i'm affraid i will run into unexpected
>>>>>>>>>> behavior when packets start to be lost and fragmented.
>>>>>>>>>>
>>>>>>>>>> Does gob expect a single encoder and decoder to own the stream? Not
>>>>>>>>>> allowing two encoders on the server for one decoder on the client?
>>>>>>>>>> --
>>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>>> Groups "golang-nuts" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>>> send an email to golang-nuts+unsubscr...@googlegroups.com.
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/0562184e-bbcc-44c9-adbf-37e8d5411c7cn%40googlegroups.com.
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "golang-nuts" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/CAE%3DAWBXN46idvqUbCsGs%2BZbZt%2BCj4MowJ4Ozj3_U9_6-68OWDw%40mail.gmail.com.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "golang-nuts" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/golang-nuts/214752B6-2666-4892-A9B8-E4BC4127FD42%40ix.netcom.com.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups
>>>>> "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGWtULh8Q3Jqu_gq5m5Si4PvJ1oVSZY7DVhu%3D6hGK83bg%40mail.gmail.com.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/CAE%3DAWBUsmp2sbiEh%3D3z0cC9EhjLig%2B8exXyA05YngBJ-tsC_uA%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAE%3DAWBU%3DZBR8O9na44vsXCg9NXrH9tdo5xxuBjYqMOYoGU4N0w%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/9FF32BF3-8C8C-48DE-A024-787E529701A9%40ix.netcom.com.
