Re: [go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[Jonathan Amsterdam]

I changed the encoding so that the previous generated code is no longer 
necessary. Encoded data now carries all the information needed to decode 
struct fields correctly, even if fields have been added and reordered.


On Sunday, January 24, 2021 at 8:29:58 AM UTC-5 Jonathan Amsterdam wrote:

> Thanks for the suggestions. I created https://github.com/jba/codec/pull/1 
> to address them. You can comment in more detail there if you'd like.
>
> On Wed, Jan 20, 2021 at 12:13 PM roger peppe  wrote:
>
>> On Wed, 20 Jan 2021 at 13:31, Jonathan Amsterdam  
>> wrote:
>>
>>> The encoding scheme is described briefly in the README[0] and the 
>>> code[1].
>>>
>>> To answer your two specific questions, interfaces are represented as a 
>>> pair (typeNumber, value) where typeNumber maps to a registered type. (Like 
>>> gob, types must be registered.) Structs are represented as: startCode 
>>> (fieldNumber value)* endCode. The field numbers are assigned by the 
>>> generator.
>>>
>>
>> It might be good to be more explicit about how the field numbers are 
>> assigned. From a brief experiment, it seems like there's not a deterministic
>> relationship between a struct and its wire representation, and instead 
>> the generated field numbers are taken from the generated code file
>> when it's present. So ISTM that any user of this must be very careful to 
>> preserve that file, and realise that it's not OK to generate
>> the codec code for a type independently.
>>
>> I'd also suggest that it would be good to fully document the syntax and 
>> explain the trade-offs of this format and when
>> it might or might not be appropriate to use.
>>
>> One other question: how are the type numbers maintained as stable 
>> entities over time?
>>
>>   cheers,
>> rog.
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ad1d2cc1-acfc-4278-93b7-05e09bb8b8aan%40googlegroups.com.

Re: [go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[Jonathan Amsterdam]

Thanks for the suggestions. I created https://github.com/jba/codec/pull/1
to address them. You can comment in more detail there if you'd like.

On Wed, Jan 20, 2021 at 12:13 PM roger peppe  wrote:

> On Wed, 20 Jan 2021 at 13:31, Jonathan Amsterdam 
> wrote:
>
>> The encoding scheme is described briefly in the README[0] and the code[1].
>>
>> To answer your two specific questions, interfaces are represented as a
>> pair (typeNumber, value) where typeNumber maps to a registered type. (Like
>> gob, types must be registered.) Structs are represented as: startCode
>> (fieldNumber value)* endCode. The field numbers are assigned by the
>> generator.
>>
>
> It might be good to be more explicit about how the field numbers are
> assigned. From a brief experiment, it seems like there's not a deterministic
> relationship between a struct and its wire representation, and instead the
> generated field numbers are taken from the generated code file
> when it's present. So ISTM that any user of this must be very careful to
> preserve that file, and realise that it's not OK to generate
> the codec code for a type independently.
>
> I'd also suggest that it would be good to fully document the syntax and
> explain the trade-offs of this format and when
> it might or might not be appropriate to use.
>
> One other question: how are the type numbers maintained as stable entities
> over time?
>
>   cheers,
> rog.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEymQsQ%3D4WkteR1t6o5TXfzp0egR%3DvsGii8On_mKBU%2B_YRM%2BfA%40mail.gmail.com.

Re: [go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[roger peppe]

On Wed, 20 Jan 2021 at 13:31, Jonathan Amsterdam 
wrote:

> The encoding scheme is described briefly in the README[0] and the code[1].
>
> To answer your two specific questions, interfaces are represented as a
> pair (typeNumber, value) where typeNumber maps to a registered type. (Like
> gob, types must be registered.) Structs are represented as: startCode
> (fieldNumber value)* endCode. The field numbers are assigned by the
> generator.
>

It might be good to be more explicit about how the field numbers are
assigned. From a brief experiment, it seems like there's not a deterministic
relationship between a struct and its wire representation, and instead the
generated field numbers are taken from the generated code file
when it's present. So ISTM that any user of this must be very careful to
preserve that file, and realise that it's not OK to generate
the codec code for a type independently.

I'd also suggest that it would be good to fully document the syntax and
explain the trade-offs of this format and when
it might or might not be appropriate to use.

One other question: how are the type numbers maintained as stable entities
over time?

  cheers,
rog.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgaciaEnx8r7169%3D8f9YPpFav9JmGf0wc%2BwGZEx72NTTO8tQ%40mail.gmail.com.

Re: [go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[Jonathan Amsterdam]

The encoding scheme is described briefly in the README[0] and the code[1].

To answer your two specific questions, interfaces are represented as a pair
(typeNumber, value) where typeNumber maps to a registered type. (Like gob,
types must be registered.) Structs are represented as: startCode
(fieldNumber value)* endCode. The field numbers are assigned by the
generator.

The schema is implicit. The model is protobufs, where there is just enough
information in the wire protocol to skip values, but you need something
external to interpret types and struct field numbers.


[0] https://github.com/jba/codec#encoding-scheme
[1] https://github.com/jba/codec/blob/v0.3.1/codecapi/codecapi.go#L200


On Tue, Jan 19, 2021 at 10:50 AM roger peppe  wrote:

> This is interesting, thanks! Is there a full description of the encoding
> somewhere? (e.g. how are structs represented? what about interface values,
> etc? is the schema implicit or sent on the wire?)
>   cheers,
> rog.
>
> On Tue, 19 Jan 2021 at 14:59, Jonathan Amsterdam 
> wrote:
>
>> Uses code generation for fast encoding and decoding of Go values to
>> bytes. Handles sharing and cycles too.
>>
>> https://pkg.go.dev/github.com/jba/codec
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/2cc71201-ddbf-4524-88ba-7d0875072d80n%40googlegroups.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEymQsRAJKsu2Fe5PwHvTyYW2d1L7ksiJnGfcmPQDohfXtw-yg%40mail.gmail.com.

Re: [go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[roger peppe]

This is interesting, thanks! Is there a full description of the encoding
somewhere? (e.g. how are structs represented? what about interface values,
etc? is the schema implicit or sent on the wire?)
  cheers,
rog.

On Tue, 19 Jan 2021 at 14:59, Jonathan Amsterdam 
wrote:

> Uses code generation for fast encoding and decoding of Go values to bytes.
> Handles sharing and cycles too.
>
> https://pkg.go.dev/github.com/jba/codec
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2cc71201-ddbf-4524-88ba-7d0875072d80n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgachxdFvGX_WYk0t-Lw7SnhM-Hd%3DhWj2gDy191ZTaU-sQOg%40mail.gmail.com.

[go-nuts] [ANN] github.com/jba/codec, a fast encoder for Go

[Jonathan Amsterdam]

Uses code generation for fast encoding and decoding of Go values to bytes. 
Handles sharing and cycles too.

https://pkg.go.dev/github.com/jba/codec


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2cc71201-ddbf-4524-88ba-7d0875072d80n%40googlegroups.com.