+1. Would be a good contribution! Reuven
On Fri, Sep 27, 2019 at 5:33 PM Brian Hulette <bhule...@google.com> wrote: > +1, thank you! > > Note In my Row Coder PR I added a new section for "Additional Standard > Coders" - i.e. coders that have a URN, but aren't required for a new > runner/sdk to implement the beam model: > https://github.com/apache/beam/pull/9188/files#diff-f0d64c2cfc4583bfe2a7e5ee59818ae2R646 > > I think this would belong there as well, assuming that is a distinction we > want to make. > > On Fri, Sep 27, 2019 at 5:22 PM Thomas Weise <t...@apache.org> wrote: > >> +1 for adding the coder >> >> Please also add a test here: >> https://github.com/apache/beam/blob/master/model/fn-execution/src/main/resources/org/apache/beam/model/fnexecution/v1/standard_coders.yaml >> >> >> On Fri, Sep 27, 2019 at 5:17 PM Chad Dombrova <chad...@gmail.com> wrote: >> >>> Are there any dissenting votes to making a BooleanCoder a standard >>> (portable) coder? >>> >>> I'm happy to make a PR to implement a BooleanCoder in python (and to add >>> the Java BooleanCoder to the ModelCoderRegistrar) if everyone agrees that >>> this is useful. >>> >>> -chad >>> >>> >>> On Fri, Sep 27, 2019 at 3:32 PM Robert Bradshaw <rober...@google.com> >>> wrote: >>> >>>> I think boolean is useful to have. What I'm more skeptical of is >>>> adding standard types for variations like UnsignedInteger16, etc. that >>>> don't have natural representations in all languages. >>>> >>>> On Fri, Sep 27, 2019 at 2:46 PM Brian Hulette <bhule...@google.com> >>>> wrote: >>>> > >>>> > Some more context from an offline discussion I had with +Robert >>>> Bradshaw a while ago: We both agreed all of the coders listed in BEAM-7996 >>>> should be implemented in Python, but didn't come to a conclusion on whether >>>> or not they should actually be _standard_ coders, versus just being >>>> implicitly standard as part of row coder. >>>> > >>>> > On Fri, Sep 27, 2019 at 2:29 PM Kenneth Knowles <k...@apache.org> >>>> wrote: >>>> >> >>>> >> Yes, noted here: >>>> https://github.com/apache/beam/pull/9188/files#diff-f0d64c2cfc4583bfe2a7e5ee59818ae2R678 >>>> and that links to https://issues.apache.org/jira/browse/BEAM-7996 >>>> >> >>>> >> Kenn >>>> >> >>>> >> On Fri, Sep 27, 2019 at 12:57 PM Reuven Lax <re...@google.com> >>>> wrote: >>>> >>> >>>> >>> Java has one, implemented as a byte coder. My guess is that nobody >>>> has gotten around to implementing it yet for portability. >>>> >>> >>>> >>> On Fri, Sep 27, 2019 at 12:44 PM Chad Dombrova <chad...@gmail.com> >>>> wrote: >>>> >>>> >>>> >>>> Hi all, >>>> >>>> It seems a bit unfortunate that there isn’t a portable way to >>>> serialize a boolean value. >>>> >>>> >>>> >>>> I’m working on porting my external PubsubIO PR over to use the >>>> improved schema-based external transform API in python, but because of this >>>> limitation I can’t use boolean values. For example, this fails: >>>> >>>> >>>> >>>> ReadFromPubsubSchema = typing.NamedTuple( >>>> >>>> 'ReadFromPubsubSchema', >>>> >>>> [ >>>> >>>> ('topic', typing.Optional[unicode]), >>>> >>>> ('subscription', typing.Optional[unicode]), >>>> >>>> ('id_label', typing.Optional[unicode]), >>>> >>>> ('with_attributes', bool), >>>> >>>> ('timestamp_attribute', typing.Optional[unicode]), >>>> >>>> ] >>>> >>>> ) >>>> >>>> >>>> >>>> It fails because coders.get_coder(bool) returns the non-portable >>>> pickle coder. >>>> >>>> >>>> >>>> In the short term I can hack something into the external transform >>>> API to use varint coder for bools, but this kind of hacky approach to >>>> portability won’t work in scenarios where round-tripping is required >>>> without user intervention. In other words, in python it is not uncommon to >>>> test if x is True, in which case the integer 1 would fail this test. All of >>>> that is to say that a BooleanCoder would be a convenient way to ensure the >>>> proper type is used everywhere. >>>> >>>> >>>> >>>> So, I was just wondering why it’s not there? Are there concerns >>>> over whether booleans are universal enough to make part of the portability >>>> standard? >>>> >>>> >>>> >>>> -chad >>>> >>>
