Hey all, The effort is complete: all transforms have been brought in accordance with the style guide and the JIRAs are closed!
In nearly all cases the fixes introduced small but backward-incompatible changes, but always with a simple migration path, and I believe the Beam API surface is overall much better for it. For example, there are no more IOs that use Coder's as their primary way of interpreting binary data; no more ugly Bound/Unbound classes; no more IOs exposing their Source or Sink API directly (instead of packaging as PTransform); the code is cleaner and shorter (due to AutoValue and a more principled distinction between factory methods and builder methods) and there are a lot more canonical examples of how to write transforms for future authors, now that every transform shipped with the SDK is a canonical example :) The only thing remaining is adjusting the website documentation, release notes, etc. - I'll work on this tomorrow. On Thu, Apr 20, 2017 at 10:55 PM Jean-Baptiste Onofré <[email protected]> wrote: > No problem ;) > > Happy to review if needed ;) > > Regards > JB > > On 04/21/2017 07:50 AM, Eugene Kirpichov wrote: > > Guys, apologies, but I already have Kinesis in review, and Pubsub ready > for > > review. I'm afraid there's not much left for volunteers to take on right > > now. > > > > On Thu, Apr 20, 2017 at 10:47 PM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > >> Cool, I gonna take a look on PubSub later today (I would like to finish > >> CassandraIO, HDFS refactoring and Spark 2 support first ;)). > >> > >> Regards > >> JB > >> > >> On 04/21/2017 06:03 AM, tarush grover wrote: > >>> Hi, > >>> > >>> I can take kinesis one. > >>> > >>> Regards, > >>> Tarush > >>> > >>> > >>> On Thu, 20 Apr 2017 at 11:18 AM, Jean-Baptiste Onofré <[email protected] > > > >>> wrote: > >>> > >>>> Gonna take a look on the pending IOs. > >>>> > >>>> Thanks ! > >>>> Regards > >>>> JB > >>>> > >>>> On 04/19/2017 10:05 PM, Eugene Kirpichov wrote: > >>>>> A few more knocked down > >>>>> - I finished Map/FlatMap, XML, TFRecordIO > >>>>> - I'm working on CountingInput; it's nontrivial. > >>>>> - Reuven is working on Text/Avro > >>>>> - @peay is working on removing coders from KafkaIO > >>>>> > >>>>> Kinesis and PubsubIO remain; of these, Kinesis is the easier one. > >>>>> > >>>>> Any takers? > >>>>> > >>>>> On Fri, Apr 7, 2017 at 10:47 PM Jean-Baptiste Onofré < > [email protected]> > >>>>> wrote: > >>>>> > >>>>>> Hi Eugene, > >>>>>> > >>>>>> thanks for the update. I'm volunteer to tackle some those IOs (and > >> make > >>>>>> them > >>>>>> conform with PTransform style guide). I'm pretty sure other people > >> will > >>>>>> jump on ;) > >>>>>> > >>>>>> Regards > >>>>>> JB > >>>>>> > >>>>>> On 04/08/2017 12:20 AM, Eugene Kirpichov wrote: > >>>>>>> Hey all, > >>>>>>> > >>>>>>> More progress has been made and we're nearing completion. ParDo, > >>>>>> BigQueryIO > >>>>>>> and Window are fixed; Map/FlatMapElements are in review. > >>>>>>> > >>>>>>> The remaining unclaimed ones are all IOs of some form, and here's a > >>>> list. > >>>>>>> I've marked them all as "starter" in JIRA. > >>>>>>> > >>>>>>> XML - https://issues.apache.org/jira/browse/BEAM-1914 > >>>>>>> TFRecordIO (Tensorflow) - > >>>>>> https://issues.apache.org/jira/browse/BEAM-1913 > >>>>>>> KinesisIO - https://issues.apache.org/jira/browse/BEAM-1428 > >>>>>>> PubsubIO - https://issues.apache.org/jira/browse/BEAM-1415 > >>>>>>> CountingInput - https://issues.apache.org/jira/browse/BEAM-1414 > >>>>>>> > >>>>>>> https://github.com/apache/beam/pull/2149 , which fixes BigQueryIO, > >> is > >>>> a > >>>>>>> good model to follow when taking these on, as well as e.g. > >>>>>>> https://github.com/apache/beam/pull/1927 (TextIO) > >>>>>>> > >>>>>>> These are all actually easy to fix, but need volunteers (I do not > >> have > >>>>>> time > >>>>>>> to fix all of these myself, but happy to be a reviewer - @jkff). > >>>>>>> Let's finish this up in time for the first Beam stable release, so > >>>> Beam's > >>>>>>> stable API surface is consistent and polished! > >>>>>>> > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Jean-Baptiste Onofré > >>>> [email protected] > >>>> http://blog.nanthrax.net > >>>> Talend - http://www.talend.com > >>>> > >>> > >> > >> -- > >> Jean-Baptiste Onofré > >> [email protected] > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
