Good points. With the suggested workflow I think I can support a quarterly newsletter. I'm also happy to get more involvement from others to do this work and we can see what cadence that allows.
On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <k...@apache.org> wrote: > Good points Melissa & Austin. > > - archive in the repo & on the website > - put missed items on the next newsletter, so anyone following sees them > > Kenn > > On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <smar...@apache.org> wrote: > >> I believe there was also a Beam workshop or working session in Warsaw >> last week. >> >> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett < >> whatwouldausti...@gmail.com> wrote: >> >>> +1 for archive in our repo. >>> >>> I do follow the newsletter, but am unlikely to go back and look into the >>> past for changes/updates. >>> >>> Would suggest that things that get missed in one newsletter (a concrete >>> example, Suneel's talks not mentioned in the newsletter) would get >>> published in the next iteration, rather than editing the past 'published' >>> newsletter. Put another way, save editing the past for corrections (typos, >>> things being incorrect). Else, I imagine that I'm unlikely to catch a >>> great announcement that warranted being in the newsletter in the first >>> place. This certainly works better with a regular/frequent release >>> cadence, like we arrived at for version releases (then, if something misses >>> one cut, it is not too big a deal, as the next release is coming soon). >>> >>> >>> >>> >>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <meliss...@google.com> >>> wrote: >>> >>>> >>>> For step #2 (publishing onto the website), I think it would be good to >>>> stay consistent with our existing workflows if possible. Rather than using >>>> an external tool, what about: >>>> >>>> After a google doc newsletter draft is ready, convert it into a >>>> standard markdown file and put it into our GitHub repo, perhaps in a new >>>> newsletter directory in the website community directory [1]. These would be >>>> listed for browsing on a Newsletters page as mentioned in step #4. People >>>> can then just open a PR to add missing things to the pages later, and the >>>> newsletter will be automatically updated on the website through our >>>> standard website workflow. It also avoids the potential issue of the source >>>> google docs disappearing in the future, as they are stored in a community >>>> location. >>>> >>>> [1] https://github.com/apache/beam/tree/master/website/src/community >>>> >>>> >>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rtngu...@google.com> >>>> wrote: >>>> >>>>> I think that would be a great idea to change formats to help with >>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to >>>>> collect and edit, then copy/paste sending the newsletter out directly, >>>>> based >>>>> on an interpretation of this discussion >>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E> >>>>> . >>>>> >>>>> How about this doc->website->Beam site workflow?: >>>>> >>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can >>>>> contribute to the google doc, with soft deadlines for when I'll >>>>> publish. >>>>> 2. I'll publish the doc itself onto a website. >>>>> 3. The newsletter is mailed out in the same way, but now with a >>>>> shareable website link. >>>>> 4. We'll keep an index of archived newsletter web pages on the >>>>> Beam site, under the Community tab. >>>>> 5. If you want to submit more content after the soft deadline, add >>>>> it to the google doc and let me know to republish. I don't want to >>>>> make the >>>>> publication changes automatic because that leaves us open to tampering. >>>>> >>>>> >>>>> This process is more laggy, so I'd suggest doing a 2 month vs monthly >>>>> newsletter cadence. If we're happy with this idea, I'll send in a website >>>>> PR for a new "Newsletter" left nav item under Community. >>>>> >>>>> Here's an example of a published newsletter: Apache Beam >>>>> February-March 2019 >>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB> >>>>> >>>>> >>>>> - This link is permanent unless the principal google doc is >>>>> deleted. >>>>> - Changes to the google doc after web publication are not >>>>> automatically published on the website to protect the information >>>>> integrity. >>>>> - Republishing is quick and easy for me if you let me know you've >>>>> added more. >>>>> - I'll improve the formatting later if we go with this route. >>>>> >>>>> Any thoughts? >>>>> >>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <t...@apache.org> wrote: >>>>> >>>>>> Similar to blog posts. A link that can be shared would also help to >>>>>> distribute over other channels, such as Twitter. >>>>>> >>>>>> >>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ieme...@gmail.com> wrote: >>>>>> >>>>>>> We should have these newsletters published somewhere with a fixed >>>>>>> URL so we can add missing updates, I have at least missed putting stuff >>>>>>> in >>>>>>> the last 3 ones just to realize when the newsletter has been already >>>>>>> 'published' (and sadly interesting features like zstd have not been >>>>>>> announced because of this). >>>>>>> >>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot < >>>>>>> echauc...@apache.org> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> I would add in what's been done: >>>>>>>> >>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, >>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix >>>>>>>> >>>>>>>> Etienne >>>>>>>> >>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit : >>>>>>>> >>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech >>>>>>>> Warsaw last Wednesday - I can send the updates offline. >>>>>>>> >>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rtngu...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> [image: Beam.png] >>>>>>>> >>>>>>>> February-March 2019 | Newsletter >>>>>>>> >>>>>>>> What’s been done >>>>>>>> >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> Apache Beam 2.10.0 released (by: many contributors) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Download the release here. >>>>>>>> <https://beam.apache.org/get-started/downloads/> >>>>>>>> - >>>>>>>> >>>>>>>> See the blog post >>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for >>>>>>>> more details. >>>>>>>> >>>>>>>> >>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award! >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year >>>>>>>> Award. >>>>>>>> - >>>>>>>> >>>>>>>> See this article >>>>>>>> >>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true> >>>>>>>> for more details. >>>>>>>> >>>>>>>> >>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and >>>>>>>> Spark. >>>>>>>> - >>>>>>>> >>>>>>>> See Matt’s Blog >>>>>>>> >>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/> >>>>>>>> for more details. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> What we’re working on... >>>>>>>> >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> Apache Beam 2.11.0 release (by: many contributors) >>>>>>>> >>>>>>>> >>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Support for plugging table providers through Beam SQL API to >>>>>>>> allow obtaining table schemas from external sources. >>>>>>>> - >>>>>>>> >>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more >>>>>>>> details. >>>>>>>> >>>>>>>> >>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Working on expanding the variety of user defined types that can >>>>>>>> be a member of a PCollection in the Go SDK. >>>>>>>> - >>>>>>>> >>>>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> >>>>>>>> for more details. >>>>>>>> >>>>>>>> >>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark >>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Beam 2.11.0 is the first release offering partial Python 3 >>>>>>>> support. >>>>>>>> - >>>>>>>> >>>>>>>> Many thanks to all contributors who helped to reach this >>>>>>>> milestone. >>>>>>>> - >>>>>>>> >>>>>>>> IO availablility on Python 3 is currently limited and only >>>>>>>> Python 3.5 version has been tested extensively. >>>>>>>> - >>>>>>>> >>>>>>>> Stay tuned on BEAM-1251 for more details. >>>>>>>> >>>>>>>> >>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Adding IPython notebooks and snippets >>>>>>>> - >>>>>>>> >>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for >>>>>>>> more details. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> New members >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> New PMC member! >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Etienne Chauchot, Nantes, France >>>>>>>> >>>>>>>> >>>>>>>> New Committers! >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Gleb Kanterov, Stockholm, Sweden >>>>>>>> - >>>>>>>> >>>>>>>> Michael Luckey >>>>>>>> >>>>>>>> >>>>>>>> New Contributors! >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Kyle Weaver, San Francisco, CA >>>>>>>> - >>>>>>>> >>>>>>>> Would like to help begin implementing portability support >>>>>>>> for the Spark runner >>>>>>>> - >>>>>>>> >>>>>>>> Tanay Tummapalli, Delhi, India >>>>>>>> - >>>>>>>> >>>>>>>> Would like to contribute to Open Source this summer as part >>>>>>>> of Google Summer of Code >>>>>>>> - >>>>>>>> >>>>>>>> Brian Hulette, Seattle, WA >>>>>>>> - >>>>>>>> >>>>>>>> Contributing to Beam Portability >>>>>>>> - >>>>>>>> >>>>>>>> Michał Walenia, Warsaw, Poland >>>>>>>> - >>>>>>>> >>>>>>>> Working on integration and load testing >>>>>>>> - >>>>>>>> >>>>>>>> Daniel Chen, San Francisco, CA >>>>>>>> - >>>>>>>> >>>>>>>> Working on Beam Samza runner >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Talks & meetups >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> >>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ >>>>>>>> London >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Watch the How to Run Kettle on Apache Beam video here >>>>>>>> >>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>. >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> See event details here >>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/> >>>>>>>> .. >>>>>>>> >>>>>>>> >>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San >>>>>>>> Francisco, CA >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers >>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai >>>>>>>> - >>>>>>>> >>>>>>>> See event details here >>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/> >>>>>>>> and the slides for these presentation: Overview of Apache Beam >>>>>>>> and TensorFlow Transform (TFX) with Apache Beam >>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming >>>>>>>> Pipelines with Beam on Flink >>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic >>>>>>>> pricing of Lyft rides using streaming >>>>>>>> >>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming> >>>>>>>> >>>>>>>> . >>>>>>>> >>>>>>>> Flink meetup - Feb 21@ Seattle, WA >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache >>>>>>>> Flink with Hive, Tensorflow, Beam, and AthenaX. >>>>>>>> - >>>>>>>> >>>>>>>> See event details here >>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and >>>>>>>> presentations here >>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>. >>>>>>>> >>>>>>>> >>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20. >>>>>>>> - >>>>>>>> >>>>>>>> Speaker CfP and other details to follow soon! >>>>>>>> - >>>>>>>> >>>>>>>> Twitter announcement! >>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Resources >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> A guide >>>>>>>> >>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide> >>>>>>>> to introduce Beam contributors to the basics of using the Apache >>>>>>>> Jira for >>>>>>>> Beam development. Feedback welcomed! >>>>>>>> >>>>>>>> >>>>>>>> An approach to community building from Apache Beam (by: Kenn >>>>>>>> Knowles) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> The Apache Software Foundation has published committer >>>>>>>> guidelines to help Beam's community building work. >>>>>>>> - >>>>>>>> >>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on >>>>>>>> the ASF blog. >>>>>>>> >>>>>>>> >>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley) >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> “In this article, I’ll dive into this new feature of Beam, and >>>>>>>> see how it works by using a pipeline to read a data file from GCS, >>>>>>>> transform it, and then perform a basic calculation on the values >>>>>>>> contained >>>>>>>> in the file”. >>>>>>>> - >>>>>>>> >>>>>>>> See article >>>>>>>> >>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4> >>>>>>>> and full source code >>>>>>>> >>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java> >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> *Until Next Time!* >>>>>>>> -- >>>>>>>> Rose Thị Nguyễn >>>>>>>> >>>>>>>> >>>>> >>>>> -- >>>>> Rose Thị Nguyễn >>>>> >>>> -- Rose Thị Nguyễn
