Thank you Reuven! I tweeted the release announcement on Beam's account. On Mon, Dec 4, 2017 at 9:49 PM, Reuven Lax <re...@google.com> wrote:
> Technically it's a backwards-incompatible change, however if we are > convinced the risk is low we could do it. > > As mentioned on the original thread, it's not clear that all Beam users > read user@ - e.g. most Dataflow users definitely do not. I think we need > to separately reach out to users of each runner through runner-specific > channels. > > Reuven > > On Mon, Dec 4, 2017 at 9:37 PM, Eugene Kirpichov <kirpic...@google.com> > wrote: > >> On the original thread https://lists.apache.or >> g/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e3919790 >> 87f725d2e0@%3Cuser.beam.apache.org%3E , Robert and Ismaël were in favor >> of no major version change [Ismaël said:* Also I am afraid that if we >> wait* >> *until we have enough changes to switch Beam to a new major version the >> switch to Java 8 will happen too late, probably after Java 8's end of life. >> And I am not exaggerating, Java 8 is planned to EOL next march 2018!*]; >> JB and now Reuven are in favor of a major version change; nobody so far >> argued against switching to Java8 in general. >> >> I'm personally in favor of no major version change (i.e. not waiting >> until all other large changes for Beam 3.0 converge, which will likely be >> many months), because: >> - Reasons Ismaël cited; plus the reason that most people are likely >> already using Java 8. >> - Going Java8-only earlier will make other Beam 3.0 APIs better for Java8 >> users, because we (Beam contributors) will have experience working with >> them within the SDK in Java8 (e.g. writing tests with use of lambdas and >> noticing whether it's clunky, or whether some other Beam APIs need better >> Java8 support). >> - Going Java8 will make it more reasonable to include (mostly or only) >> Java8 snippets in Beam documentation, which will obviously look more >> concise and attractive, addressing one of the common concerns of Beam users >> that it has a heavyweight API compared to functional-style APIs of Spark >> etc. >> >> I think resolving this via a poll of users would be reasonable. I'd >> suggest e.g. the following phrasing: >> >> Apache Beam is considering dropping support for Java 7, and supporting >> only Java 8 and above in a subsequent release. How would it impact your >> usage of Beam? >> - I am already using only Java 8+ for building my Beam code >> - I am using Java 7 for building my Beam code, but I would have no >> trouble switching to Java 8 >> - I am using Java 7 for building my Beam code, and dropping Java 7 would >> be a blocker or hindrance to adopting the new release for me >> >> We could tweet this poll on Apache Beam twitter and publish on user@, >> and, say, if we receive 5% or fewer votes for option 3 after keeping it >> open for 2 weeks, then adopt Java 8 without a major version change. >> >> WDYT? >> >> On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >> >>> Good idea ! Definitely +1 >>> >>> Regards >>> JB >>> >>> On 12/05/2017 05:25 AM, Reuven Lax wrote: >>> > We should bring this up on the Beam 3.0 thread. Since it's technically >>> a >>> > backwards-incompatible change, it might make a good item for Beam 3.0. >>> > >>> > Reuven >>> > >>> > On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré <j...@nanthrax.net >>> > <mailto:j...@nanthrax.net>> wrote: >>> > >>> > My apologizes, I thought we had a consensus already. >>> > >>> > Regards >>> > JB >>> > >>> > On 12/04/2017 11:22 PM, Eugene Kirpichov wrote: >>> > >>> > Thanks JB for sending the detailed notes about new stuff in >>> 2.2.0! A lot >>> > of exciting things indeed. >>> > >>> > Regarding Java 8: I thought our consensus was to have the >>> release notes >>> > say that we're *considering* going Java8-only, and use that to >>> get more >>> > opinions from the user community - but I can't find the emails >>> that made >>> > me think so. >>> > >>> > +Ismaël Mejía <mailto:ieme...@gmail.com <mailto: >>> ieme...@gmail.com>> - do >>> > you think we should formally conclude the vote on the >>> thread [VOTE] >>> > [DISCUSSION] Remove support for Java 7? >>> > Or should we take more steps - e.g. perhaps tweet a link to >>> that thread >>> > from the Beam twitter account, ask people to chime in, and >>> wait for say >>> > 2 weeks before declaring a conclusion? >>> > >>> > Let's also have a process JIRA for going Java8. I've filed one: >>> > https://issues.apache.org/jira/browse/BEAM-3285 >>> > <https://issues.apache.org/jira/browse/BEAM-3285> >>> > >>> > On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré < >>> j...@nanthrax.net >>> > <mailto:j...@nanthrax.net> <mailto:j...@nanthrax.net >>> > <mailto:j...@nanthrax.net>>> wrote: >>> > >>> > Just an important note that we forgot to mention. >>> > >>> > !! The 2.2.0 release will be the last one supporting >>> Spark 1.x and >>> > Java 7 !! >>> > >>> > Starting from Beam 2.3.0, the Spark runner will work only >>> with >>> > Spark 2.x and we >>> > will focus only Java 8. >>> > >>> > Regards >>> > JB >>> > >>> > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote: >>> > > Thanks Reuven ! >>> > > >>> > > I would like to emphasize on some highlights in 2.2.0 >>> release: >>> > > >>> > > - New IOs have been introduced: >>> > > * TikaIO leveraging Apache Tika, allowing the deal >>> with a lot >>> > of different >>> > > data formats >>> > > * RedisIO to read and write key/value pairs from a >>> Redis >>> > server. This >>> > IO will >>> > > be soon extended to Redis PubSub. >>> > > * FileIO provides transforms for working with files >>> (raw). >>> > Especially, it >>> > > provides matching file patterns and read on patterns. >>> It can be >>> > easily >>> > extended >>> > > for a specific format (like we do in AvroIO or TextIO >>> now). >>> > > * SolrIO to interact with Apache Solr (Lucene) >>> > > >>> > > - On the other hand, improvements have been performed >>> on >>> > existing IOs: >>> > > * We started to introduce readAll pattern in IOs >>> (AvroIO, >>> > TextIO, JdbcIO, >>> > > ...), allowing to pass "request" arguments via an >>> input PCollection. >>> > > * ElasticsearchIO has an improved support of >>> different >>> > Elasticsearch >>> > version >>> > > (including Elasticsearch 5.x). It also now supports >>> SSL/TLS. >>> > > * HBaseIO is now able to do dynamic work rebalancing >>> > > * KinesisIO uses a more accurate watermark (based on >>> > approximateArrivalTimestamp) >>> > > * TextIO now supports custom delimiter and like >>> AvroIO, >>> > supports the >>> > readAll >>> > > pattern, >>> > > * Performance improvements on JdbcIO when it has to >>> read lot >>> > of rows >>> > > * Kafka write supports Exactly-Once pattern >>> (introduce in >>> > Kafka 0.11.x) >>> > > >>> > > - A new DSL has been introduced: the SQL DSL ! >>> > > >>> > > We are now focus on 2.3.0 release with new >>> improvements and >>> > features ! >>> > > >>> > > Stay tuned ! >>> > > >>> > > JB on behalf of the Apache Beam community. >>> > > >>> > > On 12/02/2017 11:40 PM, Reuven Lax wrote: >>> > >> The Apache Beam community is pleased to announce the >>> > availability of the >>> > >> 2.2.0 release. >>> > >> >>> > >> This release adds support for generic file sources >>> and sinks >>> > (beyond TextIO >>> > >> and AvroIO) using FileIO, including support for >>> dynamic >>> > filenames using >>> > >> readAll; this allows streaming pipelines to now read >>> from files by >>> > >> continuously monitoring a directory for new filw. >>> Many other >>> > IOs are >>> > improved, >>> > >> notably including exactly-once support for the Kafka >>> sink. Initial >>> > support for >>> > >> BEAM-SQL is also included in this release. For a >>> more-complete >>> > list of major >>> > >> changes in the release, please refer to the release >>> notes [2]. >>> > >> >>> > >> The 2.2.0 release is now the recommended version; we >>> encourage >>> > everyone to >>> > >> upgrade from any earlier releases. >>> > >> >>> > >> We’d like to invite everyone to try out Apache Beam >>> today and >>> > consider >>> > >> joining our vibrant community. We welcome feedback, >>> > contribution and >>> > >> participation through our mailing lists, issue >>> tracker, pull >>> > requests, and >>> > >> events. >>> > >> >>> > >> - Reuven Lax, on behalf of the Apache Beam community. >>> > >> >>> > >> [1] https://beam.apache.org/get-started/downloads/ >>> > <https://beam.apache.org/get-started/downloads/> >>> > >> [2] >>> > >> >>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj >>> ectId=12319527&version=12341044 >>> > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?pro >>> jectId=12319527&version=12341044> >>> > >> >>> > > >>> > >>> > -- >>> > Jean-Baptiste Onofré >>> > jbono...@apache.org <mailto:jbono...@apache.org> >>> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>> >>> > http://blog.nanthrax.net >>> > Talend - http://www.talend.com >>> > >>> > >>> > -- >>> > Jean-Baptiste Onofré >>> > jbono...@apache.org <mailto:jbono...@apache.org> >>> > http://blog.nanthrax.net >>> > Talend - http://www.talend.com >>> > >>> > >>> >>> -- >>> Jean-Baptiste Onofré >>> jbono...@apache.org >>> http://blog.nanthrax.net >>> Talend - http://www.talend.com >>> >> >
