On the original thread https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E , Robert and Ismaël were in favor of no major version change [Ismaël said:* Also I am afraid that if we wait* *until we have enough changes to switch Beam to a new major version the switch to Java 8 will happen too late, probably after Java 8's end of life. And I am not exaggerating, Java 8 is planned to EOL next march 2018!*]; JB and now Reuven are in favor of a major version change; nobody so far argued against switching to Java8 in general.
I'm personally in favor of no major version change (i.e. not waiting until all other large changes for Beam 3.0 converge, which will likely be many months), because: - Reasons Ismaël cited; plus the reason that most people are likely already using Java 8. - Going Java8-only earlier will make other Beam 3.0 APIs better for Java8 users, because we (Beam contributors) will have experience working with them within the SDK in Java8 (e.g. writing tests with use of lambdas and noticing whether it's clunky, or whether some other Beam APIs need better Java8 support). - Going Java8 will make it more reasonable to include (mostly or only) Java8 snippets in Beam documentation, which will obviously look more concise and attractive, addressing one of the common concerns of Beam users that it has a heavyweight API compared to functional-style APIs of Spark etc. I think resolving this via a poll of users would be reasonable. I'd suggest e.g. the following phrasing: Apache Beam is considering dropping support for Java 7, and supporting only Java 8 and above in a subsequent release. How would it impact your usage of Beam? - I am already using only Java 8+ for building my Beam code - I am using Java 7 for building my Beam code, but I would have no trouble switching to Java 8 - I am using Java 7 for building my Beam code, and dropping Java 7 would be a blocker or hindrance to adopting the new release for me We could tweet this poll on Apache Beam twitter and publish on user@, and, say, if we receive 5% or fewer votes for option 3 after keeping it open for 2 weeks, then adopt Java 8 without a major version change. WDYT? On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré <[email protected]> wrote: > Good idea ! Definitely +1 > > Regards > JB > > On 12/05/2017 05:25 AM, Reuven Lax wrote: > > We should bring this up on the Beam 3.0 thread. Since it's technically a > > backwards-incompatible change, it might make a good item for Beam 3.0. > > > > Reuven > > > > On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré <[email protected] > > <mailto:[email protected]>> wrote: > > > > My apologizes, I thought we had a consensus already. > > > > Regards > > JB > > > > On 12/04/2017 11:22 PM, Eugene Kirpichov wrote: > > > > Thanks JB for sending the detailed notes about new stuff in > 2.2.0! A lot > > of exciting things indeed. > > > > Regarding Java 8: I thought our consensus was to have the > release notes > > say that we're *considering* going Java8-only, and use that to > get more > > opinions from the user community - but I can't find the emails > that made > > me think so. > > > > +Ismaël Mejía <mailto:[email protected] <mailto: > [email protected]>> - do > > you think we should formally conclude the vote on the > thread [VOTE] > > [DISCUSSION] Remove support for Java 7? > > Or should we take more steps - e.g. perhaps tweet a link to that > thread > > from the Beam twitter account, ask people to chime in, and wait > for say > > 2 weeks before declaring a conclusion? > > > > Let's also have a process JIRA for going Java8. I've filed one: > > https://issues.apache.org/jira/browse/BEAM-3285 > > <https://issues.apache.org/jira/browse/BEAM-3285> > > > > On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré < > [email protected] > > <mailto:[email protected]> <mailto:[email protected] > > <mailto:[email protected]>>> wrote: > > > > Just an important note that we forgot to mention. > > > > !! The 2.2.0 release will be the last one supporting Spark > 1.x and > > Java 7 !! > > > > Starting from Beam 2.3.0, the Spark runner will work only > with > > Spark 2.x and we > > will focus only Java 8. > > > > Regards > > JB > > > > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote: > > > Thanks Reuven ! > > > > > > I would like to emphasize on some highlights in 2.2.0 > release: > > > > > > - New IOs have been introduced: > > > * TikaIO leveraging Apache Tika, allowing the deal > with a lot > > of different > > > data formats > > > * RedisIO to read and write key/value pairs from a > Redis > > server. This > > IO will > > > be soon extended to Redis PubSub. > > > * FileIO provides transforms for working with files > (raw). > > Especially, it > > > provides matching file patterns and read on patterns. It > can be > > easily > > extended > > > for a specific format (like we do in AvroIO or TextIO > now). > > > * SolrIO to interact with Apache Solr (Lucene) > > > > > > - On the other hand, improvements have been performed on > > existing IOs: > > > * We started to introduce readAll pattern in IOs > (AvroIO, > > TextIO, JdbcIO, > > > ...), allowing to pass "request" arguments via an input > PCollection. > > > * ElasticsearchIO has an improved support of different > > Elasticsearch > > version > > > (including Elasticsearch 5.x). It also now supports > SSL/TLS. > > > * HBaseIO is now able to do dynamic work rebalancing > > > * KinesisIO uses a more accurate watermark (based on > > approximateArrivalTimestamp) > > > * TextIO now supports custom delimiter and like AvroIO, > > supports the > > readAll > > > pattern, > > > * Performance improvements on JdbcIO when it has to > read lot > > of rows > > > * Kafka write supports Exactly-Once pattern (introduce > in > > Kafka 0.11.x) > > > > > > - A new DSL has been introduced: the SQL DSL ! > > > > > > We are now focus on 2.3.0 release with new improvements > and > > features ! > > > > > > Stay tuned ! > > > > > > JB on behalf of the Apache Beam community. > > > > > > On 12/02/2017 11:40 PM, Reuven Lax wrote: > > >> The Apache Beam community is pleased to announce the > > availability of the > > >> 2.2.0 release. > > >> > > >> This release adds support for generic file sources and > sinks > > (beyond TextIO > > >> and AvroIO) using FileIO, including support for dynamic > > filenames using > > >> readAll; this allows streaming pipelines to now read > from files by > > >> continuously monitoring a directory for new filw. Many > other > > IOs are > > improved, > > >> notably including exactly-once support for the Kafka > sink. Initial > > support for > > >> BEAM-SQL is also included in this release. For a > more-complete > > list of major > > >> changes in the release, please refer to the release > notes [2]. > > >> > > >> The 2.2.0 release is now the recommended version; we > encourage > > everyone to > > >> upgrade from any earlier releases. > > >> > > >> We’d like to invite everyone to try out Apache Beam > today and > > consider > > >> joining our vibrant community. We welcome feedback, > > contribution and > > >> participation through our mailing lists, issue tracker, > pull > > requests, and > > >> events. > > >> > > >> - Reuven Lax, on behalf of the Apache Beam community. > > >> > > >> [1] https://beam.apache.org/get-started/downloads/ > > <https://beam.apache.org/get-started/downloads/> > > >> [2] > > >> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > > < > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > > > > >> > > > > > > > -- > > Jean-Baptiste Onofré > > [email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > > > > -- > > Jean-Baptiste Onofré > > [email protected] <mailto:[email protected]> > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
