We should bring this up on the Beam 3.0 thread. Since it's technically a backwards-incompatible change, it might make a good item for Beam 3.0.
Reuven On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré <[email protected]> wrote: > My apologizes, I thought we had a consensus already. > > Regards > JB > > On 12/04/2017 11:22 PM, Eugene Kirpichov wrote: > >> Thanks JB for sending the detailed notes about new stuff in 2.2.0! A lot >> of exciting things indeed. >> >> Regarding Java 8: I thought our consensus was to have the release notes >> say that we're *considering* going Java8-only, and use that to get more >> opinions from the user community - but I can't find the emails that made me >> think so. >> >> +Ismaël Mejía <mailto:[email protected]> - do you think we should >> formally conclude the vote on the thread [VOTE] [DISCUSSION] Remove support >> for Java 7? >> Or should we take more steps - e.g. perhaps tweet a link to that thread >> from the Beam twitter account, ask people to chime in, and wait for say 2 >> weeks before declaring a conclusion? >> >> Let's also have a process JIRA for going Java8. I've filed one: >> https://issues.apache.org/jira/browse/BEAM-3285 >> >> On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré <[email protected] >> <mailto:[email protected]>> wrote: >> >> Just an important note that we forgot to mention. >> >> !! The 2.2.0 release will be the last one supporting Spark 1.x and >> Java 7 !! >> >> Starting from Beam 2.3.0, the Spark runner will work only with Spark >> 2.x and we >> will focus only Java 8. >> >> Regards >> JB >> >> On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote: >> > Thanks Reuven ! >> > >> > I would like to emphasize on some highlights in 2.2.0 release: >> > >> > - New IOs have been introduced: >> > * TikaIO leveraging Apache Tika, allowing the deal with a lot of >> different >> > data formats >> > * RedisIO to read and write key/value pairs from a Redis server. >> This >> IO will >> > be soon extended to Redis PubSub. >> > * FileIO provides transforms for working with files (raw). >> Especially, it >> > provides matching file patterns and read on patterns. It can be >> easily >> extended >> > for a specific format (like we do in AvroIO or TextIO now). >> > * SolrIO to interact with Apache Solr (Lucene) >> > >> > - On the other hand, improvements have been performed on existing >> IOs: >> > * We started to introduce readAll pattern in IOs (AvroIO, >> TextIO, JdbcIO, >> > ...), allowing to pass "request" arguments via an input >> PCollection. >> > * ElasticsearchIO has an improved support of different >> Elasticsearch >> version >> > (including Elasticsearch 5.x). It also now supports SSL/TLS. >> > * HBaseIO is now able to do dynamic work rebalancing >> > * KinesisIO uses a more accurate watermark (based on >> approximateArrivalTimestamp) >> > * TextIO now supports custom delimiter and like AvroIO, supports >> the >> readAll >> > pattern, >> > * Performance improvements on JdbcIO when it has to read lot of >> rows >> > * Kafka write supports Exactly-Once pattern (introduce in Kafka >> 0.11.x) >> > >> > - A new DSL has been introduced: the SQL DSL ! >> > >> > We are now focus on 2.3.0 release with new improvements and >> features ! >> > >> > Stay tuned ! >> > >> > JB on behalf of the Apache Beam community. >> > >> > On 12/02/2017 11:40 PM, Reuven Lax wrote: >> >> The Apache Beam community is pleased to announce the availability >> of the >> >> 2.2.0 release. >> >> >> >> This release adds support for generic file sources and sinks >> (beyond TextIO >> >> and AvroIO) using FileIO, including support for dynamic filenames >> using >> >> readAll; this allows streaming pipelines to now read from files by >> >> continuously monitoring a directory for new filw. Many other IOs >> are >> improved, >> >> notably including exactly-once support for the Kafka sink. Initial >> support for >> >> BEAM-SQL is also included in this release. For a more-complete >> list of major >> >> changes in the release, please refer to the release notes [2]. >> >> >> >> The 2.2.0 release is now the recommended version; we encourage >> everyone to >> >> upgrade from any earlier releases. >> >> >> >> We’d like to invite everyone to try out Apache Beam today and >> consider >> >> joining our vibrant community. We welcome feedback, contribution >> and >> >> participation through our mailing lists, issue tracker, pull >> requests, and >> >> events. >> >> >> >> - Reuven Lax, on behalf of the Apache Beam community. >> >> >> >> [1] https://beam.apache.org/get-started/downloads/ >> >> [2] >> >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje >> ctId=12319527&version=12341044 >> >> >> > >> >> -- >> Jean-Baptiste Onofré >> [email protected] <mailto:[email protected]> >> http://blog.nanthrax.net >> Talend - http://www.talend.com >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
