Thanks JB for sending the detailed notes about new stuff in 2.2.0! A lot of exciting things indeed.
Regarding Java 8: I thought our consensus was to have the release notes say that we're *considering* going Java8-only, and use that to get more opinions from the user community - but I can't find the emails that made me think so. +Ismaël Mejía <ieme...@gmail.com> - do you think we should formally conclude the vote on the thread [VOTE] [DISCUSSION] Remove support for Java 7? Or should we take more steps - e.g. perhaps tweet a link to that thread from the Beam twitter account, ask people to chime in, and wait for say 2 weeks before declaring a conclusion? Let's also have a process JIRA for going Java8. I've filed one: https://issues.apache.org/jira/browse/BEAM-3285 On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Just an important note that we forgot to mention. > > !! The 2.2.0 release will be the last one supporting Spark 1.x and Java 7 > !! > > Starting from Beam 2.3.0, the Spark runner will work only with Spark 2.x > and we > will focus only Java 8. > > Regards > JB > > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote: > > Thanks Reuven ! > > > > I would like to emphasize on some highlights in 2.2.0 release: > > > > - New IOs have been introduced: > > * TikaIO leveraging Apache Tika, allowing the deal with a lot of > different > > data formats > > * RedisIO to read and write key/value pairs from a Redis server. This > IO will > > be soon extended to Redis PubSub. > > * FileIO provides transforms for working with files (raw). Especially, > it > > provides matching file patterns and read on patterns. It can be easily > extended > > for a specific format (like we do in AvroIO or TextIO now). > > * SolrIO to interact with Apache Solr (Lucene) > > > > - On the other hand, improvements have been performed on existing IOs: > > * We started to introduce readAll pattern in IOs (AvroIO, TextIO, > JdbcIO, > > ...), allowing to pass "request" arguments via an input PCollection. > > * ElasticsearchIO has an improved support of different Elasticsearch > version > > (including Elasticsearch 5.x). It also now supports SSL/TLS. > > * HBaseIO is now able to do dynamic work rebalancing > > * KinesisIO uses a more accurate watermark (based on > approximateArrivalTimestamp) > > * TextIO now supports custom delimiter and like AvroIO, supports the > readAll > > pattern, > > * Performance improvements on JdbcIO when it has to read lot of rows > > * Kafka write supports Exactly-Once pattern (introduce in Kafka 0.11.x) > > > > - A new DSL has been introduced: the SQL DSL ! > > > > We are now focus on 2.3.0 release with new improvements and features ! > > > > Stay tuned ! > > > > JB on behalf of the Apache Beam community. > > > > On 12/02/2017 11:40 PM, Reuven Lax wrote: > >> The Apache Beam community is pleased to announce the availability of the > >> 2.2.0 release. > >> > >> This release adds support for generic file sources and sinks (beyond > TextIO > >> and AvroIO) using FileIO, including support for dynamic filenames using > >> readAll; this allows streaming pipelines to now read from files by > >> continuously monitoring a directory for new filw. Many other IOs are > improved, > >> notably including exactly-once support for the Kafka sink. Initial > support for > >> BEAM-SQL is also included in this release. For a more-complete list of > major > >> changes in the release, please refer to the release notes [2]. > >> > >> The 2.2.0 release is now the recommended version; we encourage everyone > to > >> upgrade from any earlier releases. > >> > >> We’d like to invite everyone to try out Apache Beam today and consider > >> joining our vibrant community. We welcome feedback, contribution and > >> participation through our mailing lists, issue tracker, pull requests, > and > >> events. > >> > >> - Reuven Lax, on behalf of the Apache Beam community. > >> > >> [1] https://beam.apache.org/get-started/downloads/ > >> [2] > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > >> > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com >
