Hi! On Fri, Mar 9, 2018 at 1:22 PM Rafal Wojdyla <[email protected]> wrote:
> Hi all, > > We have just released Scio 0.5.0. This is a major/breaking release - make > sure to read the breaking changes section below. > > - rav > > https://github.com/spotify/scio/releases/tag/v0.5.0 > > *"In ictu"* > > Breaking changes > > - BigQueryIO in JobTest#output now requires a type parameter. Explicit > .map(T.toTableRow)of test data is no longer needed. > - Typed AvroIO now accepts case classes instead of Avro records in > JobTest. Explicit .map(T.toGenericRecord) of test data is no longer > needed. See this change > > <https://github.com/spotify/scio/commit/19fee4716f71827ac4affbd23d753bc074c529b8> > for more. > - Package com.spotify.scio.extra.transforms is moved from scio-extra to > scio-core, under com.spotify.scio.transforms. > > See this section > <https://github.com/spotify/scio/wiki/Apache-Beam#breaking-changes-since-scio-050> > for more details. > Features > > - Support reading BigQuery as Avro #964 > <https://github.com/spotify/scio/issues/964>, #992 > <https://github.com/spotify/scio/pull/992> > - BigQuery client now supports load and export #1060 > <https://github.com/spotify/scio/pull/1060> > - Add TFRecordSpec support for Featran #1002 > <https://github.com/spotify/scio/pull/1002> > - Add AsyncLookupDoFn #1012 <https://github.com/spotify/scio/pull/1012> > > This looks like a very useful general-purpose tool, Beam users have asked for something like this many times; and the implementation looks very high-quality as well. Any interest in backporting it into Beam? > > - Bump sparkey to 2.2.1, protobuf-generic to 0.2.4 #1028 > <https://github.com/spotify/scio/pull/1028> > - Added ser/der support for joda DateTime #1038 > <https://github.com/spotify/scio/pull/1038> > - Password is now optional for jdbc connection #1040 > <https://github.com/spotify/scio/pull/1040> > - Add job cancellation option to ScioResult#waitUntilDone #1056 > <https://github.com/spotify/scio/issues/1056> #1058 > <https://github.com/spotify/scio/pull/1058> #1062 > <https://github.com/spotify/scio/pull/1062> #1066 > <https://github.com/spotify/scio/pull/1066> > > Bug fixes > > - Fix transform name in joins #1034 > <https://github.com/spotify/scio/issues/1034> #1035 > <https://github.com/spotify/scio/pull/1035> > - Add applyKvTransform #1020 > <https://github.com/spotify/scio/issues/1020> #1032 > <https://github.com/spotify/scio/pull/1032> > - Add helpers to initialize counters #1026 > <https://github.com/spotify/scio/issues/1026> #1027 > <https://github.com/spotify/scio/pull/1027> > - Fix SCollectionMatchers serialization #1001 > <https://github.com/spotify/scio/pull/1001> > - Check runner version #1008 > <https://github.com/spotify/scio/issues/1008> #1009 > <https://github.com/spotify/scio/pull/1009> > - Log exception in AsyncLookupDoFn only if cache put fails #1039 > <https://github.com/spotify/scio/pull/1039> > - ProjectId nonEmpty string check in BigQueryClient #1045 > <https://github.com/spotify/scio/pull/1045> > - Fix SCollection#withSlidingWindows #1054 > <https://github.com/spotify/scio/pull/1054> > > >
