This makes me worry that the performance regression is elsewhere, in which case we should investigate.
On Fri, Mar 9, 2018 at 1:02 AM Etienne Chauchot <echauc...@apache.org> wrote: > Le jeudi 08 mars 2018 à 20:01 +0000, Reuven Lax a écrit : > > Does Nexmark use SerializableCoder? > > > Actually SerializableCoder is registered in Nexmark but the default and > the current configuration are set to use "by hand serialization". See: > > public static void setupPipeline(CoderStrategy coderStrategy, Pipeline p) { > CoderRegistry registry = p.getCoderRegistry(); > switch (coderStrategy) { > case HAND: > registry.registerCoderForClass(Auction.class, Auction.CODER); > registry.registerCoderForClass(AuctionBid.class, AuctionBid.CODER); > registry.registerCoderForClass(AuctionCount.class, AuctionCount.CODER); > registry.registerCoderForClass(AuctionPrice.class, AuctionPrice.CODER); > registry.registerCoderForClass(Bid.class, Bid.CODER); > registry.registerCoderForClass(CategoryPrice.class, > CategoryPrice.CODER); > registry.registerCoderForClass(Event.class, Event.CODER); > registry.registerCoderForClass(IdNameReserve.class, > IdNameReserve.CODER); > registry.registerCoderForClass(NameCityStateId.class, > NameCityStateId.CODER); > registry.registerCoderForClass(Person.class, Person.CODER); > registry.registerCoderForClass(SellerPrice.class, SellerPrice.CODER); > registry.registerCoderForClass(Done.class, Done.CODER); > registry.registerCoderForClass(BidsPerSession.class, > BidsPerSession.CODER); > break; > case AVRO: > registry.registerCoderProvider(AvroCoder.getCoderProvider()); > break; > case JAVA: > registry.registerCoderProvider(SerializableCoder.getCoderProvider()); > break; > } > } > > > Etienne > > > > On Thu, Mar 8, 2018 at 10:42 AM Robert Bradshaw <rober...@google.com> > wrote: > > I put the validation checklist spreadsheet is up at > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit?ts=5a1c7310#gid=1663314475 > > Regarding the direct runner regression on query 10, this is understandable > given how mutation detection has been changed for serializable coders (and > should be tracked, probably fixed by avoiding SerializableCoder). It should > not affect other runners. Could you file a bug? > > Regarding waitUntilFinish, this is a bug but not a blocker--it's been > this way since teardown was introduced. There are many nice-to-haves that > one could merge from master to the release branch, but we've seen where > that trend leads. > > Regarding the backwards incompatible changes in restriction tracker, this > is (as I understand it) a change to the experimental SDF API. Eugene, do > you want to comment on this? > > > > On Thu, Mar 8, 2018 at 2:07 AM Ismaël Mejía <ieme...@gmail.com> wrote: > > I confirm that the new release fixes both problems reported previously: > > - python package name > - nexmark query 10 mutability issue with the direct runner. > > One extra regression is that the the fix produced a way longer > execution time on the query. > Not sure if a blocker but worth tracking. > > Query 10 - Batch/Bounded > Version Runtime(sec) Events(/sec) Results > 2.3.0 3.6 27609.1 1 > 2.4.0 30.8 3244.3 1 > > Query 10 - Streaming/Unbounded > Version Runtime(sec) Events(/sec) Results > 2.3.0 6.3 15873.0 1 > 2.4.0 101.1 989.4 1 > > On Thu, Mar 8, 2018 at 8:54 AM, Romain Manni-Bucau > <rmannibu...@gmail.com> wrote: > > -1: > > a) still consider waitUntilFinish broken and a big blocker > > b) restrictiontracker api changed and is not backward compatible > > ( > https://github.com/apache/beam/commit/e0034314ad196d2274cef9831ed63e090bf4d4c1#diff-098d7247eb1e9d9423bfa2ae2da38a9d > ) > > > > with workarounds and fixes for these two issues the other parts work > (spark, > > flink, direct runner, java core) on my projects > > > > > > > > Romain Manni-Bucau > > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > > > > 2018-03-08 6:26 GMT+01:00 Robert Bradshaw <rober...@google.com>: > >> > >> Hi everyone, > >> > >> Please review and vote on the release candidate #2 for the version > 2.4.0, > >> as follows: > >> [ ] +1, Approve the release > >> [ ] -1, Do not approve the release (please provide specific comments) > >> > >> The complete staging area is available for your review, which includes: > >> * JIRA release notes [1], > >> * the official Apache source release to be deployed to dist.apache.org > >> [2], > >> which is signed with the key with fingerprint BDC9 89B0 1BD2 A463 6010 > >> A1CA 8F15 5E09 610D 69FB [3], > >> * all artifacts to be deployed to the Maven Central Repository [4], > >> * source code tag "v2.4.0-RC2" [5], > >> * website pull request listing the release and publishing the API > >> reference > >> manual [6]. > >> * Java artifacts were built with Maven 3.2.5 and OpenJDK 1.8.0_112. > >> * Python artifact are deployed along with the source release to the > >> dist.apache.org [2]. If I am able to figure out how to build the > wheels, I > >> will post them there as well. > >> > >> The vote will be open for at least 72 hours. It is adopted by majority > >> approval, with at least 3 PMC affirmative votes. > >> > >> Thanks, > >> - Robert > >> > >> [1] > >> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342682&projectId=12319527 > >> [2] https://dist.apache.org/repos/dist/dev/beam/2.4.0/ > >> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS > >> [4] > https://repository.apache.org/content/repositories/orgapachebeam-1030/ > >> [5] https://github.com/apache/beam/tree/v2.4.0-RC2 > >> [6] https://github.com/apache/beam-site/pull/398 > > > > > > >