We only recently started running Chicago Taxi Example. +Michał Walenia <michal.wale...@polidea.com> I don't see it in the dashboards. Do you know if it's possible to see any trends in the data?
We have a few tests running now: - Combine tests: https://apache-beam-testing.appspot.com/explore?dashboard=5763764733345792&widget=201943890&container=1334074373 - GBK tests: https://apache-beam-testing.appspot.com/explore?dashboard=5763764733345792&widget=201943890&container=1334074373 They don't seem to show a very drastic jump either, but they aren't very old. There is also work ongoing to add alerting for this sort of regressions by Kasia and Kamil (added). The work is not there yet (it's in progress). Best -P. On Thu, Sep 5, 2019 at 3:35 PM Thomas Weise <t...@apache.org> wrote: > It probably won't be practical to do a bisect due to the high cost of each > iteration with our fork/deploy setup. > > Perhaps it is time to setup something with the synthetic source that works > just with Beam as dependency. > > On Thu, Sep 5, 2019 at 3:23 PM Ahmet Altay <al...@google.com> wrote: > >> There are a few in this dashboard [1], but not very useful in this case >> because they do not go back more than a month and not very comprehensive. I >> do not see a jump there. Thomas, would it be possible to bisect to find >> what commit caused the regression? >> >> +Pablo Estrada <pabl...@google.com> do we have any python on flink >> benchmarks for chicago example? >> +Alan Myrvold <amyrv...@google.com> +Yifan Zou <yifan...@google.com> It >> would be good to have alerts on benchmarks. Do we have such an ability >> today? >> >> [1] https://apache-beam-testing.appspot.com/dashboard-admin >> >> On Thu, Sep 5, 2019 at 3:15 PM Thomas Weise <t...@apache.org> wrote: >> >>> Hi, >>> >>> Are there any performance tests run for the Python SDK as part of >>> release verification (or otherwise as well)? >>> >>> I see what appears to be a regression in master (compared to 2.14) with >>> our in-house application (~ 25% jump in cpu utilization and corresponds >>> drop in throughput). >>> >>> I wanted to see if there is anything available to verify that within >>> Beam. >>> >>> Thanks, >>> Thomas >>> >>>
