Hi Andrew,
Yes I saw that, except dedicating jenkins nodes to nexmark, I see no other way.
Also, did you see query 3 output size on direct runner? Should be a straight
line and it is not, I'm wondering if there
is a problem with sate and timers impl in direct runner.
Etienne
Le mardi 17 juillet 2018 à 11:38 -0700, Andrew Pilloud a écrit :
> I'm noticing the graphs are really noisy. It looks like we are running these
> on shared Jenkins executors, so our perf
> tests are fighting with other builds for CPU. I've opened an issue
> https://issues.apache.org/jira/browse/BEAM-4804 and
> am wondering if anyone knows an easy fix to isolate these jobs.
> Andrew
> On Fri, Jul 13, 2018 at 2:39 AM Łukasz Gajowy <lgaj...@apache.org> wrote:
> > @Etienne: Nice to see the graphs! :)
> >
> > @Ismael: Good idea, there's no document yet. I think we could create a
> > small google doc with instructions on how to
> > do this.
> >
> > pt., 13 lip 2018 o 10:46 Etienne Chauchot <echauc...@apache.org> napisał(a):
> > > Hi,
> > > @Andrew, this is because I did not find a way to set 2 scales on the Y
> > > axis on the perfkit graphs. Indeed
> > > numResults varies from 1 to 100 000 and runtimeSec is usually bellow 10s.
> > > Etienne
> > > Le jeudi 12 juillet 2018 à 12:04 -0700, Andrew Pilloud a écrit :
> > > > This is great, should make performance work much easier! I'm going to
> > > > get the Beam SQL Nexmark jobs publishing
> > > > as well. (Opened https://issues.apache.org/jira/browse/BEAM-4774 to
> > > > track.) I might take on the Dataflow runner
> > > > as well if no one else volunteers.
> > > >
> > > > I am curious as to why you have two separate graphs for runtime and
> > > > count rather then graphing runtime/count to
> > > > get the throughput rate for each run? Or should that be a third graph?
> > > > Looks like it would just be a small tweak
> > > > to the query in perfkit.
> > > > Andrew
> > > > On Thu, Jul 12, 2018 at 11:40 AM Pablo Estrada <pabl...@google.com>
> > > > wrote:
> > > > > This is really cool Etienne : ) thanks for working on this.Our of
> > > > > curiosity, do you know how often the tests
> > > > > run on each runner?
> > > > >
> > > > > Best
> > > > > -P.
> > > > >
> > > > > On Thu, Jul 12, 2018 at 2:15 AM Romain Manni-Bucau
> > > > > <rmannibu...@gmail.com> wrote:
> > > > > > Awesome Etienne, this is really important for the (user) community
> > > > > > to have that visibility since it is one
> > > > > > of the most important aspect of the Beam's quality, kudo!
> > > > > >
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book
> > > > > >
> > > > > > Le jeu. 12 juil. 2018 à 10:59, Jean-Baptiste Onofré
> > > > > > <j...@nanthrax.net> a écrit :
> > > > > > > It's really great to have these dashboards and integration in
> > > > > > > Jenkins !
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks Etienne for driving this !
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Regards
> > > > > > >
> > > > > > > JB
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 11/07/2018 15:13, Etienne Chauchot wrote:
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Hi guys,
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > I'm glad to announce that the CI of Beam has much improved !
> > > > > > > > Indeed
> > > > > > >
> > > > > > > > Nexmark is now included in the perfkit dashboards.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > At each commit on master, nexmark suites are run and plots are
> > > > > > > > created
> > > > > > >
> > > > > > > > on the graphs.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > I've created 2 kind of dashboards:
> > > > > > >
> > > > > > > > - one for performances (run times of the queries)
> > > > > > >
> > > > > > > > - one for the size of the output PCollection (which should be
> > > > > > > > constant)
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > There are dashboards for these runners:
> > > > > > >
> > > > > > > > - spark
> > > > > > >
> > > > > > > > - flink
> > > > > > >
> > > > > > > > - direct runner
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Each dashboard contains:
> > > > > > >
> > > > > > > > - graphs in batch mode
> > > > > > >
> > > > > > > > - graphs in streaming mode
> > > > > > >
> > > > > > > > - graphs for the 13 queries.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > That gives more than a hundred of graphs (my right finger hurts
> > > > > > > > after so
> > > > > > >
> > > > > > > > many clics on the mouse :) ). It is detailed that much so that
> > > > > > > > anyone
> > > > > > >
> > > > > > > > can focus on the area they have interest in.
> > > > > > >
> > > > > > > > Feel free to also create new dashboards with more aggregated
> > > > > > > > data.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Thanks to Lukasz and Cham for reviewing my PRs and showing how
> > > > > > > > to use
> > > > > > >
> > > > > > > > perfkit dashboards.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Dashboards are there:
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424
> > > > > > >
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384
> > > > > > >
> > > > > > > > <https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712>https://apache-beam-testing.
> > > > > > > appspot.com/explore?dashboard=5138380291571712
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5099379773931520
> > > > > > >
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5731568492478464
> > > > > > >
> > > > > > > > https://apache-beam-testing.appspot.com/explore?dashboard=5163657986048000
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Enjoy,
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > Etienne
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
