Re: [DISCUSS] - Create new repository for geode benchmarks

[Dan Smith]

Seems like there is enough consensus. I will go ahead and create the repo.

Thanks!
-Dan

On Fri, Nov 16, 2018 at 1:37 PM Kirk Lund  wrote:

> +1 total agreement then!
>
> On Fri, Nov 16, 2018 at 10:05 AM Dan Smith  wrote:
>
> > Hi Kirk,
> >
> > What we're thinking of putting in geode-benchmarks are new, multi host
> > benchmarks of the full system with the public APIs, not microbenchmarks.
> We
> > weren't planning on doing anything with the JMH benchmarks at the
> moment. I
> > agree with you those should stay in the geode module they are testing,
> > since they generally are microbenchmarking internal APIs of that module.
> >
> > I appreciate you bringing those up though - I would like to get to the
> > point where we are running those microbenchmarks in CI as well!
> >
> > -Dan
> >
> > On Fri, Nov 16, 2018 at 9:07 AM Kirk Lund  wrote:
> >
> > > That makes sense for some benchmarks but not others. For example, while
> > > working on the Logging changes, I wrote a some benchmarks that directly
> > use
> > > some new internal code to ensure that the new changes perform well.
> > >
> > > +1 to creating a benchmarks repo that has general perf tests that will
> be
> > > run in the pipelines
> > >
> > > -1 to getting rid of benchmarks from geode-core or any other submodule
> > > because this will discourage developers from writing benchmarks
> specific
> > to
> > > new code as they write it -- we shouldn't be forced to write benchmarks
> > > AFTER we commit to the main geode repo (or worse, after a release)
> > >
> > > On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:
> > >
> > > > Hi all,
> > > >
> > > > We (Naba, Sean, Brian and I) would like to add some benchmarks for
> > geode,
> > > > with a goal of eventually running them as part of our concourse build
> > and
> > > > detecting performance changes.
> > > >
> > > > We think it makes sense to put these benchmarks in a separate
> > > > geode-benchmarks repository. That will make them less tightly coupled
> > to
> > > a
> > > > specific revision of geode. What do you all think? If folks are okay
> > with
> > > > this, I will go ahead and create the repository.
> > > >
> > > > We have some prototype code in this repository with a simple
> > > client/server
> > > > put benchmark:  https://github.com/upthewaterspout/geode-performance
> .
> > > >
> > > > -Dan
> > > >
> > >
> >
>

Re: [DISCUSS] - Create new repository for geode benchmarks

[Kirk Lund]

+1 total agreement then!

On Fri, Nov 16, 2018 at 10:05 AM Dan Smith  wrote:

> Hi Kirk,
>
> What we're thinking of putting in geode-benchmarks are new, multi host
> benchmarks of the full system with the public APIs, not microbenchmarks. We
> weren't planning on doing anything with the JMH benchmarks at the moment. I
> agree with you those should stay in the geode module they are testing,
> since they generally are microbenchmarking internal APIs of that module.
>
> I appreciate you bringing those up though - I would like to get to the
> point where we are running those microbenchmarks in CI as well!
>
> -Dan
>
> On Fri, Nov 16, 2018 at 9:07 AM Kirk Lund  wrote:
>
> > That makes sense for some benchmarks but not others. For example, while
> > working on the Logging changes, I wrote a some benchmarks that directly
> use
> > some new internal code to ensure that the new changes perform well.
> >
> > +1 to creating a benchmarks repo that has general perf tests that will be
> > run in the pipelines
> >
> > -1 to getting rid of benchmarks from geode-core or any other submodule
> > because this will discourage developers from writing benchmarks specific
> to
> > new code as they write it -- we shouldn't be forced to write benchmarks
> > AFTER we commit to the main geode repo (or worse, after a release)
> >
> > On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:
> >
> > > Hi all,
> > >
> > > We (Naba, Sean, Brian and I) would like to add some benchmarks for
> geode,
> > > with a goal of eventually running them as part of our concourse build
> and
> > > detecting performance changes.
> > >
> > > We think it makes sense to put these benchmarks in a separate
> > > geode-benchmarks repository. That will make them less tightly coupled
> to
> > a
> > > specific revision of geode. What do you all think? If folks are okay
> with
> > > this, I will go ahead and create the repository.
> > >
> > > We have some prototype code in this repository with a simple
> > client/server
> > > put benchmark:  https://github.com/upthewaterspout/geode-performance.
> > >
> > > -Dan
> > >
> >
>

Re: [DISCUSS] - Create new repository for geode benchmarks

[Kenneth Howe]

+1 to Dan’s proposal - just makes sense

> On Nov 16, 2018, at 10:05 AM, Dan Smith  wrote:
> 
> Hi Kirk,
> 
> What we're thinking of putting in geode-benchmarks are new, multi host
> benchmarks of the full system with the public APIs, not microbenchmarks. We
> weren't planning on doing anything with the JMH benchmarks at the moment. I
> agree with you those should stay in the geode module they are testing,
> since they generally are microbenchmarking internal APIs of that module.
> 
> I appreciate you bringing those up though - I would like to get to the
> point where we are running those microbenchmarks in CI as well!
> 
> -Dan
> 
> On Fri, Nov 16, 2018 at 9:07 AM Kirk Lund  wrote:
> 
>> That makes sense for some benchmarks but not others. For example, while
>> working on the Logging changes, I wrote a some benchmarks that directly use
>> some new internal code to ensure that the new changes perform well.
>> 
>> +1 to creating a benchmarks repo that has general perf tests that will be
>> run in the pipelines
>> 
>> -1 to getting rid of benchmarks from geode-core or any other submodule
>> because this will discourage developers from writing benchmarks specific to
>> new code as they write it -- we shouldn't be forced to write benchmarks
>> AFTER we commit to the main geode repo (or worse, after a release)
>> 
>> On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:
>> 
>>> Hi all,
>>> 
>>> We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
>>> with a goal of eventually running them as part of our concourse build and
>>> detecting performance changes.
>>> 
>>> We think it makes sense to put these benchmarks in a separate
>>> geode-benchmarks repository. That will make them less tightly coupled to
>> a
>>> specific revision of geode. What do you all think? If folks are okay with
>>> this, I will go ahead and create the repository.
>>> 
>>> We have some prototype code in this repository with a simple
>> client/server
>>> put benchmark:  https://github.com/upthewaterspout/geode-performance.
>>> 
>>> -Dan
>>> 
>> 

Re: [DISCUSS] - Create new repository for geode benchmarks

[Dan Smith]

Hi Kirk,

What we're thinking of putting in geode-benchmarks are new, multi host
benchmarks of the full system with the public APIs, not microbenchmarks. We
weren't planning on doing anything with the JMH benchmarks at the moment. I
agree with you those should stay in the geode module they are testing,
since they generally are microbenchmarking internal APIs of that module.

I appreciate you bringing those up though - I would like to get to the
point where we are running those microbenchmarks in CI as well!

-Dan

On Fri, Nov 16, 2018 at 9:07 AM Kirk Lund  wrote:

> That makes sense for some benchmarks but not others. For example, while
> working on the Logging changes, I wrote a some benchmarks that directly use
> some new internal code to ensure that the new changes perform well.
>
> +1 to creating a benchmarks repo that has general perf tests that will be
> run in the pipelines
>
> -1 to getting rid of benchmarks from geode-core or any other submodule
> because this will discourage developers from writing benchmarks specific to
> new code as they write it -- we shouldn't be forced to write benchmarks
> AFTER we commit to the main geode repo (or worse, after a release)
>
> On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:
>
> > Hi all,
> >
> > We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
> > with a goal of eventually running them as part of our concourse build and
> > detecting performance changes.
> >
> > We think it makes sense to put these benchmarks in a separate
> > geode-benchmarks repository. That will make them less tightly coupled to
> a
> > specific revision of geode. What do you all think? If folks are okay with
> > this, I will go ahead and create the repository.
> >
> > We have some prototype code in this repository with a simple
> client/server
> > put benchmark:  https://github.com/upthewaterspout/geode-performance.
> >
> > -Dan
> >
>

Re: [DISCUSS] - Create new repository for geode benchmarks

[Kirk Lund]

That makes sense for some benchmarks but not others. For example, while
working on the Logging changes, I wrote a some benchmarks that directly use
some new internal code to ensure that the new changes perform well.

+1 to creating a benchmarks repo that has general perf tests that will be
run in the pipelines

-1 to getting rid of benchmarks from geode-core or any other submodule
because this will discourage developers from writing benchmarks specific to
new code as they write it -- we shouldn't be forced to write benchmarks
AFTER we commit to the main geode repo (or worse, after a release)

On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:

> Hi all,
>
> We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
> with a goal of eventually running them as part of our concourse build and
> detecting performance changes.
>
> We think it makes sense to put these benchmarks in a separate
> geode-benchmarks repository. That will make them less tightly coupled to a
> specific revision of geode. What do you all think? If folks are okay with
> this, I will go ahead and create the repository.
>
> We have some prototype code in this repository with a simple client/server
> put benchmark:  https://github.com/upthewaterspout/geode-performance.
>
> -Dan
>

Re: [DISCUSS] - Create new repository for geode benchmarks

[Juan José Ramos]

+1

On Thu, Nov 15, 2018 at 8:22 PM Udo Kohlmeyer  wrote:

> +1, never thought there was any other option...
>
>
> On 11/15/18 10:47, Dan Smith wrote:
> > Hi all,
> >
> > We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
> > with a goal of eventually running them as part of our concourse build and
> > detecting performance changes.
> >
> > We think it makes sense to put these benchmarks in a separate
> > geode-benchmarks repository. That will make them less tightly coupled to
> a
> > specific revision of geode. What do you all think? If folks are okay with
> > this, I will go ahead and create the repository.
> >
> > We have some prototype code in this repository with a simple
> client/server
> > put benchmark:  https://github.com/upthewaterspout/geode-performance.
> >
> > -Dan
> >
>
>

-- 
Juan José Ramos Cassella
Senior Technical Support Engineer
Email: jra...@pivotal.io
Office#: +353 21 4238611
Mobile#: +353 87 2074066
After Hours Contact#: +1 877 477 2269
Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
How to upload artifacts:
https://support.pivotal.io/hc/en-us/articles/204369073
How to escalate a ticket:
https://support.pivotal.io/hc/en-us/articles/203809556

[image: support]  [image: twitter]
 [image: linkedin]
 [image: facebook]
 [image: google plus]
 [image: youtube]

Re: [DISCUSS] - Create new repository for geode benchmarks

[Udo Kohlmeyer]

+1, never thought there was any other option...


On 11/15/18 10:47, Dan Smith wrote:

Hi all,

We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
with a goal of eventually running them as part of our concourse build and
detecting performance changes.

We think it makes sense to put these benchmarks in a separate
geode-benchmarks repository. That will make them less tightly coupled to a
specific revision of geode. What do you all think? If folks are okay with
this, I will go ahead and create the repository.

We have some prototype code in this repository with a simple client/server
put benchmark:  https://github.com/upthewaterspout/geode-performance.

-Dan

Re: [DISCUSS] - Create new repository for geode benchmarks

[Ryan McMahon]

+1 I think a separate repo makes sense, and I'm excited for this effort!


On Thu, Nov 15, 2018 at 10:47 AM Dan Smith  wrote:

> Hi all,
>
> We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
> with a goal of eventually running them as part of our concourse build and
> detecting performance changes.
>
> We think it makes sense to put these benchmarks in a separate
> geode-benchmarks repository. That will make them less tightly coupled to a
> specific revision of geode. What do you all think? If folks are okay with
> this, I will go ahead and create the repository.
>
> We have some prototype code in this repository with a simple client/server
> put benchmark:  https://github.com/upthewaterspout/geode-performance.
>
> -Dan
>

[DISCUSS] - Create new repository for geode benchmarks

[Dan Smith]

Hi all,

We (Naba, Sean, Brian and I) would like to add some benchmarks for geode,
with a goal of eventually running them as part of our concourse build and
detecting performance changes.

We think it makes sense to put these benchmarks in a separate
geode-benchmarks repository. That will make them less tightly coupled to a
specific revision of geode. What do you all think? If folks are okay with
this, I will go ahead and create the repository.

We have some prototype code in this repository with a simple client/server
put benchmark:  https://github.com/upthewaterspout/geode-performance.

-Dan