We are supposed to be including all of the source necessary to test Geode in the source release [1] - I think that would include benchmarks as well.
I don't really see any compelling reason *not* to include the benchmarks, let's go ahead and get them into our source release! [1] http://www.apache.org/legal/release-policy.html#what-must-every-release-contain On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <onich...@pivotal.io> wrote: > +1 for no changes > > On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jbarr...@pivotal.io> wrote: > > > We can live in areas of gray that don’t require any changes. Nobody is > > asking for benchmarks so let’s not do work to add them. Nobody is > > complaining they CI is included so let’s not do work to remove them. Is > it > > ideal, meh... > > > > > On Jan 15, 2020, at 5:50 PM, Mark Hanson <mhan...@pivotal.io> wrote: > > > > > > Just my two cents. > > > > > > I think that we should probably strip CI into a separate repo. The key > > indicator is that if something were wrong in the CI yaml, would I hold a > > release for that? I think no. So that suggests to me it is a separate > > thing. Same goes for benchmarks. If we were failing a benchmark I would > be > > concerned, but if the script were broken, would I hold the release? I > think > > no as well. > > > > > > I think that says that the CI code should also be a separate repo. > > > > > > Thanks, > > > Mark > > > > > >> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jbarr...@pivotal.io> > > wrote: > > >> > > >> Until someone outside of the geode ci community is asking for it I > just > > don’t see utility in going through the motions of making a release for > it. > > >> > > >>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onich...@pivotal.io> > > wrote: > > >>> > > >>> The source is already public, so on some level a source release is > no > > different from a git tag. Benchmarks has matured enough that I think it > > makes sense to at least start branching and tagging the geode-benchmarks > > repo to capture exactly what was used in that Geode release. > > >>> > > >>> Others in the dev and user community may find the benchmarks useful > in > > other ways than we use them. While our focus for CI is on tuning for > > repeatability, someone else might just want a load generator to break in > a > > new cluster or get some rough numbers. Some might want to get under the > > hood and tinker and tune, or contribute their own benchmarks, with the > > understanding that it’s not a turnkey or standalone product, but a tool > > that requires getting your hands dirty. > > >>> > > >>> Would a “1 page” readme with a few tips on “how to run on a laptop” > be > > enough to let other interested contributors help get geode-benchmarks to > a > > “better state”? > > >>> > > >>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarr...@pivotal.io> > > wrote: > > >>>> > > >>>> I don’t think the benchmarks provide any material benefit to a user > > in their current state. They are heavily tuned for our CI process which > > relies on very beefy machines. Usage on other hardware will require more > > tuning. I don’t think it’s worth putting the source in the release until > > they are in a better state. > > >>>> > > >>>> -Jake > > >>>> > > >>>> > > >>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <dsm...@pivotal.io> wrote: > > >>>>> > > >>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onich...@pivotal.io > > > > wrote: > > >>>>> > > >>>>>> I believe the desire is to include the source code for > > geode-benchmarks as > > >>>>>> part of the official geode release, much like how we include > > geode-examples. > > >>>>>> > > >>>>> > > >>>>> Oh! I thought you meant running the benchmarks in the release > > pipeline - I > > >>>>> think last release we were running them but decided they were too > > flaky to > > >>>>> use. > > >>>>> > > >>>>> +1 to including the benchmark source in the source release. > > >>>>> > > >>>>> -Dan > > >>> > > > > > >
