+1 for no changes On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jbarr...@pivotal.io> wrote:
> We can live in areas of gray that don’t require any changes. Nobody is > asking for benchmarks so let’s not do work to add them. Nobody is > complaining they CI is included so let’s not do work to remove them. Is it > ideal, meh... > > > On Jan 15, 2020, at 5:50 PM, Mark Hanson <mhan...@pivotal.io> wrote: > > > > Just my two cents. > > > > I think that we should probably strip CI into a separate repo. The key > indicator is that if something were wrong in the CI yaml, would I hold a > release for that? I think no. So that suggests to me it is a separate > thing. Same goes for benchmarks. If we were failing a benchmark I would be > concerned, but if the script were broken, would I hold the release? I think > no as well. > > > > I think that says that the CI code should also be a separate repo. > > > > Thanks, > > Mark > > > >> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jbarr...@pivotal.io> > wrote: > >> > >> Until someone outside of the geode ci community is asking for it I just > don’t see utility in going through the motions of making a release for it. > >> > >>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onich...@pivotal.io> > wrote: > >>> > >>> The source is already public, so on some level a source release is no > different from a git tag. Benchmarks has matured enough that I think it > makes sense to at least start branching and tagging the geode-benchmarks > repo to capture exactly what was used in that Geode release. > >>> > >>> Others in the dev and user community may find the benchmarks useful in > other ways than we use them. While our focus for CI is on tuning for > repeatability, someone else might just want a load generator to break in a > new cluster or get some rough numbers. Some might want to get under the > hood and tinker and tune, or contribute their own benchmarks, with the > understanding that it’s not a turnkey or standalone product, but a tool > that requires getting your hands dirty. > >>> > >>> Would a “1 page” readme with a few tips on “how to run on a laptop” be > enough to let other interested contributors help get geode-benchmarks to a > “better state”? > >>> > >>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarr...@pivotal.io> > wrote: > >>>> > >>>> I don’t think the benchmarks provide any material benefit to a user > in their current state. They are heavily tuned for our CI process which > relies on very beefy machines. Usage on other hardware will require more > tuning. I don’t think it’s worth putting the source in the release until > they are in a better state. > >>>> > >>>> -Jake > >>>> > >>>> > >>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <dsm...@pivotal.io> wrote: > >>>>> > >>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onich...@pivotal.io> > wrote: > >>>>> > >>>>>> I believe the desire is to include the source code for > geode-benchmarks as > >>>>>> part of the official geode release, much like how we include > geode-examples. > >>>>>> > >>>>> > >>>>> Oh! I thought you meant running the benchmarks in the release > pipeline - I > >>>>> think last release we were running them but decided they were too > flaky to > >>>>> use. > >>>>> > >>>>> +1 to including the benchmark source in the source release. > >>>>> > >>>>> -Dan > >>> > > >
