Just my two cents. I think that we should probably strip CI into a separate repo. The key indicator is that if something were wrong in the CI yaml, would I hold a release for that? I think no. So that suggests to me it is a separate thing. Same goes for benchmarks. If we were failing a benchmark I would be concerned, but if the script were broken, would I hold the release? I think no as well.
I think that says that the CI code should also be a separate repo. Thanks, Mark > On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jbarr...@pivotal.io> wrote: > > Until someone outside of the geode ci community is asking for it I just don’t > see utility in going through the motions of making a release for it. > >> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onich...@pivotal.io> wrote: >> >> The source is already public, so on some level a source release is no >> different from a git tag. Benchmarks has matured enough that I think it >> makes sense to at least start branching and tagging the geode-benchmarks >> repo to capture exactly what was used in that Geode release. >> >> Others in the dev and user community may find the benchmarks useful in other >> ways than we use them. While our focus for CI is on tuning for >> repeatability, someone else might just want a load generator to break in a >> new cluster or get some rough numbers. Some might want to get under the >> hood and tinker and tune, or contribute their own benchmarks, with the >> understanding that it’s not a turnkey or standalone product, but a tool that >> requires getting your hands dirty. >> >> Would a “1 page” readme with a few tips on “how to run on a laptop” be >> enough to let other interested contributors help get geode-benchmarks to a >> “better state”? >> >>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarr...@pivotal.io> wrote: >>> >>> I don’t think the benchmarks provide any material benefit to a user in >>> their current state. They are heavily tuned for our CI process which relies >>> on very beefy machines. Usage on other hardware will require more tuning. I >>> don’t think it’s worth putting the source in the release until they are in >>> a better state. >>> >>> -Jake >>> >>> >>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <dsm...@pivotal.io> wrote: >>>> >>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onich...@pivotal.io> wrote: >>>> >>>>> I believe the desire is to include the source code for geode-benchmarks as >>>>> part of the official geode release, much like how we include >>>>> geode-examples. >>>>> >>>> >>>> Oh! I thought you meant running the benchmarks in the release pipeline - I >>>> think last release we were running them but decided they were too flaky to >>>> use. >>>> >>>> +1 to including the benchmark source in the source release. >>>> >>>> -Dan >>
