The language “sufficient for a user to build and test the release” suggest to me that "./gradlew test” should work (i.e. don’t release *just* src). It would be a slippery slope to construe this guideline as a mandate to include any and all tools, scripts, etc that may ever have been used in conjunction with validating a release.
> On Jan 16, 2020, at 10:23 AM, Dan Smith <dsm...@pivotal.io> wrote: > > We are supposed to be including all of the source necessary to test Geode > in the source release [1] - I think that would include benchmarks as well. > > I don't really see any compelling reason *not* to include the benchmarks, > let's go ahead and get them into our source release! > > [1] > http://www.apache.org/legal/release-policy.html#what-must-every-release-contain > > On Wed, Jan 15, 2020 at 10:26 PM Owen Nichols <onich...@pivotal.io> wrote: > >> +1 for no changes >> >> On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett <jbarr...@pivotal.io> wrote: >> >>> We can live in areas of gray that don’t require any changes. Nobody is >>> asking for benchmarks so let’s not do work to add them. Nobody is >>> complaining they CI is included so let’s not do work to remove them. Is >> it >>> ideal, meh... >>> >>>> On Jan 15, 2020, at 5:50 PM, Mark Hanson <mhan...@pivotal.io> wrote: >>>> >>>> Just my two cents. >>>> >>>> I think that we should probably strip CI into a separate repo. The key >>> indicator is that if something were wrong in the CI yaml, would I hold a >>> release for that? I think no. So that suggests to me it is a separate >>> thing. Same goes for benchmarks. If we were failing a benchmark I would >> be >>> concerned, but if the script were broken, would I hold the release? I >> think >>> no as well. >>>> >>>> I think that says that the CI code should also be a separate repo. >>>> >>>> Thanks, >>>> Mark >>>> >>>>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett <jbarr...@pivotal.io> >>> wrote: >>>>> >>>>> Until someone outside of the geode ci community is asking for it I >> just >>> don’t see utility in going through the motions of making a release for >> it. >>>>> >>>>>>> On Jan 14, 2020, at 10:13 PM, Owen Nichols <onich...@pivotal.io> >>> wrote: >>>>>> >>>>>> The source is already public, so on some level a source release is >> no >>> different from a git tag. Benchmarks has matured enough that I think it >>> makes sense to at least start branching and tagging the geode-benchmarks >>> repo to capture exactly what was used in that Geode release. >>>>>> >>>>>> Others in the dev and user community may find the benchmarks useful >> in >>> other ways than we use them. While our focus for CI is on tuning for >>> repeatability, someone else might just want a load generator to break in >> a >>> new cluster or get some rough numbers. Some might want to get under the >>> hood and tinker and tune, or contribute their own benchmarks, with the >>> understanding that it’s not a turnkey or standalone product, but a tool >>> that requires getting your hands dirty. >>>>>> >>>>>> Would a “1 page” readme with a few tips on “how to run on a laptop” >> be >>> enough to let other interested contributors help get geode-benchmarks to >> a >>> “better state”? >>>>>> >>>>>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett <jbarr...@pivotal.io> >>> wrote: >>>>>>> >>>>>>> I don’t think the benchmarks provide any material benefit to a user >>> in their current state. They are heavily tuned for our CI process which >>> relies on very beefy machines. Usage on other hardware will require more >>> tuning. I don’t think it’s worth putting the source in the release until >>> they are in a better state. >>>>>>> >>>>>>> -Jake >>>>>>> >>>>>>> >>>>>>>>> On Jan 14, 2020, at 4:14 PM, Dan Smith <dsm...@pivotal.io> wrote: >>>>>>>> >>>>>>>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols <onich...@pivotal.io >>> >>> wrote: >>>>>>>> >>>>>>>>> I believe the desire is to include the source code for >>> geode-benchmarks as >>>>>>>>> part of the official geode release, much like how we include >>> geode-examples. >>>>>>>>> >>>>>>>> >>>>>>>> Oh! I thought you meant running the benchmarks in the release >>> pipeline - I >>>>>>>> think last release we were running them but decided they were too >>> flaky to >>>>>>>> use. >>>>>>>> >>>>>>>> +1 to including the benchmark source in the source release. >>>>>>>> >>>>>>>> -Dan >>>>>> >>>> >>> >>
