Re: [DISCUSS] include geode-benchmarks in 1.12 release
I can see your argument that geode-benchmarks is strictly part of Geode CI for now, and CI is not “part of Geode” or generally useful to anyone outside the Geode CI community. If so, I think it would also be a good idea to exclude geode/ci from Geode source releases. > On Jan 14, 2020, at 10:21 PM, Jacob Barrett wrote: > > Until someone outside of the geode ci community is asking for it I just don’t > see utility in going through the motions of making a release for it. > >> On Jan 14, 2020, at 10:13 PM, Owen Nichols wrote: >> >> The source is already public, so on some level a source release is no >> different from a git tag. Benchmarks has matured enough that I think it >> makes sense to at least start branching and tagging the geode-benchmarks >> repo to capture exactly what was used in that Geode release. >> >> Others in the dev and user community may find the benchmarks useful in other >> ways than we use them. While our focus for CI is on tuning for >> repeatability, someone else might just want a load generator to break in a >> new cluster or get some rough numbers. Some might want to get under the >> hood and tinker and tune, or contribute their own benchmarks, with the >> understanding that it’s not a turnkey or standalone product, but a tool that >> requires getting your hands dirty. >> >> Would a “1 page” readme with a few tips on “how to run on a laptop” be >> enough to let other interested contributors help get geode-benchmarks to a >> “better state”? >> >>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett wrote: >>> >>> I don’t think the benchmarks provide any material benefit to a user in >>> their current state. They are heavily tuned for our CI process which relies >>> on very beefy machines. Usage on other hardware will require more tuning. I >>> don’t think it’s worth putting the source in the release until they are in >>> a better state. >>> >>> -Jake >>> >>> > On Jan 14, 2020, at 4:14 PM, Dan Smith wrote: On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols wrote: > I believe the desire is to include the source code for geode-benchmarks as > part of the official geode release, much like how we include > geode-examples. > Oh! I thought you meant running the benchmarks in the release pipeline - I think last release we were running them but decided they were too flaky to use. +1 to including the benchmark source in the source release. -Dan >>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Until someone outside of the geode ci community is asking for it I just don’t see utility in going through the motions of making a release for it. > On Jan 14, 2020, at 10:13 PM, Owen Nichols wrote: > > The source is already public, so on some level a source release is no > different from a git tag. Benchmarks has matured enough that I think it > makes sense to at least start branching and tagging the geode-benchmarks repo > to capture exactly what was used in that Geode release. > > Others in the dev and user community may find the benchmarks useful in other > ways than we use them. While our focus for CI is on tuning for > repeatability, someone else might just want a load generator to break in a > new cluster or get some rough numbers. Some might want to get under the hood > and tinker and tune, or contribute their own benchmarks, with the > understanding that it’s not a turnkey or standalone product, but a tool that > requires getting your hands dirty. > > Would a “1 page” readme with a few tips on “how to run on a laptop” be enough > to let other interested contributors help get geode-benchmarks to a “better > state”? > >> On Jan 14, 2020, at 9:38 PM, Jacob Barrett wrote: >> >> I don’t think the benchmarks provide any material benefit to a user in their >> current state. They are heavily tuned for our CI process which relies on >> very beefy machines. Usage on other hardware will require more tuning. I >> don’t think it’s worth putting the source in the release until they are in a >> better state. >> >> -Jake >> >> On Jan 14, 2020, at 4:14 PM, Dan Smith wrote: >>> >>> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols wrote: >>> I believe the desire is to include the source code for geode-benchmarks as part of the official geode release, much like how we include geode-examples. >>> >>> Oh! I thought you meant running the benchmarks in the release pipeline - I >>> think last release we were running them but decided they were too flaky to >>> use. >>> >>> +1 to including the benchmark source in the source release. >>> >>> -Dan >
Re: [DISCUSS] include geode-benchmarks in 1.12 release
The source is already public, so on some level a source release is no different from a git tag. Benchmarks has matured enough that I think it makes sense to at least start branching and tagging the geode-benchmarks repo to capture exactly what was used in that Geode release. Others in the dev and user community may find the benchmarks useful in other ways than we use them. While our focus for CI is on tuning for repeatability, someone else might just want a load generator to break in a new cluster or get some rough numbers. Some might want to get under the hood and tinker and tune, or contribute their own benchmarks, with the understanding that it’s not a turnkey or standalone product, but a tool that requires getting your hands dirty. Would a “1 page” readme with a few tips on “how to run on a laptop” be enough to let other interested contributors help get geode-benchmarks to a “better state”? > On Jan 14, 2020, at 9:38 PM, Jacob Barrett wrote: > > I don’t think the benchmarks provide any material benefit to a user in their > current state. They are heavily tuned for our CI process which relies on very > beefy machines. Usage on other hardware will require more tuning. I don’t > think it’s worth putting the source in the release until they are in a better > state. > > -Jake > > >> On Jan 14, 2020, at 4:14 PM, Dan Smith wrote: >> >> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols wrote: >> >>> I believe the desire is to include the source code for geode-benchmarks as >>> part of the official geode release, much like how we include geode-examples. >>> >> >> Oh! I thought you meant running the benchmarks in the release pipeline - I >> think last release we were running them but decided they were too flaky to >> use. >> >> +1 to including the benchmark source in the source release. >> >> -Dan
Re: [DISCUSS] include geode-benchmarks in 1.12 release
On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols wrote: > I believe the desire is to include the source code for geode-benchmarks as > part of the official geode release, much like how we include geode-examples. > Oh! I thought you meant running the benchmarks in the release pipeline - I think last release we were running them but decided they were too flaky to use. +1 to including the benchmark source in the source release. -Dan
Re: [DISCUSS] include geode-benchmarks in 1.12 release
I believe the desire is to include the source code for geode-benchmarks as part of the official geode release, much like how we include geode-examples. > On Jan 14, 2020, at 4:07 PM, Dan Smith wrote: > > Looks like the benchmarks have passed for around 30 builds in a row. +1 to > including them if they don't fail for spurious reasons between now and > cutting the release. > > BTW - Do we have stats on the variance we see in the benchmark numbers? Are > we getting close to failing occasionally? What is the criteria that will > cause them to fail? Is this on the wiki somewhere? > > -Dan > > On Tue, Jan 14, 2020 at 3:58 PM Owen Nichols wrote: > >> @Anthony has twice proposed[1][2] that we start including geode-benchmarks >> in our release process. Is this worth considering for the upcoming 1.12.0 >> release? >> >> [1] >> https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E >> [2] >> https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E >> >>> On Jan 14, 2020, at 3:44 PM, Mark Hanson wrote: >>> >>> It is that time again. It is time to cut a new release branch for 1.12 >> on February 3rd. >>> >> >>
Re: [DISCUSS] include geode-benchmarks in 1.12 release
Looks like the benchmarks have passed for around 30 builds in a row. +1 to including them if they don't fail for spurious reasons between now and cutting the release. BTW - Do we have stats on the variance we see in the benchmark numbers? Are we getting close to failing occasionally? What is the criteria that will cause them to fail? Is this on the wiki somewhere? -Dan On Tue, Jan 14, 2020 at 3:58 PM Owen Nichols wrote: > @Anthony has twice proposed[1][2] that we start including geode-benchmarks > in our release process. Is this worth considering for the upcoming 1.12.0 > release? > > [1] > https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E > [2] > https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E > > > On Jan 14, 2020, at 3:44 PM, Mark Hanson wrote: > > > > It is that time again. It is time to cut a new release branch for 1.12 > on February 3rd. > > > >
[DISCUSS] include geode-benchmarks in 1.12 release
@Anthony has twice proposed[1][2] that we start including geode-benchmarks in our release process. Is this worth considering for the upcoming 1.12.0 release? [1] https://lists.apache.org/thread.html/c7bd84b6e6f5464ed674ed447fe8922097237932967b4ed1966e79d3%40%3Cdev.geode.apache.org%3E [2] https://lists.apache.org/thread.html/c9c73e726742c78a1ff9739379a51f7eeab33a05d99531a38b537670%40%3Cdev.geode.apache.org%3E > On Jan 14, 2020, at 3:44 PM, Mark Hanson wrote: > > It is that time again. It is time to cut a new release branch for 1.12 on > February 3rd. >
privacy protection
Hi geode-dev, Is it possible somehow to protect all files that containing user data(or user data itself) being stored in disk for Geode. This includes all persistence data(OpLogs), backups and possible other files containing user data. Also protection is needed for all of the files potentailly used for replication and cluster high availability mechanism. If this feature is not available, do you have it in the plan already? Do you included it in Geode roadmap? BR, Mario
