Re: [DISCUSS] include geode-benchmarks in 1.12 release

2020-01-15 Thread Owen Nichols 
+1 for no changes

On Wed, Jan 15, 2020 at 5:57 PM Jacob Barrett  wrote:

> We can live in areas of gray that don’t require any changes. Nobody is
> asking for benchmarks so let’s not do work to add them. Nobody is
> complaining they CI is included so let’s not do work to remove them. Is it
> ideal, meh...
>
> > On Jan 15, 2020, at 5:50 PM, Mark Hanson  wrote:
> >
> > Just my two cents.
> >
> > I think that we should probably strip CI into a separate repo. The key
> indicator is that if something were wrong in the CI yaml, would I hold a
> release for that? I think no. So that suggests to me it is a separate
> thing. Same goes for benchmarks. If we were failing a benchmark I would be
> concerned, but if the script were broken, would I hold the release? I think
> no as well.
> >
> > I think that says that the CI code should also be a separate repo.
> >
> > Thanks,
> > Mark
> >
> >> On Jan 14, 2020, at 10:21 PM, Jacob Barrett 
> wrote:
> >>
> >> Until someone outside of the geode ci community is asking for it I just
> don’t see utility in going through the motions of making a release for it.
> >>
>  On Jan 14, 2020, at 10:13 PM, Owen Nichols 
> wrote:
> >>>
> >>> The source is already public, so on some level a source release is no
> different from a git tag.  Benchmarks has matured enough that I think it
> makes sense to at least start branching and tagging the geode-benchmarks
> repo to capture exactly what was used in that Geode release.
> >>>
> >>> Others in the dev and user community may find the benchmarks useful in
> other ways than we use them.  While our focus for CI is on tuning for
> repeatability, someone else might just want a load generator to break in a
> new cluster or get some rough numbers.  Some might want to get under the
> hood and tinker and tune, or contribute their own benchmarks, with the
> understanding that it’s not a turnkey or standalone product, but a tool
> that requires getting your hands dirty.
> >>>
> >>> Would a “1 page” readme with a few tips on “how to run on a laptop” be
> enough to let other interested contributors help get geode-benchmarks to a
> “better state”?
> >>>
>  On Jan 14, 2020, at 9:38 PM, Jacob Barrett 
> wrote:
> 
>  I don’t think the benchmarks provide any material benefit to a user
> in their current state. They are heavily tuned for our CI process which
> relies on very beefy machines. Usage on other hardware will require more
> tuning. I don’t think it’s worth putting the source in the release until
> they are in a better state.
> 
>  -Jake
> 
> 
> >> On Jan 14, 2020, at 4:14 PM, Dan Smith  wrote:
> >
> > On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols 
> wrote:
> >
> >> I believe the desire is to include the source code for
> geode-benchmarks as
> >> part of the official geode release, much like how we include
> geode-examples.
> >>
> >
> > Oh! I thought you meant running the benchmarks in the release
> pipeline - I
> > think last release we were running them but decided they were too
> flaky to
> > use.
> >
> > +1 to including the benchmark source in the source release.
> >
> > -Dan
> >>>
> >
>

Re: [DISCUSS] include geode-benchmarks in 1.12 release

2020-01-15 Thread Jacob Barrett 
We can live in areas of gray that don’t require any changes. Nobody is asking 
for benchmarks so let’s not do work to add them. Nobody is complaining they CI 
is included so let’s not do work to remove them. Is it ideal, meh...

> On Jan 15, 2020, at 5:50 PM, Mark Hanson  wrote:
> 
> Just my two cents.
> 
> I think that we should probably strip CI into a separate repo. The key 
> indicator is that if something were wrong in the CI yaml, would I hold a 
> release for that? I think no. So that suggests to me it is a separate thing. 
> Same goes for benchmarks. If we were failing a benchmark I would be 
> concerned, but if the script were broken, would I hold the release? I think 
> no as well.
> 
> I think that says that the CI code should also be a separate repo.
> 
> Thanks,
> Mark
> 
>> On Jan 14, 2020, at 10:21 PM, Jacob Barrett  wrote:
>> 
>> Until someone outside of the geode ci community is asking for it I just 
>> don’t see utility in going through the motions of making a release for it. 
>> 
 On Jan 14, 2020, at 10:13 PM, Owen Nichols  wrote:
>>> 
>>> The source is already public, so on some level a source release is no 
>>> different from a git tag.  Benchmarks has matured enough that I think it 
>>> makes sense to at least start branching and tagging the geode-benchmarks 
>>> repo to capture exactly what was used in that Geode release.
>>> 
>>> Others in the dev and user community may find the benchmarks useful in 
>>> other ways than we use them.  While our focus for CI is on tuning for 
>>> repeatability, someone else might just want a load generator to break in a 
>>> new cluster or get some rough numbers.  Some might want to get under the 
>>> hood and tinker and tune, or contribute their own benchmarks, with the 
>>> understanding that it’s not a turnkey or standalone product, but a tool 
>>> that requires getting your hands dirty.
>>> 
>>> Would a “1 page” readme with a few tips on “how to run on a laptop” be 
>>> enough to let other interested contributors help get geode-benchmarks to a 
>>> “better state”?
>>> 
 On Jan 14, 2020, at 9:38 PM, Jacob Barrett  wrote:
 
 I don’t think the benchmarks provide any material benefit to a user in 
 their current state. They are heavily tuned for our CI process which 
 relies on very beefy machines. Usage on other hardware will require more 
 tuning. I don’t think it’s worth putting the source in the release until 
 they are in a better state.
 
 -Jake
 
 
>> On Jan 14, 2020, at 4:14 PM, Dan Smith  wrote:
> 
> On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols  wrote:
> 
>> I believe the desire is to include the source code for geode-benchmarks 
>> as
>> part of the official geode release, much like how we include 
>> geode-examples.
>> 
> 
> Oh! I thought you meant running the benchmarks in the release pipeline - I
> think last release we were running them but decided they were too flaky to
> use.
> 
> +1 to including the benchmark source in the source release.
> 
> -Dan
>>> 
>

A Release Hero/Manager is needed for 1.12 (February 3rd)

2020-01-15 Thread Mark Hanson 
Just a reminder,  still looking for that release manager…  Please don’t all 
volunteer at once!

Glory and street cred await the intrepid release manager volunteer.

Hello All,

It is that time again. It is time to cut a new release branch for 1.12 on 
February 3rd.

We need a volunteer! No experience required. Committer status would be helpful, 
but not required.

In the mean time, we should focus on ensuring the CI is stable and start 
planning to cut the branch.
I would recommend our handy wiki as a great place to see what you are  in for 
should you choose to volunteer (please do)
https://cwiki.apache.org/confluence/display/GEODE/Releasing+Apache+Geode 


If we could have a volunteer by the end of the week, that would be amazing!

Thanks,
Mark

Re: [DISCUSS] include geode-benchmarks in 1.12 release

2020-01-15 Thread Mark Hanson 
Just my two cents.

I think that we should probably strip CI into a separate repo. The key 
indicator is that if something were wrong in the CI yaml, would I hold a 
release for that? I think no. So that suggests to me it is a separate thing. 
Same goes for benchmarks. If we were failing a benchmark I would be concerned, 
but if the script were broken, would I hold the release? I think no as well.

I think that says that the CI code should also be a separate repo.

Thanks,
Mark

> On Jan 14, 2020, at 10:21 PM, Jacob Barrett  wrote:
> 
> Until someone outside of the geode ci community is asking for it I just don’t 
> see utility in going through the motions of making a release for it. 
> 
>> On Jan 14, 2020, at 10:13 PM, Owen Nichols  wrote:
>> 
>> The source is already public, so on some level a source release is no 
>> different from a git tag.  Benchmarks has matured enough that I think it 
>> makes sense to at least start branching and tagging the geode-benchmarks 
>> repo to capture exactly what was used in that Geode release.
>> 
>> Others in the dev and user community may find the benchmarks useful in other 
>> ways than we use them.  While our focus for CI is on tuning for 
>> repeatability, someone else might just want a load generator to break in a 
>> new cluster or get some rough numbers.  Some might want to get under the 
>> hood and tinker and tune, or contribute their own benchmarks, with the 
>> understanding that it’s not a turnkey or standalone product, but a tool that 
>> requires getting your hands dirty.
>> 
>> Would a “1 page” readme with a few tips on “how to run on a laptop” be 
>> enough to let other interested contributors help get geode-benchmarks to a 
>> “better state”?
>> 
>>> On Jan 14, 2020, at 9:38 PM, Jacob Barrett  wrote:
>>> 
>>> I don’t think the benchmarks provide any material benefit to a user in 
>>> their current state. They are heavily tuned for our CI process which relies 
>>> on very beefy machines. Usage on other hardware will require more tuning. I 
>>> don’t think it’s worth putting the source in the release until they are in 
>>> a better state.
>>> 
>>> -Jake
>>> 
>>> 
> On Jan 14, 2020, at 4:14 PM, Dan Smith  wrote:
 
 On Tue, Jan 14, 2020 at 4:11 PM Owen Nichols  wrote:
 
> I believe the desire is to include the source code for geode-benchmarks as
> part of the official geode release, much like how we include 
> geode-examples.
> 
 
 Oh! I thought you meant running the benchmarks in the release pipeline - I
 think last release we were running them but decided they were too flaky to
 use.
 
 +1 to including the benchmark source in the source release.
 
 -Dan
>>

Odg: GW sender dispatcher threads & order policy

2020-01-15 Thread Mario Kevo 
Yes, sorry I re-read mail and see that it is not the same issue.
Here is how it is working now:


OrderPolicy cannot have an invalid string as it doesn't allow any string except 
this from the OrderPolicy.ENUM: KEY, THREAD and PARTITION.

  1.  If dispatcher threads is more than 1 you need to set order-policy to one 
of this OrderPolicy.ENUM. In case you set any other string you will get:
java.lang.IllegalArgumentException: Failed to convert 'XXX' to type OrderPolicy 
for option 'order-policy'
No enum constant org.apache.geode.cache.wan.GatewaySender.OrderPolicy.XXX
  2.  If dispatcher threads is equal 1 it doesn't need order-policy as it is 
alone there(You can set order-policy to some of this values, but they don't 
have any impact as it will not be used).
  3.  If it is lower than 1 you will get exception and message that dispatcher 
threads cannot be less than 1.

I guess that you refer to the documentation where is set that default value of 
order-policy is key, in that case the correction in documentation is needed or 
code change to always use default if it is not specified while creating gateway 
sender.

BR,
Mario

Šalje: Alberto Bustamante Reyes 
Poslano: 15. siječnja 2020. 18:52
Prima: Mario Kevo ; dev@geode.apache.org 

Predmet: RE: GW sender dispatcher threads & order policy

Hi Mario,

My code contains that fix, its not the same issue. GEODE-7561 solves the issue 
with the value "1" for dispatcher threads, but an explicit value for 
order-policy is still required if you specify a value for dispatcher threads.

BR/

Alberto B.

De: Mario Kevo 
Enviado: miércoles, 15 de enero de 2020 18:22
Para: Alberto Bustamante Reyes ; 
dev@geode.apache.org 
Asunto: Odg: GW sender dispatcher threads & order policy

Hi Alberto,

This is already solved in Geode 1.12.0.

https://issues.apache.org/jira/browse/GEODE-7561

BR,
Mario

Šalje: Alberto Bustamante Reyes 
Poslano: 15. siječnja 2020. 18:14
Prima: dev@geode.apache.org 
Predmet: GW sender dispatcher threads & order policy

Hi,

I have seen that if I change the default number of dispatcher threads ( 5 ) 
when creating a gateway sender, I get an error saying I must specify an order 
policy:

"Must specify --order-policy when --dispatcher-threads is larger than 1."

I find this odd, taking into account that the default value is already larger 
than 1 and order-policy has a default value. Actually, the error is shown if 
you specify "--dispatcher-threads=5". I was going to create a ticket to report 
this but I have a question: what is the use case for having a sender with less 
than 1 dispatcher thread?

Thanks!

BR/

Alberto B.

RE: GW sender dispatcher threads & order policy

2020-01-15 Thread Alberto Bustamante Reyes 
Hi Mario,

My code contains that fix, its not the same issue. GEODE-7561 solves the issue 
with the value "1" for dispatcher threads, but an explicit value for 
order-policy is still required if you specify a value for dispatcher threads.

BR/

Alberto B.

De: Mario Kevo 
Enviado: miércoles, 15 de enero de 2020 18:22
Para: Alberto Bustamante Reyes ; 
dev@geode.apache.org 
Asunto: Odg: GW sender dispatcher threads & order policy

Hi Alberto,

This is already solved in Geode 1.12.0.

https://issues.apache.org/jira/browse/GEODE-7561

BR,
Mario

Šalje: Alberto Bustamante Reyes 
Poslano: 15. siječnja 2020. 18:14
Prima: dev@geode.apache.org 
Predmet: GW sender dispatcher threads & order policy

Hi,

I have seen that if I change the default number of dispatcher threads ( 5 ) 
when creating a gateway sender, I get an error saying I must specify an order 
policy:

"Must specify --order-policy when --dispatcher-threads is larger than 1."

I find this odd, taking into account that the default value is already larger 
than 1 and order-policy has a default value. Actually, the error is shown if 
you specify "--dispatcher-threads=5". I was going to create a ticket to report 
this but I have a question: what is the use case for having a sender with less 
than 1 dispatcher thread?

Thanks!

BR/

Alberto B.

Odg: GW sender dispatcher threads & order policy

2020-01-15 Thread Mario Kevo 
Hi Alberto,

This is already solved in Geode 1.12.0.

https://issues.apache.org/jira/browse/GEODE-7561

BR,
Mario

Šalje: Alberto Bustamante Reyes 
Poslano: 15. siječnja 2020. 18:14
Prima: dev@geode.apache.org 
Predmet: GW sender dispatcher threads & order policy

Hi,

I have seen that if I change the default number of dispatcher threads ( 5 ) 
when creating a gateway sender, I get an error saying I must specify an order 
policy:

"Must specify --order-policy when --dispatcher-threads is larger than 1."

I find this odd, taking into account that the default value is already larger 
than 1 and order-policy has a default value. Actually, the error is shown if 
you specify "--dispatcher-threads=5". I was going to create a ticket to report 
this but I have a question: what is the use case for having a sender with less 
than 1 dispatcher thread?

Thanks!

BR/

Alberto B.

GW sender dispatcher threads & order policy

2020-01-15 Thread Alberto Bustamante Reyes 
Hi,

I have seen that if I change the default number of dispatcher threads ( 5 ) 
when creating a gateway sender, I get an error saying I must specify an order 
policy:

"Must specify --order-policy when --dispatcher-threads is larger than 1."

I find this odd, taking into account that the default value is already larger 
than 1 and order-policy has a default value. Actually, the error is shown if 
you specify "--dispatcher-threads=5". I was going to create a ticket to report 
this but I have a question: what is the use case for having a sender with less 
than 1 dispatcher thread?

Thanks!

BR/

Alberto B.

Re: [DISCUSSION] De/un-deprecate IndexType ENUM

2020-01-15 Thread Joris Melchior 
Since the feedback has been to proceed with this proposal I will open a
JIRA ticket and start the work on this.

Thanks for the feedback on this.

Joris.

On Fri, Jan 3, 2020 at 1:11 PM Kirk Lund  wrote:

> I'm for un-deprecating it.
>
> On Fri, Jan 3, 2020 at 7:01 AM Joris Melchior 
> wrote:
>
> > Yes, the code indicates that the deprecation is related to the different
> > QueryService methods but then these methods end up using the IndexType
> > themselves which is the main reason I want to un-deprecate the ENUM
> itself.
> >
> > The changes to the QueryService interface in the proposal were suggested
> by
> > John to make the code show its intentions more clearly.
> >
> > On Thu, Jan 2, 2020 at 4:40 PM John Blum  wrote:
> >
> > > I thought I recall that the IndexType
> > > <
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/IndexType.html
> > > >
> > > [1]
> > > was *deprecated* in favor of specific methods on the QueryService
> > > <
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html
> > > >
> > > interface
> > > [2] used to create Indexes of a specific type, e.g. like a Key Index
> > using
> > > QueryService.createKeyIndex(..)
> > > <
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createKeyIndex-java.lang.String-java.lang.String-java.lang.String-
> > > >
> > > [3]
> > > (or one of the "overloaded" variants), which is in contrast to the
> > generic
> > > createIndex(..)
> > > <
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createIndex-java.lang.String-org.apache.geode.cache.query.IndexType-java.lang.String-java.lang.String-java.lang.String-
> > > >
> > > method [4] that accepted the (now deprecated) IndexType Enum as an
> > > argument.
> > >
> > > However, I still feel that the IndexType Enum should NOT be deprecated,
> > > especially given that the Index.getType():IndexType
> > > <
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/Index.html#getType--
> > > >
> > > method [5] is quite useful to assess an Index (e.g. think
> > > Management/Monitoring tools or other analysis tools to ascertain the
> > > state/configuration of the system).
> > >
> > > -j
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/IndexType.html
> > > [2]
> > >
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html
> > > [3]
> > >
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createKeyIndex-java.lang.String-java.lang.String-java.lang.String-
> > > [4]
> > >
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createIndex-java.lang.String-org.apache.geode.cache.query.IndexType-java.lang.String-java.lang.String-java.lang.String-
> > > [5]
> > >
> > >
> >
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/Index.html#getType--
> > >
> > >
> > > On Thu, Jan 2, 2020 at 1:26 PM Joris Melchior 
> > > wrote:
> > >
> > > > Hi Kirk,
> > > >
> > > > No, I've tried to figure that out but was unsuccessful in doing so.
> It
> > > > would be helpful if someone would be able to shed some light on that.
> > > >
> > > >
> > > > On Thu, Jan 2, 2020 at 1:34 PM Kirk Lund  wrote:
> > > >
> > > > > Hi Joris, I've read the proposal and reviewed the code some. It's
> not
> > > > clear
> > > > > to me why it was originally deprecated or what the intended new
> > > direction
> > > > > (instead of IndexType) was ever going to be. Do you know more about
> > why
> > > > it
> > > > > was deprecated or what the devs were going to replace it with?
> > > > >
> > > > > On Thu, Jan 2, 2020 at 6:31 AM Joris Melchior <
> jmelch...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > Apart from Bruce's response (thanks!) it's been very quiet on
> this
> > > > item.
> > > > > >
> > > > > > I'll extend the response time to Jan 10.
> > > > > >
> > > > > > Details at
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=135863477
> > > > > >
> > > > > > Thanks, Joris.
> > > > > >
> > > > > > On Wed, Dec 4, 2019 at 1:03 PM Bruce Schuchardt <
> > > > bschucha...@pivotal.io>
> > > > > > wrote:
> > > > > >
> > > > > > > This proposal seems reasonable to me
> > > > > > >
> > > > > > > On 12/3/19 10:19 AM, Joris Melchior wrote:
> > > > > > > > Ah, that makes sense. I will update!
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Dec 3, 2019 at 12:41 PM Alexander Murmann <
> > > > > amurm...@pivotal.io
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Joris, the "to be reviewed by" field is for a target date by
> > > which
> > > > > to
> > > > > > > wrap
> > > > > > > >> up the