Re: Gateway-sender alert-threshold function not working for gateway sender

[Barry Oglesby]

Mario,

Yes, you're right. The GatewaySender case does not log that warning if the
alert-threshold is set. I updated the ticket with the behavior I see.

Thanks,
Barry Oglesby



On Thu, Jul 11, 2019 at 10:20 AM Mario Ivanac  wrote:

> Description from documentation
>
>
> --alert-threshold
> Maximum number of milliseconds that a region event can remain in the
> gateway sender queue before Geode logs an alert.
>
>
> 
> Šalje: Mario Ivanac 
> Poslano: 8. srpnja 2019. 10:52:20
> Prima: dev@geode.apache.org
> Predmet: Gateway-sender alert-threshold function not working for gateway
> sender
>
> Hi, according to implementation, events which exceed alert threshold are
> reported only for AsyncEventQueue,
>
> but not for GatewaySender (<
> https://issues.apache.org/jira/browse/GEODE-6933>
> https://issues.apache.org/jira/browse/GEODE-6933).
>
>
> Is there any reason for this behavior?
>

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

Hello All,

Since there is no one disagreeing, we are going to start moving forward with 
this.

Thanks,
Mark


> On Jul 11, 2019, at 10:33 AM, Darrel Schneider  wrote:
> 
> Originally we just had a single instance of CachePerfStats per jvm. So all
> the region related stats were always rolled up onto the single
> CachePerfStats. Later we added RegionPerfStats so that users could see what
> was happening on a per region basis. When RegionPerfStats was added it was
> made to extend CachePerfStats but no new stats were added to it.
> 
> I think we now have some stats that only make sense on a Cache (like
> "regions" and "partitionedRegions") but we also have them on
> RegionPerfStats. I thought these used to always return zero on the region
> but I just looked at the implementation and it looks like they just return
> 1 or 0 on the region (depending on if it is partitioned or not). The help
> text says it will be the number of regions on the cache (so the help is
> correct for CachePerfStats but wrong for RegionPerfStats). I would be okay
> with us dropping the stats that only make sense at the cache level from the
> per region stats.
> 
> Something I could not tell from you diagram is if you are cleaning this up.
> Does CacheStats also have everything that is on RegionStats? Or will it now
> just have the stats that are unique to a cache?
> 
> If you change the internal names (like CachePerfStats -> CacheStats) keep
> in mind that you should use the same type name when calling "createType"
> (in this case "CachePerfStats") for backwards compatibility.
> 
> On Thu, Jul 11, 2019 at 9:45 AM Mark Hanson  wrote:
> 
>> See my comments inline..
>> 
>>> On Jul 11, 2019, at 9:36 AM, Darrel Schneider 
>> wrote:
>>> 
>>> Currently we do not make visible per bucket stats. Is the above proposal
>>> just to change the internal implementation but the stats visible in tools
>>> like vsd are unchanged?
>>> 
>> As with my previous e-mail exchange with Jake, I would like to back burner
>> per bucket stats. If it turns out to be a problem, I will bring it back
>> before the group.
>> 
>> My goal is these are internal changes. I would see it as a problem
>> requiring re-evaluation, if this were a meaningful breaking change. E.g.
>> breaks vsd, changes public API
>> An important note would be that fixing a bug in a stat, is not a
>> meaningful breaking change.
>> 
>>> Currently we have an internal CachePerfStats which the internal
>>> RegionPerfStats extends. Does your CacheStats replace CachePerfStats and
>>> your RegionStats replace RegionPerfStats?
>>> 
>> 
>> You are 100% correct.
>> 
>>> Currently we have an internal PartitionedRegionStats class. Does
>>> your PartitionedRegionStats replace it?
>>> 
>> 
>> Yes. I considered that under the “RegionPerfStats” umbrella.
>> 
>>> Are your "*Stats" interfaces and your "*StatsImpl" classes?
>> 
>> Yes.
>> 
>>> 
>>> On Thu, Jul 11, 2019 at 9:29 AM Mark Hanson  wrote:
>>> 
 It depends to be honest. This is just a plan. I would hope to roll them
 up, but I can see a case where you have buckets on different servers,
>> that
 would seem to necessitate that.
 
> On Jul 11, 2019, at 9:26 AM, Jacob Barrett 
>> wrote:
> 
> There isn’t currently a partition stat instance per bucket. Are you
 saying you’re making that a thing now?
> 
>> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
>> 
>> Correct.
>> 
>>> On Jul 11, 2019, at 9:23 AM, Darrel Schneider >> 
 wrote:
>>> 
>>> Why would a PartitionedRegionStatsImpl contain more than one
 RegionStats?
>>> Are these representing the local buckets?
>>> 
 On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson 
 wrote:
 
 PartitionRegionStatsImpl can contain one to many RegionStats
 
> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> 
> Seems reasonable. I'm guessing that CachePerfImpl contains many
 RegionStats. Does PartitionRegionStatsImpl just contain a single
 RegionStats?
> 
> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson >>> >>> mhan...@pivotal.io>> wrote:
> Hi All,
> 
> As many of you may know our structure for our perf stats is not
 great. I
 would like to propose we refactor the code to have the following
 inheritance model, which Kirk and I came up with.
> 
> It is my belief that fixing this will allow future features to be
 implemented in a much less painful way.
> 
> Thoughts?
> 
> Thanks,
> Mark
> 
 
 
>> 
 
 
>> 
>> 

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Michael Stolz]

One thing I will mention regarding DATA:READ:RegionName allowing query
behavior is that we have been asked by some users already to separate
DATA:READ:RegionName from DATA:QUERY:RegionName. This request is to protect
against arbitrary query execution by administrators that can cause huge
resource consumption.

So regardless of all the rest of the proposal, that's something we should
probably consider standardizing on.

--
Mike Stolz
Principal Engineer, Pivotal Cloud Cache
Mobile: +1-631-835-4771



On Thu, Jul 11, 2019 at 11:36 AM Juan José Ramos  wrote:

> Hello all,
>
> Friendly reminder regarding the deadline to rise concerns and/or objections
> regarding the *OQL Method InvocationSecurity Proposal [1]*, I'll go ahead
> and move it to *Development* on July 13th.
> Best regards.
>
> [1]:
>
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt
>
>
> On Mon, Jul 8, 2019 at 3:29 PM Juan José Ramos  wrote:
>
> > Done [1]!.
> > Please remember that, if no major concerns arise before Friday this week,
> > I'll go ahead and move the proposal to *Development* on July 13th.
> > Best regards.
> >
> > [1]:
> >
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt
> >
> > On Fri, Jul 5, 2019 at 3:48 PM Jacob Barrett 
> wrote:
> >
> >> Can you please add a Prior Art section to your proposal discussing these
> >> alternative solutions and why they are insufficient?
> >>
> >> Thanks,
> >> Jake
> >>
> >>
> >> > On Jul 5, 2019, at 10:41 AM, Juan José Ramos 
> wrote:
> >> >
> >> > Hello Jake,
> >> >
> >> > I've replied something similar *here [1]*.
> >> > Long story short, I haven't found anything that really applies to our
> >> use
> >> > case. The "most similar solution" is *Spring Method Security [2]*,
> which
> >> > basically implies annotating methods with explicit configuration about
> >> the
> >> > roles required to execute them. The same goes for *Shiro
> >> **Annotation-based
> >> > Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]*
> approach
> >> > from the proposal is somewhat similar to this, but I've discarded it
> >> > because if forces the user to annotate classes with our own
> annotations,
> >> > basically forcing them to modify their domain model.
> >> > The proposal basically allows our users to use one of the default of
> the
> >> > box implementations and, if they don't like them for whatever reason,
> is
> >> > flexible enough so they can ultimately provide their own.
> >> > Hope this helps.
> >> > Cheers.
> >> >
> >> > [1]:
> >> >
> >>
> https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results
> >> > [2]:
> >> >
> >>
> https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method
> >> > [3]:
> >> >
> >>
> https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization
> >> > [4]:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer
> >> >
> >> > On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett 
> >> wrote:
> >> >
> >> >> So if we don’t want to use the Java built in SecurityManager to solve
> >> >> this, because we feel it's too big or too inflexible for our needs,
> >> have
> >> >> other projects implemented something we can borrow? We can’t be the
> >> first
> >> >> to need something like this if Java’s solution isn’t a good fit.
> >> >>
> >> >> Again I want to avoid inventing something new. What prior art is out
> >> there?
> >> >>
> >> >>
> >> >>> On Jul 4, 2019, at 1:29 PM, Juan José Ramos 
> >> wrote:
> >> >>>
> >> >>> Hello all,
> >> >>>
> >> >>> If you haven't added my email to the spam folder already :-), then
> I'd
> >> >> like
> >> >>> to let you know that I've update again the *Proposal [1]* and
> >> >> incorporated
> >> >>> most of the feedback provided, along with some additional
> information
> >> and
> >> >>> context I missed on the previous versions, thanks all that brought
> >> >> concerns
> >> >>> and suggestions to the discussion. Please take some time to review
> it
> >> >>> thoroughly, adding comments and/or concerns *only on this email
> >> thread*,
> >> >>> all feedback is more than welcome.
> >> >>> If no major concerns arise before July 12th 2019, I'll go ahead and
> >> mark
> >> >>> move the proposal to *Development* on July 13th.
> >> >>> Best regards.
> >> >>>
> >> >>> [1]:
> >> >>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security
> >> >>
> >> >>
> >> >
> >> > --
> >> > Juan José Ramos Cassella
> >> > Senior Technical Support Engineer
> >> > Email: jra...@pivotal.io
> >> > Office#: +353 21 4238611
> >> > Mobile#: +353 87 2074066
> >> > After Hours Contact#: +1 877 477 2269
> >> > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
> >> > How to upload artifacts:
> >> > https://sup

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

My personal thinking was:

Step 1: fix the object structure this round, and try to keep the data output
the same. The downside here is that a chunk of stuff from CachePerfStats would 
then be duplicated into 
RegionPerfStats directly where it was there effectively.  This also means that 
PartitionedRegionStats stops being a standalone class.
It becomes an Implementation of RegionStats (note the name).

Step 2: Add in Metrics facade to track the existing stats.

Step 3: Move existing stats calls to use Metrics decorator.

Step 3: Optional, but desirable,  remove unnecessary metrics from RegionStats 
(note the name change), that really are CacheStats. This would break backward 
compatibility.

Thanks,
Mark


> On Jul 11, 2019, at 10:33 AM, Darrel Schneider  wrote:
> 
> Originally we just had a single instance of CachePerfStats per jvm. So all
> the region related stats were always rolled up onto the single
> CachePerfStats. Later we added RegionPerfStats so that users could see what
> was happening on a per region basis. When RegionPerfStats was added it was
> made to extend CachePerfStats but no new stats were added to it.
> 
> I think we now have some stats that only make sense on a Cache (like
> "regions" and "partitionedRegions") but we also have them on
> RegionPerfStats. I thought these used to always return zero on the region
> but I just looked at the implementation and it looks like they just return
> 1 or 0 on the region (depending on if it is partitioned or not). The help
> text says it will be the number of regions on the cache (so the help is
> correct for CachePerfStats but wrong for RegionPerfStats). I would be okay
> with us dropping the stats that only make sense at the cache level from the
> per region stats.
> 
> Something I could not tell from you diagram is if you are cleaning this up.
> Does CacheStats also have everything that is on RegionStats? Or will it now
> just have the stats that are unique to a cache?
> 
> If you change the internal names (like CachePerfStats -> CacheStats) keep
> in mind that you should use the same type name when calling "createType"
> (in this case "CachePerfStats") for backwards compatibility.
> 
> On Thu, Jul 11, 2019 at 9:45 AM Mark Hanson  wrote:
> 
>> See my comments inline..
>> 
>>> On Jul 11, 2019, at 9:36 AM, Darrel Schneider 
>> wrote:
>>> 
>>> Currently we do not make visible per bucket stats. Is the above proposal
>>> just to change the internal implementation but the stats visible in tools
>>> like vsd are unchanged?
>>> 
>> As with my previous e-mail exchange with Jake, I would like to back burner
>> per bucket stats. If it turns out to be a problem, I will bring it back
>> before the group.
>> 
>> My goal is these are internal changes. I would see it as a problem
>> requiring re-evaluation, if this were a meaningful breaking change. E.g.
>> breaks vsd, changes public API
>> An important note would be that fixing a bug in a stat, is not a
>> meaningful breaking change.
>> 
>>> Currently we have an internal CachePerfStats which the internal
>>> RegionPerfStats extends. Does your CacheStats replace CachePerfStats and
>>> your RegionStats replace RegionPerfStats?
>>> 
>> 
>> You are 100% correct.
>> 
>>> Currently we have an internal PartitionedRegionStats class. Does
>>> your PartitionedRegionStats replace it?
>>> 
>> 
>> Yes. I considered that under the “RegionPerfStats” umbrella.
>> 
>>> Are your "*Stats" interfaces and your "*StatsImpl" classes?
>> 
>> Yes.
>> 
>>> 
>>> On Thu, Jul 11, 2019 at 9:29 AM Mark Hanson  wrote:
>>> 
 It depends to be honest. This is just a plan. I would hope to roll them
 up, but I can see a case where you have buckets on different servers,
>> that
 would seem to necessitate that.
 
> On Jul 11, 2019, at 9:26 AM, Jacob Barrett 
>> wrote:
> 
> There isn’t currently a partition stat instance per bucket. Are you
 saying you’re making that a thing now?
> 
>> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
>> 
>> Correct.
>> 
>>> On Jul 11, 2019, at 9:23 AM, Darrel Schneider >> 
 wrote:
>>> 
>>> Why would a PartitionedRegionStatsImpl contain more than one
 RegionStats?
>>> Are these representing the local buckets?
>>> 
 On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson 
 wrote:
 
 PartitionRegionStatsImpl can contain one to many RegionStats
 
> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> 
> Seems reasonable. I'm guessing that CachePerfImpl contains many
 RegionStats. Does PartitionRegionStatsImpl just contain a single
 RegionStats?
> 
> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson >>> >>> mhan...@pivotal.io>> wrote:
> Hi All,
> 
> As many of you may know our structure for our perf stats is not
 great. I
 would like to propose we refactor the code to have the following
 inheritance model, which Kirk a

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Darrel Schneider]

Originally we just had a single instance of CachePerfStats per jvm. So all
the region related stats were always rolled up onto the single
CachePerfStats. Later we added RegionPerfStats so that users could see what
was happening on a per region basis. When RegionPerfStats was added it was
made to extend CachePerfStats but no new stats were added to it.

I think we now have some stats that only make sense on a Cache (like
"regions" and "partitionedRegions") but we also have them on
RegionPerfStats. I thought these used to always return zero on the region
but I just looked at the implementation and it looks like they just return
1 or 0 on the region (depending on if it is partitioned or not). The help
text says it will be the number of regions on the cache (so the help is
correct for CachePerfStats but wrong for RegionPerfStats). I would be okay
with us dropping the stats that only make sense at the cache level from the
per region stats.

Something I could not tell from you diagram is if you are cleaning this up.
Does CacheStats also have everything that is on RegionStats? Or will it now
just have the stats that are unique to a cache?

If you change the internal names (like CachePerfStats -> CacheStats) keep
in mind that you should use the same type name when calling "createType"
(in this case "CachePerfStats") for backwards compatibility.

On Thu, Jul 11, 2019 at 9:45 AM Mark Hanson  wrote:

> See my comments inline..
>
> > On Jul 11, 2019, at 9:36 AM, Darrel Schneider 
> wrote:
> >
> > Currently we do not make visible per bucket stats. Is the above proposal
> > just to change the internal implementation but the stats visible in tools
> > like vsd are unchanged?
> >
> As with my previous e-mail exchange with Jake, I would like to back burner
> per bucket stats. If it turns out to be a problem, I will bring it back
> before the group.
>
> My goal is these are internal changes. I would see it as a problem
> requiring re-evaluation, if this were a meaningful breaking change. E.g.
> breaks vsd, changes public API
> An important note would be that fixing a bug in a stat, is not a
> meaningful breaking change.
>
> > Currently we have an internal CachePerfStats which the internal
> > RegionPerfStats extends. Does your CacheStats replace CachePerfStats and
> > your RegionStats replace RegionPerfStats?
> >
>
> You are 100% correct.
>
> > Currently we have an internal PartitionedRegionStats class. Does
> > your PartitionedRegionStats replace it?
> >
>
> Yes. I considered that under the “RegionPerfStats” umbrella.
>
> > Are your "*Stats" interfaces and your "*StatsImpl" classes?
>
> Yes.
>
> >
> > On Thu, Jul 11, 2019 at 9:29 AM Mark Hanson  wrote:
> >
> >> It depends to be honest. This is just a plan. I would hope to roll them
> >> up, but I can see a case where you have buckets on different servers,
> that
> >> would seem to necessitate that.
> >>
> >>> On Jul 11, 2019, at 9:26 AM, Jacob Barrett 
> wrote:
> >>>
> >>> There isn’t currently a partition stat instance per bucket. Are you
> >> saying you’re making that a thing now?
> >>>
>  On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
> 
>  Correct.
> 
> > On Jul 11, 2019, at 9:23 AM, Darrel Schneider  >
> >> wrote:
> >
> > Why would a PartitionedRegionStatsImpl contain more than one
> >> RegionStats?
> > Are these representing the local buckets?
> >
> >> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson 
> >> wrote:
> >>
> >> PartitionRegionStatsImpl can contain one to many RegionStats
> >>
> >>> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> >>>
> >>> Seems reasonable. I'm guessing that CachePerfImpl contains many
> >> RegionStats. Does PartitionRegionStatsImpl just contain a single
> >> RegionStats?
> >>>
> >>> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson  >>  >> mhan...@pivotal.io>> wrote:
> >>> Hi All,
> >>>
> >>> As many of you may know our structure for our perf stats is not
> >> great. I
> >> would like to propose we refactor the code to have the following
> >> inheritance model, which Kirk and I came up with.
> >>>
> >>> It is my belief that fixing this will allow future features to be
> >> implemented in a much less painful way.
> >>>
> >>> Thoughts?
> >>>
> >>> Thanks,
> >>> Mark
> >>>
> >>
> >>
> 
> >>
> >>
>
>

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Kirk Lund]

I would recommend doing a spike for this refactoring before trying to
provide a class diagram -- I would personally learn a lot that would have a
major effect on the resulting class design that I can only guess about up
front.

Another major challenge is in trying to keep the resulting .gfs file
unchanged while straightening up the class design. The effort may be
worthwhile, but I don't think the spike will be quick or trivial.

The main reason to have any 1-to-many relationships in this class design
(ex: CacheStats to RegionStats) is to build in aggregation, such as number
of puts per Region and number of puts across every Region in the Cache. We
also need PartitionedRegion aggregates such as number of puts across every
BucketRegion for a specific PartitionedRegion -- which is also a 1-to-many
of PartitionedRegionStats to (Bucket)RegionStats relationship (and of
course a Cache can have many PartitionedRegions). I would prefer to come up
with some sort of reusable statistic aggregation design as part of the
spike.

On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson  wrote:

> Hi All,
>
> As many of you may know our structure for our perf stats is not great. I
> would like to propose we refactor the code to have the following
> inheritance model, which Kirk and I came up with.
>
> It is my belief that fixing this will allow future features to be
> implemented in a much less painful way.
>
> Thoughts?
>
> Thanks,
> Mark
>

Re: Gateway-sender alert-threshold function not working for gateway sender

[Mario Ivanac]

Description from documentation


--alert-threshold
Maximum number of milliseconds that a region event can remain in the gateway 
sender queue before Geode logs an alert.



Šalje: Mario Ivanac 
Poslano: 8. srpnja 2019. 10:52:20
Prima: dev@geode.apache.org
Predmet: Gateway-sender alert-threshold function not working for gateway sender

Hi, according to implementation, events which exceed alert threshold are 
reported only for AsyncEventQueue,

but not for GatewaySender 
(https://issues.apache.org/jira/browse/GEODE-6933).


Is there any reason for this behavior?

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

See my comments inline..

> On Jul 11, 2019, at 9:36 AM, Darrel Schneider  wrote:
> 
> Currently we do not make visible per bucket stats. Is the above proposal
> just to change the internal implementation but the stats visible in tools
> like vsd are unchanged?
> 
As with my previous e-mail exchange with Jake, I would like to back burner per 
bucket stats. If it turns out to be a problem, I will bring it back before the 
group.

My goal is these are internal changes. I would see it as a problem requiring 
re-evaluation, if this were a meaningful breaking change. E.g. breaks vsd, 
changes public API
An important note would be that fixing a bug in a stat, is not a meaningful 
breaking change.

> Currently we have an internal CachePerfStats which the internal
> RegionPerfStats extends. Does your CacheStats replace CachePerfStats and
> your RegionStats replace RegionPerfStats?
> 

You are 100% correct.

> Currently we have an internal PartitionedRegionStats class. Does
> your PartitionedRegionStats replace it?
> 

Yes. I considered that under the “RegionPerfStats” umbrella.

> Are your "*Stats" interfaces and your "*StatsImpl" classes?

Yes.

> 
> On Thu, Jul 11, 2019 at 9:29 AM Mark Hanson  wrote:
> 
>> It depends to be honest. This is just a plan. I would hope to roll them
>> up, but I can see a case where you have buckets on different servers, that
>> would seem to necessitate that.
>> 
>>> On Jul 11, 2019, at 9:26 AM, Jacob Barrett  wrote:
>>> 
>>> There isn’t currently a partition stat instance per bucket. Are you
>> saying you’re making that a thing now?
>>> 
 On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
 
 Correct.
 
> On Jul 11, 2019, at 9:23 AM, Darrel Schneider 
>> wrote:
> 
> Why would a PartitionedRegionStatsImpl contain more than one
>> RegionStats?
> Are these representing the local buckets?
> 
>> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson 
>> wrote:
>> 
>> PartitionRegionStatsImpl can contain one to many RegionStats
>> 
>>> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
>>> 
>>> Seems reasonable. I'm guessing that CachePerfImpl contains many
>> RegionStats. Does PartitionRegionStatsImpl just contain a single
>> RegionStats?
>>> 
>>> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson > > mhan...@pivotal.io>> wrote:
>>> Hi All,
>>> 
>>> As many of you may know our structure for our perf stats is not
>> great. I
>> would like to propose we refactor the code to have the following
>> inheritance model, which Kirk and I came up with.
>>> 
>>> It is my belief that fixing this will allow future features to be
>> implemented in a much less painful way.
>>> 
>>> Thoughts?
>>> 
>>> Thanks,
>>> Mark
>>> 
>> 
>> 
 
>> 
>> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

I accept that point. I would certainly like to minimize the work and per bucket 
status would seem undesirable.  I can totally agree to back burner that until 
some point in the future at which point we agree to move forward on it. I was 
just anticipating that would fall out necessarily. That said, let’s consider 
that back burnered for this proposal.

The main classes I really want to cleanup with this proposal are CachePerfStats 
and RegionPerfStats.

Thanks for the good constructive feedback.

> On Jul 11, 2019, at 9:36 AM, Jacob Barrett  wrote:
> 
> So is the root of this proposal really about expanding our current stats 
> collection to include per-bucket stats. If not I would really separate these 
> two ideas. First focus on refactoring these stats classes to be more 
> manageable and readable. Then propose adding per-bucket stats as a thing 
> separately. If this discussion is really about per-bucket stats then let’s 
> focus the subject on that and not really worry about any internal refactoring 
> that must happen to make it work.
> 
>> On Jul 11, 2019, at 9:29 AM, Mark Hanson  wrote:
>> 
>> It depends to be honest. This is just a plan. I would hope to roll them up, 
>> but I can see a case where you have buckets on different servers, that would 
>> seem to necessitate that.
>> 
>>> On Jul 11, 2019, at 9:26 AM, Jacob Barrett  wrote:
>>> 
>>> There isn’t currently a partition stat instance per bucket. Are you saying 
>>> you’re making that a thing now?
>>> 
 On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
 
 Correct.
 
> On Jul 11, 2019, at 9:23 AM, Darrel Schneider  
> wrote:
> 
> Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
> Are these representing the local buckets?
> 
>> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:
>> 
>> PartitionRegionStatsImpl can contain one to many RegionStats
>> 
>>> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
>>> 
>>> Seems reasonable. I'm guessing that CachePerfImpl contains many
>> RegionStats. Does PartitionRegionStatsImpl just contain a single
>> RegionStats?
>>> 
>>> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson > mhan...@pivotal.io>> wrote:
>>> Hi All,
>>> 
>>> As many of you may know our structure for our perf stats is not great. I
>> would like to propose we refactor the code to have the following
>> inheritance model, which Kirk and I came up with.
>>> 
>>> It is my belief that fixing this will allow future features to be
>> implemented in a much less painful way.
>>> 
>>> Thoughts?
>>> 
>>> Thanks,
>>> Mark
>>> 
>> 
>> 
 
>> 
> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Darrel Schneider]

Currently we do not make visible per bucket stats. Is the above proposal
just to change the internal implementation but the stats visible in tools
like vsd are unchanged?

Currently we have an internal CachePerfStats which the internal
RegionPerfStats extends. Does your CacheStats replace CachePerfStats and
your RegionStats replace RegionPerfStats?

Currently we have an internal PartitionedRegionStats class. Does
your PartitionedRegionStats replace it?

Are your "*Stats" interfaces and your "*StatsImpl" classes?

On Thu, Jul 11, 2019 at 9:29 AM Mark Hanson  wrote:

> It depends to be honest. This is just a plan. I would hope to roll them
> up, but I can see a case where you have buckets on different servers, that
> would seem to necessitate that.
>
> > On Jul 11, 2019, at 9:26 AM, Jacob Barrett  wrote:
> >
> > There isn’t currently a partition stat instance per bucket. Are you
> saying you’re making that a thing now?
> >
> >> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
> >>
> >> Correct.
> >>
> >>> On Jul 11, 2019, at 9:23 AM, Darrel Schneider 
> wrote:
> >>>
> >>> Why would a PartitionedRegionStatsImpl contain more than one
> RegionStats?
> >>> Are these representing the local buckets?
> >>>
>  On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson 
> wrote:
> 
>  PartitionRegionStatsImpl can contain one to many RegionStats
> 
> > On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> >
> > Seems reasonable. I'm guessing that CachePerfImpl contains many
>  RegionStats. Does PartitionRegionStatsImpl just contain a single
>  RegionStats?
> >
> > On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson    mhan...@pivotal.io>> wrote:
> > Hi All,
> >
> > As many of you may know our structure for our perf stats is not
> great. I
>  would like to propose we refactor the code to have the following
>  inheritance model, which Kirk and I came up with.
> >
> > It is my belief that fixing this will allow future features to be
>  implemented in a much less painful way.
> >
> > Thoughts?
> >
> > Thanks,
> > Mark
> >
> 
> 
> >>
>
>

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Jacob Barrett]

So is the root of this proposal really about expanding our current stats 
collection to include per-bucket stats. If not I would really separate these 
two ideas. First focus on refactoring these stats classes to be more manageable 
and readable. Then propose adding per-bucket stats as a thing separately. If 
this discussion is really about per-bucket stats then let’s focus the subject 
on that and not really worry about any internal refactoring that must happen to 
make it work.

> On Jul 11, 2019, at 9:29 AM, Mark Hanson  wrote:
> 
> It depends to be honest. This is just a plan. I would hope to roll them up, 
> but I can see a case where you have buckets on different servers, that would 
> seem to necessitate that.
> 
>> On Jul 11, 2019, at 9:26 AM, Jacob Barrett  wrote:
>> 
>> There isn’t currently a partition stat instance per bucket. Are you saying 
>> you’re making that a thing now?
>> 
>>> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
>>> 
>>> Correct.
>>> 
 On Jul 11, 2019, at 9:23 AM, Darrel Schneider  
 wrote:
 
 Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
 Are these representing the local buckets?
 
> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:
> 
> PartitionRegionStatsImpl can contain one to many RegionStats
> 
>> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
>> 
>> Seems reasonable. I'm guessing that CachePerfImpl contains many
> RegionStats. Does PartitionRegionStatsImpl just contain a single
> RegionStats?
>> 
>> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson  mhan...@pivotal.io>> wrote:
>> Hi All,
>> 
>> As many of you may know our structure for our perf stats is not great. I
> would like to propose we refactor the code to have the following
> inheritance model, which Kirk and I came up with.
>> 
>> It is my belief that fixing this will allow future features to be
> implemented in a much less painful way.
>> 
>> Thoughts?
>> 
>> Thanks,
>> Mark
>> 
> 
> 
>>> 
> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

It depends to be honest. This is just a plan. I would hope to roll them up, but 
I can see a case where you have buckets on different servers, that would seem 
to necessitate that.

> On Jul 11, 2019, at 9:26 AM, Jacob Barrett  wrote:
> 
> There isn’t currently a partition stat instance per bucket. Are you saying 
> you’re making that a thing now?
> 
>> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
>> 
>> Correct.
>> 
>>> On Jul 11, 2019, at 9:23 AM, Darrel Schneider  wrote:
>>> 
>>> Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
>>> Are these representing the local buckets?
>>> 
 On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:
 
 PartitionRegionStatsImpl can contain one to many RegionStats
 
> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> 
> Seems reasonable. I'm guessing that CachePerfImpl contains many
 RegionStats. Does PartitionRegionStatsImpl just contain a single
 RegionStats?
> 
> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson >>> mhan...@pivotal.io>> wrote:
> Hi All,
> 
> As many of you may know our structure for our perf stats is not great. I
 would like to propose we refactor the code to have the following
 inheritance model, which Kirk and I came up with.
> 
> It is my belief that fixing this will allow future features to be
 implemented in a much less painful way.
> 
> Thoughts?
> 
> Thanks,
> Mark
> 
 
 
>> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Jacob Barrett]

There isn’t currently a partition stat instance per bucket. Are you saying 
you’re making that a thing now?

> On Jul 11, 2019, at 9:24 AM, Mark Hanson  wrote:
> 
> Correct.
> 
>> On Jul 11, 2019, at 9:23 AM, Darrel Schneider  wrote:
>> 
>> Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
>> Are these representing the local buckets?
>> 
>>> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:
>>> 
>>> PartitionRegionStatsImpl can contain one to many RegionStats
>>> 
 On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
 
 Seems reasonable. I'm guessing that CachePerfImpl contains many
>>> RegionStats. Does PartitionRegionStatsImpl just contain a single
>>> RegionStats?
 
 On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson >> mhan...@pivotal.io>> wrote:
 Hi All,
 
 As many of you may know our structure for our perf stats is not great. I
>>> would like to propose we refactor the code to have the following
>>> inheritance model, which Kirk and I came up with.
 
 It is my belief that fixing this will allow future features to be
>>> implemented in a much less painful way.
 
 Thoughts?
 
 Thanks,
 Mark
 
>>> 
>>> 
> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Mark Hanson]

Correct.

> On Jul 11, 2019, at 9:23 AM, Darrel Schneider  wrote:
> 
> Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
> Are these representing the local buckets?
> 
> On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:
> 
>> PartitionRegionStatsImpl can contain one to many RegionStats
>> 
>>> On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
>>> 
>>> Seems reasonable. I'm guessing that CachePerfImpl contains many
>> RegionStats. Does PartitionRegionStatsImpl just contain a single
>> RegionStats?
>>> 
>>> On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson > mhan...@pivotal.io>> wrote:
>>> Hi All,
>>> 
>>> As many of you may know our structure for our perf stats is not great. I
>> would like to propose we refactor the code to have the following
>> inheritance model, which Kirk and I came up with.
>>> 
>>> It is my belief that fixing this will allow future features to be
>> implemented in a much less painful way.
>>> 
>>> Thoughts?
>>> 
>>> Thanks,
>>> Mark
>>> 
>> 
>> 

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Darrel Schneider]

Why would a PartitionedRegionStatsImpl contain more than one RegionStats?
Are these representing the local buckets?

On Wed, Jul 10, 2019 at 4:57 PM Mark Hanson  wrote:

> PartitionRegionStatsImpl can contain one to many RegionStats
>
> > On Jul 10, 2019, at 4:53 PM, Dan Smith  wrote:
> >
> > Seems reasonable. I'm guessing that CachePerfImpl contains many
> RegionStats. Does PartitionRegionStatsImpl just contain a single
> RegionStats?
> >
> > On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson  mhan...@pivotal.io>> wrote:
> > Hi All,
> >
> > As many of you may know our structure for our perf stats is not great. I
> would like to propose we refactor the code to have the following
> inheritance model, which Kirk and I came up with.
> >
> > It is my belief that fixing this will allow future features to be
> implemented in a much less painful way.
> >
> > Thoughts?
> >
> > Thanks,
> > Mark
> >
>
>

Re: [DISCUSS]: ClusterManagementService configuration objects

[Jinmei Liao]

They will be public API, currently, mainly used by the
ClusterManagementServivice to configure the cluster.

On Wed, Jul 10, 2019 at 9:06 PM Jacob Barrett  wrote:

> Are these new objects public API or internal?
>
> > On Jul 10, 2019, at 1:16 PM, Jinmei Liao  wrote:
> >
> > We've been working on a new and improved ClusterManagmentService for a
> > while now. It allows developers/administrators to manage the clusters
> > through rest calls instead of having to use gfsh (more info here:
> >
> https://cwiki.apache.org/confluence/display/GEODE/Cluster+Management+Service
> > ).
> >
> > Up until now, we've been using the auto-generated POJOs from cache.xsd as
> > configuration objects because they contain all cache.xml has to offer. We
> > do this with the hope that whatever you can configure using cache.xml,
> you
> > can configure using this new model. But we also ran into these problems:
> > 1. auto-generated code is messy, we had to make way too many adjustments
> to
> > the generated code in order to improve usage.
> > 2. when xsd change, we would have a hard time to re-incorporate the
> change
> > we made to these objects to the newly generated code.
> > 3. these domain objects have deprecated constructs that we do not want to
> > expose as public api.
> > 4. configuration objects needs to be more intuitive/simple for users to
> > configure.
> >
> > So we are proposing to introduce a new set of configuration objects that
> > are dedicated to CMS, and keep the auto-generated xml domain objects as
> > internal. And yes, it won't have as much attributes as those xml domain
> > objects in the beginning, but we are hoping they will catch up soon.
> >
> > Concerns/comments?
> > --
> > Cheers
> >
> > Jinmei
>


-- 
Cheers

Jinmei

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Juan José Ramos]

Hello all,

Friendly reminder regarding the deadline to rise concerns and/or objections
regarding the *OQL Method InvocationSecurity Proposal [1]*, I'll go ahead
and move it to *Development* on July 13th.
Best regards.

[1]:
https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt


On Mon, Jul 8, 2019 at 3:29 PM Juan José Ramos  wrote:

> Done [1]!.
> Please remember that, if no major concerns arise before Friday this week,
> I'll go ahead and move the proposal to *Development* on July 13th.
> Best regards.
>
> [1]:
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt
>
> On Fri, Jul 5, 2019 at 3:48 PM Jacob Barrett  wrote:
>
>> Can you please add a Prior Art section to your proposal discussing these
>> alternative solutions and why they are insufficient?
>>
>> Thanks,
>> Jake
>>
>>
>> > On Jul 5, 2019, at 10:41 AM, Juan José Ramos  wrote:
>> >
>> > Hello Jake,
>> >
>> > I've replied something similar *here [1]*.
>> > Long story short, I haven't found anything that really applies to our
>> use
>> > case. The "most similar solution" is *Spring Method Security [2]*, which
>> > basically implies annotating methods with explicit configuration about
>> the
>> > roles required to execute them. The same goes for *Shiro
>> **Annotation-based
>> > Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]* approach
>> > from the proposal is somewhat similar to this, but I've discarded it
>> > because if forces the user to annotate classes with our own annotations,
>> > basically forcing them to modify their domain model.
>> > The proposal basically allows our users to use one of the default of the
>> > box implementations and, if they don't like them for whatever reason, is
>> > flexible enough so they can ultimately provide their own.
>> > Hope this helps.
>> > Cheers.
>> >
>> > [1]:
>> >
>> https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results
>> > [2]:
>> >
>> https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method
>> > [3]:
>> >
>> https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization
>> > [4]:
>> >
>> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer
>> >
>> > On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett 
>> wrote:
>> >
>> >> So if we don’t want to use the Java built in SecurityManager to solve
>> >> this, because we feel it's too big or too inflexible for our needs,
>> have
>> >> other projects implemented something we can borrow? We can’t be the
>> first
>> >> to need something like this if Java’s solution isn’t a good fit.
>> >>
>> >> Again I want to avoid inventing something new. What prior art is out
>> there?
>> >>
>> >>
>> >>> On Jul 4, 2019, at 1:29 PM, Juan José Ramos 
>> wrote:
>> >>>
>> >>> Hello all,
>> >>>
>> >>> If you haven't added my email to the spam folder already :-), then I'd
>> >> like
>> >>> to let you know that I've update again the *Proposal [1]* and
>> >> incorporated
>> >>> most of the feedback provided, along with some additional information
>> and
>> >>> context I missed on the previous versions, thanks all that brought
>> >> concerns
>> >>> and suggestions to the discussion. Please take some time to review it
>> >>> thoroughly, adding comments and/or concerns *only on this email
>> thread*,
>> >>> all feedback is more than welcome.
>> >>> If no major concerns arise before July 12th 2019, I'll go ahead and
>> mark
>> >>> move the proposal to *Development* on July 13th.
>> >>> Best regards.
>> >>>
>> >>> [1]:
>> >>>
>> >>
>> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security
>> >>
>> >>
>> >
>> > --
>> > Juan José Ramos Cassella
>> > Senior Technical Support Engineer
>> > Email: jra...@pivotal.io
>> > Office#: +353 21 4238611
>> > Mobile#: +353 87 2074066
>> > After Hours Contact#: +1 877 477 2269
>> > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
>> > How to upload artifacts:
>> > https://support.pivotal.io/hc/en-us/articles/204369073
>> > How to escalate a ticket:
>> > https://support.pivotal.io/hc/en-us/articles/203809556
>> >
>> > [image: support]  [image: twitter]
>> >  [image: linkedin]
>> >  [image: facebook]
>> >  [image: google plus]
>> >  [image: youtube]
>> > <
>> https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>>
>>
>
> --
> Juan José Ramos Cassella
> Senior Technical Support Engineer
> Email: jra...@pivotal.io
> Office#: +353 21 4238611
> Mobile#: +353 87 2074066
> After Hours Contact#: +1 877 477 2269
> Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
> How to upload artifacts