Re: Strange Performance Issue with Large number Region Creation

[Michael Stolz]

I can't think of any reason why any use case could need 1500 Regions.

Regions are heavyweight constructs more similar in nature to Unix mounts
than Unix directories.

We usually use simple naming conventions for keys to simulate directory
structures.

So I would recommend that you create 1 Region, and store 1500 named hash
maps into it.

Make sense?

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 16, 2016 11:15 AM, "Avinash Dongre"  wrote:

> Hi,
>
> I am seeing strange performance issue when I want to create large number of
> regions.
> Test I am doing with 1500 Regions.
>
> ~ 2 minutes for creating first 1130 regions
> ~7 minutes for creating remaining 370 regions.
>
> Following is example code :
>
> Cache geodeCache = this.createCache();
> final String REGION_NAME = "Region_Name";
> for ( int i = 0; i < 1500; i++) {
>   RegionFactory rf =
> geodeCache.createRegionFactory(RegionShortcut.PARTITION_
> PERSISTENT_OVERFLOW);
>   rf.setDiskSynchronous(true);
>   rf.setEvictionAttributes(EvictionAttributes.createLIFOEntryAttributes(1,
> EvictionAction.OVERFLOW_TO_DISK));
>   Region region = rf.create(REGION_NAME + "_" + i);
> }
> geodeCache.close();
>
>
> If I remove following code from createVMRegion ( GemFireCacheImpl.java ),
> then I could create Regions in 2 minutes.
>
> if (!rgn.isInternalRegion()) {
>   system.handleResourceEvent(ResourceEvent.REGION_CREATE, rgn);
> }
>
> Any help or pointers why this is happening ?
>
>
> Thanks
>
> Avinash
>

Re: Open source the integrated development framework for geode/gemfire!

[Michael Stolz]

I am too!

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 11, 2016 11:35 AM, "Anthony Baker"  wrote:

> Cool, I’m very interested to learn more!
>
> Anthony
>
> > On Nov 11, 2016, at 8:27 AM, theseusyang  wrote:
> >
> > Hi All,
> >
> > I plan to open source a integrated development framework for
> geode/gemfire
> > at next month.
> >
> > this framework has already been used successfully in12306 site and other
> > financial customers from Banks.
> >
> >
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-geode-
> incubating-developers-forum.70738.x6.nabble.com/Open-
> source-the-integrated-development-framework-for-geode-gemfire-tp9561.html
> > Sent from the Apache Geode (Incubating) Developers Forum mailing list
> archive at Nabble.com.
>
>

Re: Gfsh Parsing

[Michael Stolz]

Can we suggest improvements to the Spring help capabilities? The Spring
community tends to be very responsive to good suggestions.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 4, 2016 8:27 AM, "Jinmei Liao"  wrote:

> We have several jira issues related to gfsh parsing (GEODE-1598,
> GEODE-1912). After spending some time understanding how the parsing works,
> I found out we have three components intertwined together all trying to do
> parsing: Gfsh, JoptSimple and Spring Shell. I started an experiment by
> getting rid of Gfsh and JoptSimple parsing and only using Spring Shell. The
> outcome is I am able to maintain the current parsing and tabbing completion
> capabilities (and fix a few bugs) by removing 40+ files. The only
> difference I see so far lies in the help and hint messages. It seems the
> main reason we are using these home backed Gfsh parsing is to provide  more
> readable help messages. Below are the differences:
>
> Using Spring Shell's provided help:
>
> Using Gfsh Parsing with JoptSimple:
>
>
> ​
> ​I do like the outcome of the latter, but added complexity of the code is
> too expensive to bear. So I am asking the community how important it is to
> maintain the current style of help? Can we do with the cheaper way by just
> using whatever provided by the libraries?
>
> --
> Cheers
>
> Jinmei
>

Re: [DISCUSS] Graduation

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Oct 25, 2016 8:25 PM, "Roman Shaposhnik"  wrote:

> Hi!
>
> with the 1.0.0-incubating release officially out (huge kudos
> to the team!) I think it is time we officially start our graduation
> discussion. Process-wise graduation consists of drafting
> a board resolution, getting it approved by the IPMC and finally
> submitting it to the ASF board's consideration. At the very minimum
> your resolution will contain:
> 1. A name of the project (I assume that'll be Geode)
> 2. A list of proposed PMC
> 3. A proposed PMC chair
> A good example of a resolution can be found here:
> http://markmail.org/message/7ow2kdqa3rdx3x5k
>
> On #2 my suggestion would be to have an opt-in system. Basically
> we will kick off the thread off on private@geode asking current PPMC
> members if they are willing to continue on the PMC.
>
> On #3 I typically recommend podlings I mento to setup a rotating chair
> policy. This is, in no way, an ASF requirement so feel free to ignore it,
> but it worked well before. The chair will be expected up for rotation every
> year. It will be more that ok for the same person to self-nominate once
> the year is up -- but at the same time it'll be up to the same person to
> actually kick off a thread asking if anybody else is interested in serving
> as a chair for the next year. Of course, if there multiple candidates there
> will have to be a vote.
>
> Speaking of self-nomination -- the same thread that we're going to kick
> off as part of solving for #2 will ask for folks to self-nominate as an
> initial
> chair to be listed on the resolution.
>
> Unless somebody objects strongly to my #2 and #3 proposals I'm going
> to kick of this thread on private@.
>
> With that in mind, lets make the rest of the discussion on dev@ to be
> about
> collecting the datapoints to present to IPCM as part of us asking them to
> vote YES on our graduation. You can see my initial list of these data
> points
> below. Please, please add yours:
>
> Project status:
>  http://incubator.apache.org/projects/geode.html
>
> Maturity assessment:
>  https://cwiki.apache.org/confluence/display/GEODE/Geode+Podling+Maturity+
> Assessment
>
> 4 releases
>
> 2734 commits on develop
> 269 PR”s
> 92 contributors across all branches
>
> dev list averaged ~650 msgs/month in 2016
> user list averaged ~90 msgs/month in 2016
> 267 unique posters
>
> 1900 issues created
> 1230 issues resolved
>
> 6 new committers / PPMC members added
>
> Quick bump:  as far as committer diversity goes here’s the picture:
>
> committer domains
>  reasonably active
>   pivotal.io
>   kollective.com (Mark Bretl)
>   ampool.io
>   silverspringnet.com (Sai Boorlagadda)
>
>  inactive
>   snappydata.io
>   newrelic.com (Qihong)
>   google.com (Sourabh)
>   microsoft.com (Ashvin)
>   weave.works (Stuart)
>   elastic.co (Catherine)
>   dell.com (Lyndon, …)
>   vmware.com
>   brsg.io
>   cobaltdigital.marketing
>   cdkglobal.com
>   dev9.com
>
> Thanks,
> Roman.
>

Re: [ANNOUNCE] Apache Geode 1.0.0-incubating released

[Michael Stolz]

Yay! This is great news!

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, Oct 25, 2016 at 6:33 PM, Swapnil Bawaskar 
wrote:

> The Apache Geode team is proud to announce Apache Geode release
> 1.0.0-incubating.
>
> Apache Geode (incubating) is a data management platform that provides a
> database-like consistency model, reliable transaction processing and a
> shared-nothing architecture to maintain very low latency performance with
> high concurrency processing.
>
> Download the new release here:
> http://geode.incubator.apache.org/releases/
>
> Changes since the last release include renaming packages from
> com.gemstone.gemfire to org.apache.geode, bundling docs with the source
> distribution, securing the REST API, and 225 other bug fixes and minor
> improvements.
> See the release notes for more details:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12318420&version=12332343
>
> To get started with Apache Geode check out the Geode in 5 minutes tutorial:
> https://cwiki.apache.org/confluence/display/GEODE/Index#Index-
> Geodein5minutesGeodein5minutes
>
> To learn more and get involved, please visit our website in join our
> mailing
> lists:
> http://geode.incubator.apache.org/
>
> Regards,
> The Apache Geode (incubating) team
>
> =
> *Disclaimer*
>
> Apache Geode is an effort undergoing incubation at The Apache Software
> Foundation (ASF), sponsored by the name of Apache Incubator PMC. Incubation
> is required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
>

Re: [ANNOUNCE] Donation of Geode documentation

[Michael Stolz]

Fabulous news!
This is a major step in our Apache.org journey.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Sep 29, 2016 11:27 PM, "Anthony Baker"  wrote:

> I am pleased to announce the donation of Geode documentation to the
> Geode community.
>
> The documentation provides a complete user guide for Geode. This
> donation includes the source necessary to build the documentation site
> currently hosted by Pivotal [1]. The documentation source has been
> converted into a community-friendly markdown format.
>
> The Software Grant Agreement for this code has been accepted by the ASF
> secretary.
>
> The donated source currently sits in a separate branch in the Geode
> repository named staging/docs-grant1 [2] and is awaiting community
> review. I encourage everyone in the Geode community to review this
> donation and provide feedback. Once the community has reached a
> consensus we can determine next steps and how this code might get
> merged into the develop branch [3]. This will allow the geode
> community to accept documentation contributions and self-host the
> documentation on the project website. Your suggestions are most
> welcome!
>
> Thanks,
> Anthony
>
> [1] http://geode.docs.pivotal.io
> [2] https://git-wip-us.apache.org/repos/asf?p=incubator-geode.
> git;a=tree;h=refs/heads/staging/docs-grant1;hb=refs/
> heads/staging/docs-grant1
> [3] https://issues.apache.org/jira/browse/GEODE-1952
>

Re: Hosting the docs (was Re: [VOTE] Release Apache Geode (incubating) 1.0.0-incubating - RC2)

[Michael Stolz]

Its really important to ship pdf docs because its very difficult to search
otherwise.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, Oct 18, 2016 at 12:49 PM, Joey McAllister 
wrote:

> @Dan: I didn't realize there was a docs link in the top-level README. We
> can change that for the next release, and I can look into redirecting
> geode.docs.pivotal.io to geode.incubator.apache.org/docs/ in the meantime,
> once the docs are posted there..
>
> @William: We don't currently have a PDF for these docs. I'm researching
> options.
>
> On Mon, Oct 17, 2016 at 6:53 PM William Markito 
> wrote:
>
> IHMO, it would be really nice to ship a PDF version of the docs.
>
> About the examples, if we could package and ship sources only for that
> module, that would be cool as well.
>
>
>
> On Mon, Oct 17, 2016 at 5:21 PM, Dan Smith  wrote:
>
> > Along these lines, should we distributing the docs with the binary
> release?
> > Or maybe just providing a link to them? The README.md shipped with
> 1.0.RC2
> > points to http://geode.docs.pivotal.io/ .
> >
> > What about geode-examples? Should that be part of the binary release?
> >
> > -Dan
> >
> > On Mon, Oct 17, 2016 at 2:21 PM, Joey McAllister  >
> > wrote:
> >
> > > @Roman: Nothing that I can think of, apart from giving the community
> time
> > > to offer feedback here (which, it looks like, is all positive). William
> > > Markito and I were able to build and test a local version of the
> website
> > > with the docs included.
> > >
> > > Based on the +1s here, I'd like to go ahead and push the current docs
> to
> > > the website. I'll also document the process in a README.
> > >
> > > On Mon, Oct 17, 2016 at 12:49 PM Roman Shaposhnik <
> ro...@shaposhnik.org>
> > > wrote:
> > >
> > > > On Mon, Oct 17, 2016 at 10:57 AM, Anthony Baker 
> > > wrote:
> > > > > Since the geode docs have now been merged to the develop branch,
> > let’s
> > > > start
> > > > > hosting them on http://geode.apache.org.  Thoughts?
> > > >
> > > > Huge +1! Anything stopping you from pushing the first update and
> start
> > > > maintaining
> > > > it a'la Hadoop:
> > > > https://hadoop.apache.org/docs/
> > > > ?
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> >
>
>
>
> --
>
> ~/William
>

Re: Hosting the docs (was Re: [VOTE] Release Apache Geode (incubating) 1.0.0-incubating - RC2)

[Michael Stolz]

I think we should ship the pdf docs. Geode shouldn't point to Pivotal for
its docs.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Oct 17, 2016 9:53 PM, "William Markito"  wrote:

> IHMO, it would be really nice to ship a PDF version of the docs.
>
> About the examples, if we could package and ship sources only for that
> module, that would be cool as well.
>
>
>
> On Mon, Oct 17, 2016 at 5:21 PM, Dan Smith  wrote:
>
> > Along these lines, should we distributing the docs with the binary
> release?
> > Or maybe just providing a link to them? The README.md shipped with
> 1.0.RC2
> > points to http://geode.docs.pivotal.io/ .
> >
> > What about geode-examples? Should that be part of the binary release?
> >
> > -Dan
> >
> > On Mon, Oct 17, 2016 at 2:21 PM, Joey McAllister  >
> > wrote:
> >
> > > @Roman: Nothing that I can think of, apart from giving the community
> time
> > > to offer feedback here (which, it looks like, is all positive). William
> > > Markito and I were able to build and test a local version of the
> website
> > > with the docs included.
> > >
> > > Based on the +1s here, I'd like to go ahead and push the current docs
> to
> > > the website. I'll also document the process in a README.
> > >
> > > On Mon, Oct 17, 2016 at 12:49 PM Roman Shaposhnik <
> ro...@shaposhnik.org>
> > > wrote:
> > >
> > > > On Mon, Oct 17, 2016 at 10:57 AM, Anthony Baker 
> > > wrote:
> > > > > Since the geode docs have now been merged to the develop branch,
> > let’s
> > > > start
> > > > > hosting them on http://geode.apache.org.  Thoughts?
> > > >
> > > > Huge +1! Anything stopping you from pushing the first update and
> start
> > > > maintaining
> > > > it a'la Hadoop:
> > > > https://hadoop.apache.org/docs/
> > > > ?
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> >
>
>
>
> --
>
> ~/William
>

Re: security properties in the cluster config

[Michael Stolz]

I am in favor of keeping the SSL thoughts separate from the RBAC thoughts,
but I don't see any reason they couldn't share the same repository.

That said though, does putting it all into the Cluster Configuration
Manager (CCM) make it so that you can only have security if you are using
CCM for configuration?


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Sep 23, 2016 at 1:48 PM, Jinmei Liao  wrote:

> Hi, All,
>
> I am working on this ticket:
> https://issues.apache.org/jira/browse/GEODE-1659. Basically, currently,
> any
> member(locator or server) needs to specify its own security-manager in
> order to protect its data which could leads to misconfiguration and data
> leak. So we would like to put it into the cluster configuration so any
> member who wants to join the cluster will need to apply the same security
> measures.
>
> Now Here is my question, should we only put the "security-manager" and
> "security-post-processor" in the cluster config or any "security-*"
> settings, which include SSL settings as well.
>
> Thanks!
>
> --
> Cheers
>
> Jinmei
>

Re: jvsd

[Michael Stolz]

+1 If its not part of a release its documentation shouldn't be either.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Sep 22, 2016 at 7:58 PM, Joey McAllister 
wrote:

> If it isn't on develop or planned for release, then I vote for removing it
> from the user guide altogether. I like the recommendation to keep this and
> similar (non-develop branch) information on the wiki, so the community can
> still easily access it.
>
> On Thu, Sep 22, 2016 at 4:51 PM Kirk Lund  wrote:
>
> > JVSD isn't labeled as Experimental because it isn't even on develop or in
> > part of any Geode release candidate. As far as I know, it exists only on
> an
> > incomplete and out-of-date feature branch. Hence, my vote is to remove
> any
> > mention of it from the docs until it merges to develop or at a minimum
> gets
> > updated (rebased) from develop (or M3).
> >
> > Has anyone done any rebasing or other work on the JVSD branch that I'm
> not
> > aware of?
> >
> > -Kirk
> >
> > On Thursday, September 22, 2016, Swapnil Bawaskar 
> > wrote:
> >
> > > I would vote for including it as an "experimental" feature.
> > >
> > > On Thu, Sep 22, 2016 at 4:37 PM, Dave Barnes  > > > wrote:
> > >
> > > > Process for handling Experimental docs is still being hammered out.
> > > Common
> > > > element among working scenarios is isolation from the body of the
> User
> > > > Guide proper, so I'll remove the JVSD component from the User Guide's
> > > Tools
> > > > and Modules section.
> > > > Could go on the Wiki, could go in an appendix. We'll see what
> emerges.
> > > > Any favorites among the readers of this thread?
> > > > Silence = "Docs group gets to pick"
> > > >
> > > > On Thu, Sep 22, 2016 at 3:37 PM, John Blum  > > > wrote:
> > > >
> > > > > Truthfully, I don't think this is any different than API features
> > that
> > > > have
> > > > > been annotated with "@Experimental" (e.g. LucenceService
> > > > >  > > > > javadoc/com/gemstone/gemfire/cache/lucene/LuceneService.html>).
> > > > > I.e. nothing is going to stop a user from trying to use a
> > > > > feature/function/tool and searching for relevant information on how
> > to
> > > > use
> > > > > it if they know it exists, either explicitly or implicitly.
> > > > >
> > > > > In fact, I would think it is advantageous if they know it does
> exist,
> > > > even
> > > > > prior to an official release, so that feedback can be gathered.
> > > > >
> > > > > If it is not to be part of the "official" User Guide, perhaps a
> Wiki
> > > page
> > > > > (other than the specification
> > > > >  > > > > action?pageId=61309918&src=contextnavpagetreemode>
> > > > > [1])
> > > > > or better yet, a GitHub README page along with the source code if
> > users
> > > > are
> > > > > given access to build and use the tool themselves.
> > > > >
> > > > > If part of the "official" User Guide (under tools), then perhaps a
> > > > > "Experimental" label.
> > > > >
> > > > > Food for thought.
> > > > >
> > > > > -John
> > > > >
> > > > >
> > > > > [1]
> > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > > > > action?pageId=61309918&src=contextnavpagetreemode
> > > > >
> > > > >
> > > > > On Thu, Sep 22, 2016 at 3:19 PM, Anthony Baker  > > >
> > > > wrote:
> > > > >
> > > > > > I think that providing documentation for jvsd before it is
> included
> > > in
> > > > > the
> > > > > > source and binary release distributions will only confuse users.
> > +1
> > > > for
> > > > > > removing.
> > > > > >
> > > > > > Anthony
> > > > > >
> > > > > > > On Sep 22, 2016, at 2:39 PM, Dave Barnes  > > > wrote:
> > > > > > >
> > > > > > > JVSD has appeared in the Geode user manual since M2. See
> > > > > > > http://geode.docs.pivotal.io/docs/tools_modules/jvsd.html.
> > > > > > > Kirk, are you recommending that we remove this?
> > > > > > >
> > > > > > > On Thu, Sep 22, 2016 at 10:57 AM, Kirk Lund  > > >
> > > > wrote:
> > > > > > >
> > > > > > >> I would recommend not mentioning jVSD at all in the Geode 1.0
> > > docs.
> > > > > > Right
> > > > > > >> now it's just a Jira ticket and feature branch. I think the
> docs
> > > > > should
> > > > > > >> only cover what's in Geode 1.0.
> > > > > > >>
> > > > > > >> If there's some doc or wiki page about proposed future
> features
> > or
> > > > > > features
> > > > > > >> currently looking for contributors/developers, then that would
> > > > > probably
> > > > > > be
> > > > > > >> an appropriate place to mention jVSD.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Kirk
> > > > > > >>
> > > > > > >> On Thursday, September 22, 2016, Joey McAllister <
> > > > > > jmcallis...@pivotal.io >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Bumping this. Any thoughts?
> > > > > > >>>
> > > > > > >>> On Tue, Sep 20, 2016 at 10:50 AM Dave Barnes <
> > dbar...@pivotal.io
> > > 
> > > > > > >>> > wrote:
> > > > > > >>>
> > > > > > 

Re: M3 is out - New release manager for 1.0 ?

[Michael Stolz]

I think we should do these security features incrementally after 1.0.0.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Mon, Sep 19, 2016 at 6:16 PM, Anthony Baker  wrote:

>
> > On Sep 17, 2016, at 5:29 PM, Swapnil Bawaskar 
> wrote:
> >
> > So, my proposal for issues to be targeted for 1.0 is:
> > GEODE-136 Fix possible NullPointerException in Gfsh's 'list regions'
> > command's GetRegionsFunction
> > GEODE-1466 Branding: rename gemfire.properties file to geode.properties
> file
> > GEODE-17 Provide Integrated Security
> > GEODE-37 (package renaming)
> > GEODE-1791 (LICENSE update)
>
> GEODE-17 implies:
> GEODE-1569 (add post authorization processing in JMX and CLI
> commands)
> GEODE-1570 (secure developer REST API)
> GEODE-1571 (Client security should be able to use
> Resource:Operation permissions)
> GEODE-1648 (Provide ability to disable security for some
> components) **
> GEODE-1643 (The new SecurityManager need to authenticate the
> gateway sender/receiver as well)
> GEODE-1659 (Prevent misconfiguration of Integrated Security)
>
> Were you thinking all these security enhancements should be included in
> 1.0.0?  Seems like that could potentially be a lengthy process.  Does it
> make sense to release at these changes incrementally?
>
> The other items in scope for 1.0.0 look good to me.
>
> Anthony
>
>
> ** there was a recent discussion on this topic where the suggestion was to
> not allow per-component RBAC configuration: https://mail-archives.apache.
> org/mod_mbox/incubator-geode-dev/201609.mbox/%
> 3ccangouwssfmszbmlv9eq2kavgjtj_t2f1ad-qra8gzqmeq4t...@mail.gmail.com%3e
>
>
>

Re: securing geode components

[Michael Stolz]

Cool! I agree with this completely. Wasn't clear to me in the earlier
thread that we were keeping the thoughts separate.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Sep 9, 2016 at 1:06 PM, John Blum  wrote:

> I agree with Udo here.  Securing the channel between component/services has
> really very little to do with Authentication/Authorization, and by
> "Authentication", I mean in the user-centric sense, not the SSL "trusted"
> sense (with "trusted" keys and such).
>
> Whether a user can or cannot do something (in other words, is "authorized",
> or allowed to invoke an action, given an ACL) is a function of the action
> being performed and the privileges/rights that a user has been granted.
> That is independent of whether or not the channel has been secured.
>
> I also agree with Kirk that HTTP should perhaps be renamed to WEB.
>
> Finally, while SSL is usually a cause to reboot the system, Auth has no
> such restrictions.  I could easily change Auth credentials of a user (e.g.
> revoke privileges) at runtime.
>
> My $0.02,
> -John
>
>
> On Fri, Sep 9, 2016 at 10:00 AM, Michael Stolz  wrote:
>
> > In a pure world I would go for all or nothing, but I always worry about
> the
> > upgrade path. If I have to redeploy and restart EVERYTHING around the
> whole
> > world simultaneously, it's a non-starter.
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
> > On Fri, Sep 9, 2016 at 12:51 PM, Anthony Baker 
> wrote:
> >
> > > Mike, I was suggesting ON | OFF only for RBAC security, not for SSL
> > > configuration.  Any thoughts on that?
> > >
> > > Anthony
> > >
> > > > On Sep 9, 2016, at 9:44 AM, Michael Stolz  wrote:
> > > >
> > > > I think a reason that we might need to be less than all-or-nothing is
> > for
> > > > at least these two situations:
> > > >
> > > > 1. a user who started out with SSL disabled, and now wants to enable
> > it,
> > > > but can't take a full global outage, so needs to get it enabled for
> the
> > > WAN
> > > > first, and then for server-to-server and then for client/server.
> > > >
> > > > 2. SSL enabled over the WAN because that is not a trusted network,
> but
> > > they
> > > > can live without SSL for the server/server and client/server
> > connections
> > > > because they ARE in a trusted network and they don't need to pay the
> > > > overhead for SSL on those links.
> > > >
> > > > There are probably other scenarios as well, but these are the two
> that
> > > come
> > > > to mind quickly.
> > > >
> > > >
> > > >
> > > > --
> > > > Mike Stolz
> > > > Principal Engineer, GemFire Product Manager
> > > > Mobile: 631-835-4771
> > > >
> > > > On Fri, Sep 9, 2016 at 12:05 PM, Jinmei Liao 
> > wrote:
> > > >
> > > >> That is my original thought as well. If we are protecting resources
> > > >> (CLUSTER and DATA), it should be protected no matter which way user
> is
> > > >> trying to access it. I guess I'll leave this to the PMs to decide.
> > > >>
> > > >> On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker 
> > > wrote:
> > > >>
> > > >>> Udo, Kirk - this makes sense and thanks for the discussion to help
> > > >> clarify
> > > >>> the issue.
> > > >>>
> > > >>> Regarding GEODE-1648, does it make sense to do at all?  That is,
> what
> > > if
> > > >>> role-based access control is either ON | OFF instead of per
> > component.
> > > >> If
> > > >>> we allow disabling RBAC for certain components, wouldn’t that make
> it
> > > >>> possible to create a backdoor?  Could we start with a binary option
> > and
> > > >>> enable more granular control when requested by users?
> > > >>>
> > > >>> Anthony
> > > >>>
> > > >>>> On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
> > > >>>>
> > > >>>> +1 overall with some feedback...
> > > >>>>
> > > >>>> 1) I think the list is reasonable with a few nitpicks below
> > > >>>>
>

Re: securing geode components

[Michael Stolz]

In a pure world I would go for all or nothing, but I always worry about the
upgrade path. If I have to redeploy and restart EVERYTHING around the whole
world simultaneously, it's a non-starter.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Sep 9, 2016 at 12:51 PM, Anthony Baker  wrote:

> Mike, I was suggesting ON | OFF only for RBAC security, not for SSL
> configuration.  Any thoughts on that?
>
> Anthony
>
> > On Sep 9, 2016, at 9:44 AM, Michael Stolz  wrote:
> >
> > I think a reason that we might need to be less than all-or-nothing is for
> > at least these two situations:
> >
> > 1. a user who started out with SSL disabled, and now wants to enable it,
> > but can't take a full global outage, so needs to get it enabled for the
> WAN
> > first, and then for server-to-server and then for client/server.
> >
> > 2. SSL enabled over the WAN because that is not a trusted network, but
> they
> > can live without SSL for the server/server and client/server connections
> > because they ARE in a trusted network and they don't need to pay the
> > overhead for SSL on those links.
> >
> > There are probably other scenarios as well, but these are the two that
> come
> > to mind quickly.
> >
> >
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
> > On Fri, Sep 9, 2016 at 12:05 PM, Jinmei Liao  wrote:
> >
> >> That is my original thought as well. If we are protecting resources
> >> (CLUSTER and DATA), it should be protected no matter which way user is
> >> trying to access it. I guess I'll leave this to the PMs to decide.
> >>
> >> On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker 
> wrote:
> >>
> >>> Udo, Kirk - this makes sense and thanks for the discussion to help
> >> clarify
> >>> the issue.
> >>>
> >>> Regarding GEODE-1648, does it make sense to do at all?  That is, what
> if
> >>> role-based access control is either ON | OFF instead of per component.
> >> If
> >>> we allow disabling RBAC for certain components, wouldn’t that make it
> >>> possible to create a backdoor?  Could we start with a binary option and
> >>> enable more granular control when requested by users?
> >>>
> >>> Anthony
> >>>
> >>>> On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
> >>>>
> >>>> +1 overall with some feedback...
> >>>>
> >>>> 1) I think the list is reasonable with a few nitpicks below
> >>>>
> >>>> 2) If these are Channels and not Components, then I would probably
> name
> >>> it
> >>>> SecurableChannels or SecurableCommunicationChannels or whatever.
> >>>>
> >>>> 3) I'd prefer HTTP be renamed to Web or other non-protocol word --
> HTTP
> >>> is
> >>>> a protocol and the names of the other channels are conceptual channels
> >>>> rather than a protocol (the others use TCP/IP or RMI but we don't
> label
> >>>> them as such). Or am I missing something here?
> >>>>
> >>>> 4) JMX is probably ok. Currently we are using (and securing) JMX over
> >> RMI
> >>>> (javax.management.remote.rmi.RMIConnectorServer). There are other
> >>>> connectors for JMX including HTTP (ex: mx4j.tools.adaptor.http.
> >>> HttpAdaptor)
> >>>> and SNMP (ex: com.sun.jmx.snmp.daemon.SnmpAdaptorServer). We only
> need
> >>> JMX
> >>>> over RMI for now, but would we add those others as new enums to
> >>>> SecurableChannels later if we add anything like that to Geode? Or
> would
> >>> we
> >>>> try to group those all together under the name JMX? Or decide when the
> >>> time
> >>>> comes?
> >>>>
> >>>> I think we should try to steer away from being overly controlled by
> >> specs
> >>>> especially for reasonable changes. We all follow agile process, so a
> >>>> decision made one iteration could easily be undone or changed in the
> >> next
> >>>> iteration and most of us are following weekly iterations.
> >>>>
> >>>> After a release anything exposed in a User API is very difficult to
> >>> change
> >>>> due to backwards compatibility constraints. I think we should be much
> >>> 

Re: securing geode components

[Michael Stolz]

I think a reason that we might need to be less than all-or-nothing is for
at least these two situations:

1. a user who started out with SSL disabled, and now wants to enable it,
but can't take a full global outage, so needs to get it enabled for the WAN
first, and then for server-to-server and then for client/server.

2. SSL enabled over the WAN because that is not a trusted network, but they
can live without SSL for the server/server and client/server connections
because they ARE in a trusted network and they don't need to pay the
overhead for SSL on those links.

There are probably other scenarios as well, but these are the two that come
to mind quickly.



--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Sep 9, 2016 at 12:05 PM, Jinmei Liao  wrote:

> That is my original thought as well. If we are protecting resources
> (CLUSTER and DATA), it should be protected no matter which way user is
> trying to access it. I guess I'll leave this to the PMs to decide.
>
> On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker  wrote:
>
> > Udo, Kirk - this makes sense and thanks for the discussion to help
> clarify
> > the issue.
> >
> > Regarding GEODE-1648, does it make sense to do at all?  That is, what if
> > role-based access control is either ON | OFF instead of per component.
> If
> > we allow disabling RBAC for certain components, wouldn’t that make it
> > possible to create a backdoor?  Could we start with a binary option and
> > enable more granular control when requested by users?
> >
> > Anthony
> >
> > > On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
> > >
> > > +1 overall with some feedback...
> > >
> > > 1) I think the list is reasonable with a few nitpicks below
> > >
> > > 2) If these are Channels and not Components, then I would probably name
> > it
> > > SecurableChannels or SecurableCommunicationChannels or whatever.
> > >
> > > 3) I'd prefer HTTP be renamed to Web or other non-protocol word -- HTTP
> > is
> > > a protocol and the names of the other channels are conceptual channels
> > > rather than a protocol (the others use TCP/IP or RMI but we don't label
> > > them as such). Or am I missing something here?
> > >
> > > 4) JMX is probably ok. Currently we are using (and securing) JMX over
> RMI
> > > (javax.management.remote.rmi.RMIConnectorServer). There are other
> > > connectors for JMX including HTTP (ex: mx4j.tools.adaptor.http.
> > HttpAdaptor)
> > > and SNMP (ex: com.sun.jmx.snmp.daemon.SnmpAdaptorServer). We only need
> > JMX
> > > over RMI for now, but would we add those others as new enums to
> > > SecurableChannels later if we add anything like that to Geode? Or would
> > we
> > > try to group those all together under the name JMX? Or decide when the
> > time
> > > comes?
> > >
> > > I think we should try to steer away from being overly controlled by
> specs
> > > especially for reasonable changes. We all follow agile process, so a
> > > decision made one iteration could easily be undone or changed in the
> next
> > > iteration and most of us are following weekly iterations.
> > >
> > > After a release anything exposed in a User API is very difficult to
> > change
> > > due to backwards compatibility constraints. I think we should be much
> > more
> > > careful with User APIs in Geode going forward to avoid some of the
> > problems
> > > we have with pre-existing Geode User APIs that we inherited.
> > >
> > > -Kirk
> > >
> > > On Thu, Sep 8, 2016 at 2:07 PM, Udo Kohlmeyer 
> > wrote:
> > >
> > >> As GEODE-420 deals with SSL comms configuration and GEODE-1648 with
> > >> Authentication&Authorization I think we need to be careful in what is
> > >> feasible and what is logical.
> > >>
> > >> For SSL comms it was decided that the following components are
> relevant
> > >> [1]  > >> SL+properties>:
> > >>
> > >> * Locator => The comms channel between Locators + the initial comms
> > >>   channel between clients and locator
> > >> * Cluster => Internode comms channel (peer to peer)
> > >> * Server => Client-server comms channel
> > >> * Gateway => Comms channel between WAN Gateway senders/receivers
> > >> * HTTP => Any HTTP comms. incl REST and Pulse
> > >> * JMX => Any JMX comms
> > >>
> > >> These components were selected as they seem to be logical boundaries
> and
> > >> communication interfaces.
> > >>
> > >> I think the specialization of HTTP, for Authentication&Authorization
> are
> > >> functions of those interfaces:
> > >>
> > >> * REST-admin
> > >> * REST-dev
> > >> * Pulse
> > >>
> > >> I think that comms and functions exposed by those comms should not be
> > >> mixed. I think that securing the comms channel is a factor of "trust".
> > Do I
> > >> implicitly trust the interface/system that I am connected to or are
> > >> connecting to.
> > >>
> > >> I think concepts like "management" is a concept in function. Do I
> allow
> > a
> > >> user to access admin API's? The function of managemen

Re: M3 is done, what's next?

[Michael Stolz]

+1 for renaming packages now.

We might consider having a look at the examples to make sure we have
covered off-heap and integrated security.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Sep 2, 2016 9:47 AM, "Dan Smith"  wrote:

> +1 For renaming the packages. It would be really nice to graduate ASAP! Is
> there anything else from a code perspective that we need to do before
> graduation? If so we should also get that in 1.0.
>
> It would be nice to get a few more examples in the codebase for 1.0. We
> should probably just generally review the documentation we're shipping with
> 1.0. Actually, it would be nice if the docs hosted on
> http://geode.docs.pivotal.io/ could get incorporated as well (I think
> pivotal is still planning on donating those docs?), but I don't think we
> should hold up 1.0 or graduation based on that.
>
> We should probably review our dependencies and update anything that's out
> of date for 1.0.
>
> We should also coordinate the package renaming with Spring Data Geode.
>
> -Dan
>
>
> On Fri, Sep 2, 2016 at 9:08 AM, Greg Chase  wrote:
>
> > On Fri, Sep 2, 2016 at 8:46 AM, Anthony Baker  wrote:
> >
> > > with one exception:  we need to rename our source packages to
> > > ‘org.apache.geode’ [3].
> > >
> > > I think we should move forward with package renaming *now* and include
> > > that in the scope for the 1.0.0-incubating release.
> > >
> > > As previously discussed [4] we’d like to preserve protocol
> compatibility
> > > for existing users of client/server and WAN.  This should only affect a
> > > handful of classes that would remain in the ‘com.gemstone.gemfire’
> > > namespace (we should identify those soon).
> > >
> >
> > Agreed.  Now is the time.  Later is always worse then now when it occurs.
> >
>

Re: Experimental features for 1.0

[Michael Stolz]

Yes we should definitely do this

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Aug 10, 2016 9:01 PM, "William Markito"  wrote:

> As we move forward to 1.0 I'd like to propose creating JIRAS with the
> "experimental" label to capture everything we have that is classified as
> such.
>
> This would help users and projects consume Geode to consider those
> features properly and decide to add support (or not) to them as well.
>
> AFAIK this is the current list:
>
> Redis Adapter
> Spark Connector
> OQL Aggregates
> Lucene integration
> Auto Rebalancer
>
> Is there anything else ? Are any of these not considered experimental
> anymore ?
>
>  If by the time we get to 1.0 they're complete we'd remove be label
> obviously.
>
> Thoughts ?
>
>
>
>
>

Re: [VOTE] Release Apache Geode (incubating) 1.0.0-incubating.M3 - RC7

[Michael Stolz]

+1

All indications are this attempt is correct

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Aug 8, 2016 6:09 PM, "William Markito"  wrote:

> Guys, I'll leave the voting open until tomorrow, but please vote so we can
> close and release M3.
>
> Thanks,
>
> On Fri, Aug 5, 2016 at 4:24 PM, Anthony Baker  wrote:
>
> > +1
> >
> > * Verified she’s
> > * Verified signatures
> > * Verified tag signature
> > * Build from tag
> > * Build and run from src distro
> > * Checked src distro for binaries
> > * Run from binary distro
> > * Run some examples from mvn repo
> >
> > Anthony
> >
> > > On Aug 5, 2016, at 10:34 AM, Jason Huynh  wrote:
> > >
> > > +1
> > >
> > > - built from source distribution
> > > - started locator, server, listed members and created regions in gfsh
> > from
> > > binary
> > > - started locator and server from source built gfsh
> > >
> > >
> > > On Fri, Aug 5, 2016 at 9:58 AM Dan Smith  wrote:
> > >
> > >> +1
> > >>
> > >> Verified
> > >> * Successful precheckin run of this release -
> > >> https://builds.apache.org/job/
> > >> Geode-release/24/ 
> > >> * Signatures
> > >> * Basic gfsh commands with binary dist
> > >> * Built from source dist
> > >> * Basic CRUD test with maven artifacts
> > >> * No jars in source dist
> > >>
> > >> -Dan
> > >>
> > >> On Thu, Aug 4, 2016 at 6:02 PM, William Markito 
> > >> wrote:
> > >>
> > >>> All,
> > >>>
> > >>> This is the seventh release candidate Apache Geode, version
> > >>> 1.0.0-incubating.M3.
> > >>>
> > >>> We're including the feedback received in RC6 including a fix
> > >>> (83f97ceef52febf92ef7737726548aa0865c1a59) to run REST API tests.
> > >>>
> > >>> Thanks to all the community members to drive towards this milestone!
> > >>>
> > >>> It fixes the following issues:
> > >>>   https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > >>> projectId=12318420&version=12335358
> > >>>
> > >>> *** Please download, test and vote by Monday, August 8, 0800 hrs US
> > >>> Pacific.
> > >>>
> > >>> Note that we are voting upon the source (tag):
> > >>>   rel/v1.0.0-incubating.M3.RC7
> > >>>
> > >>> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.gi
> > >>> t;a=tag;h=refs/tags/rel/v1.0.0-incubating.M3.RC7
> > >>>  > >>> git;a=tag;h=refs/tags/rel/v1.0.0-incubating.M3.RC7>
> > >>>
> > >>> Commit ID: 83f97ceef52febf92ef7737726548aa0865c1a59
> > >>>
> > >>> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.gi
> > >>> t;a=commit;h=83f97ceef52febf92ef7737726548aa0865c1a59
> > >>>
> > >>> Source and binary files:
> > >>> https://dist.apache.org/repos/dist/dev/incubator/geode/1.0.0
> > >>> -incubating.M3.RC7
> > >>>
> > >>> For this release the documentation on how to install and use Apache
> > Geode
> > >>> are hosted
> > >>> on pivotal.io:
> > >>>   http://geode.docs.pivotal.io
> > >>>
> > >>> Maven staging repo:
> > >>> https://repository.apache.org/content/repositories/
> orgapachegeode-1011
> > >>>
> > >>> Geode's KEYS file containing PGP keys we use to sign the release:
> > >>>   https://github.com/apache/incubator-geode/blob/release/
> > >>> 1.0.0-incubating.M3/KEYS
> > >>>
> > >>> Release Key: pub  4096R/7AAED8BB 2016-07-13
> > >>> Fingerprint: 8E06 B711 DB13 3AE7 0CC1  ABDE 6A14 F0BC 7AAE D8BB
> > >>>
> > >>> Thanks,
> > >>>
> > >>> --
> > >>> ~/William
> > >>>
> > >>
> >
> >
>
>
> --
>
> ~/William
>

Re: use-cluster-configuration/cache.xml/reconnect

[Michael Stolz]

There are still some things that can't be configured with cluster
configuration service so the combination of
cluster-configuration-service=true and cache.xml will have to be supported
until such time as CCS is completed.



--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Mon, Jul 25, 2016 at 11:55 AM, Barry Oglesby  wrote:

> I'm not seeing this behavior in my tests. I have a cache.xml and
> use-cluster-configuration=true. When the member recovers, I see the xml
> being recreated.
>
> The GMSMembershipManager.saveCacheXmlForReconnect method is being invoked
> but short-circuiting because sharedConfigEnabled=true.
>
> But, the GemFireCacheImpl.initializeDeclarativeCache method is invoked and
> that recreates the cache.
>
> I see this logging from the ReconnectThread:
>
> [info 2016/07/25 11:33:08.831 PDT server  tid=0x3e]
> Received cluster configuration from the locator
>
> [info 2016/07/25 11:33:08.831 PDT server  tid=0x3e]
> Received an empty shared configuration
>
> [info 2016/07/25 11:33:08.831 PDT server  tid=0x3e]
> Initializing cache using "file:/.../config/gemfire-server.xml":
>   
>"-//GemStone Systems, Inc.//GemFire Declarative Caching 7.0//EN"
> "http://www.gemstone.com/dtd/cache7_0.dtd";>
>
>   
> 
> 
> 
>   
> MembershipCycleHealthFunction
>   
> 
>   
>
> What exact configuration is being used that is failing?
>
> Thanks,
> Barry Oglesby
>
>
> On Mon, Jul 25, 2016 at 10:44 AM, Kirk Lund  wrote:
>
> > Below came to me via support engineer, and I'm trying to determine what
> the
> > correct behavior should be:
> >
> > "Let's assume that you're configuring your entire cache through spring or
> > cache.xml without using the cluster configuration service, and that
> you're
> > using the default settings when starting the cluster (meaning
> > "-Dgemfire.use-cluster-configuration=true"). Under these circumstances,
> if
> > a member is kicked out of the distributed system, it'll be useless after
> > the auto reconnect feature kicks in because it detects that the cluster
> > configuration is enabled and tries to recover the configuration from it,
> > even when it's actually empty because it's not being used at all.
> >
> > According to the user guide we actually support this kind of setup
> (cluster
> > configuration plus member group configuration plus individual member
> > configuration), but the actual source code used when reconnecting doesn't
> > support it at all."
> >
> > Apparently use-cluster-configuration=true, but the user actually used
> > cache.xml to configure the cache server.
> >
> > 1) I thought we were going to disallow mixing cluster config with old
> > config in this way.
> >
> > 2) I would expect that starting a server with a cache.xml AND
> > use-cluster-configuration=true
> > should fail fast and not be allowed to start.
> >
> > 3) I would also expect that if a locator is configured with
> > use-cluster-configuration=true
> > then it should reject any cache servers that attempt to join the cluster
> > that have use-cluster-configuration=false.
> >
> > I still need to find the relevant section in the user guide, but I would
> > propose changing this to NOT allow cluster config to be used with old
> > config. This is untested and be unsupported. The documentation should NOT
> > say this is ok to do.
> >
> > Thoughts?
> >
> > -Kirk
> >
>

Re: Proposal - Fail cache creation on persistent PR colocation misconfiguration

[Michael Stolz]

What about the case where the parent region is created via cache.xml and
the child regions are created dynamically? I believe that could be a valid
case. The right thing to do is recover what you can, when you can. Not to
make parent recovery dependant on its children.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jul 25, 2016 1:06 PM, "Kenneth Howe"  wrote:

> I’d like to propose a functional change to cache creation when a cache
> server is created via a cache.xml file. This proposal originated from work
> on GEODE-1128  dealing
> with missing colocated regions. The change is to fail cache creation if
> there are missing colocated regions in the cache.xml that will prevent
> persistent PR recovery.
>
> Discussion:
> When persistent PRs are colocated, the parent region is created first, but
> persistent data recovery isn’t done until all the colocated regions have
> been created. Currently, if a child region is not created, the cache
> creation will succeed but persistent data is not recovered. This is the
> condition reported in the Jira ticket
>
> When caches and regions are created via the APIs, or interactively with
> gfsh, the cache is created, then the parent region(s), then the child
> region(s). There will always be an unknown delay between each of these
> steps. The parent region creation succeeds, but internally Geode does not
> know when (or if) the child regions will be created. Normally the child
> regions are created after a short period and recovery proceeds, so the
> parent region having unrecovered data is a transitory state. If the child
> region is not created, the the parent region data will not be recovered. In
> this case a warning can be logged if the missing child regions aren’t
> created within a reasonable time.
>
> However, when the cache creation is done via a cache.xml file, regions are
> created as part of the cache creation. In this case it’s known fairly
> quickly that there’s a misconfiguration that will prevent persistent PR
> recovery. The cache creation can be failed immediately alerting the user to
> the misconfiguration.
>
> Ken
>
>

Re: Recovering large data for persisted regions

[Michael Stolz]

You are kinda pushing the IMDGs into a place they are not necessarily
designed for. If you want a disk based database then why are you looking at
IMDGs? The overflow capability is designed to be an escape valve in case
you get a spike in demand, not a steady state condition. I would feel more
comfortable if you used the product as it was designed for.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jul 15, 2016 5:01 PM, "Nilkanth Patel"  wrote:

> Hello,
>
> Facing issue in recovering data for persisted regions when large amount
> (more than heap) of data is persisted.
>
> brief about scenario .
>
> Creating 10 regions, lets call it R1, R2, R3, ... R10 with following
> config.
> For R1, R2, Total # of buckets = 113.
> For R3, R4, R10, #of buckets = 511.
>
> All above regions are configured with Disk persistance enabled (ASYNCH) and
> eviction action overflow to disk. like,
>
> RegionFactory<> rf =
> cache.createRegionFactory(RegionShortcut.PARTITION_PERSISTENT_OVERFLOW);
> rf.setDiskSynchronous(false) //for asynch writes.
> rf.setDiskStoreName("myDiskStore");PartitionAttributesFactory paf =
> new
> PartitionAttributesFactory().setRedundantCopies(3).paf.setTotalNumBuckets(511);
>
>
> For each server, Setting both --initial-heap and --max-heap to same, i.e
> 16gb with --eviction-heap-percentage=81 --critical-heap-percentage=90
>
> I keep the system running (puts, gets, delete) for hours to add data over
> time until i have overflowed tons of data approaching the heap size or
> more.
> Now i shutdown my cluster and then attempt to restart but it does not come
> up. It seems during this early phase of recovery (large amount of data),
> geode surpasses the critical threshold which kills it before successful
> startup.
>
> Is this observation correct and is this a known limitation...? If so any
> work around for this..?
>
> Also, Considering the above case, recovery for (1)
> ForceDisconnect--->Autoconnect case and (2) normal_shutdown-->restart case
> is a same mechanism or is there any differences?
>
> Thanks in advance,.
>
> Nilkanth.
>

Re: [VOTE] Release Apache Geode (incubating) 1.0.0-incubating.M3

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Wed, Jul 13, 2016 at 10:39 AM, William Markito 
wrote:

> All,
>
> This is the first release candidate Apache Geode, version
> 1.0.0-incubating.M3.
> Thanks to all the community members to drive towards this milestone!
>
> It fixes the following issues:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12335358
>
> *** Please download, test and vote by Monday, July 18, 0800 hrs US Pacific.
>
> Note that we are voting upon the source (tag):
>rel/v1.0.0-incubating.M3.RC1
>
>
> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;a=commit;h=05e60b977799bcb5c835cc35f9af5325f2c5f600
>
> Commit ID: 05e60b977799bcb5c835cc35f9af5325f2c5f600
>
>
> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;a=commit;h=05e60b977799bcb5c835cc35f9af5325f2c5f600
>
> Source and binary files:
>
>
> https://dist.apache.org/repos/dist/dev/incubator/geode/1.0.0-incubating.M3.RC1/
>
> For this release the documentation on how to install and use Apache Geode
> are hosted
> on pivotal.io:
>http://geode.docs.pivotal.io
>
> Maven staging repo:
>https://repository.apache.org/content/repositories/orgapachegeode-1005/
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>
>
> https://github.com/apache/incubator-geode/blob/release/1.0.0-incubating.M3/KEYS
>
> Release Key: pub  4096R/7AAED8BB 2016-07-13
> Fingerprint: 8E06 B711 DB13 3AE7 0CC1  ABDE 6A14 F0BC 7AAE D8BB
>
>
> Thanks,
>
> --
>
> ~/William
>

Re: Valid characters in region names?

[Michael Stolz]

Yes we did discuss adding hyphen in Geode, and we discussed actually
checking during create.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Jun 30, 2016 at 3:30 PM, Kevin Duling  wrote:

> I found a thread from March of this year where this was discussed and '-'
> is also included in that list.
>
> Shouldn't the creation of a region with an invalid character raise an
> exception?  I'm finding that I can create and list regions with a variety
> of non-alphanumeric characters.  But I can only destroy a subset of those.
>
>  It appears to be because some are created with quotes around the region
> name.
>
> E.g.,
>
> service=Region, name=/good, type=Member
> service=Region, name="/not*good", type=Member
>
> The name lookup fails in the Destroy call.  The rule that applied the
> quotes for /not*good during Create isn't applied in Destroy
>
> On Thu, Jun 30, 2016 at 12:06 PM, Michael Stolz  wrote:
>
> > The only characters that should be used in Region names are
> > ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890_
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
> > On Thu, Jun 30, 2016 at 2:03 PM, Kevin Duling 
> wrote:
> >
> > > I'm working on a bug where it is not possible to delete a region that
> > has a
> > > hyphen in it.  E.g., the region was created with:
> > >
> > > create region --name=not-good --type=REPLICATE
> > >
> > > I've a solution done, but see this as an opportunity to add a test for
> > > other special characters.  Is there a list of valid and invalid
> > characters
> > > for a region name?
> > >
> >
>

Re: Valid characters in region names?

[Michael Stolz]

The only characters that should be used in Region names are
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890_

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Jun 30, 2016 at 2:03 PM, Kevin Duling  wrote:

> I'm working on a bug where it is not possible to delete a region that has a
> hyphen in it.  E.g., the region was created with:
>
> create region --name=not-good --type=REPLICATE
>
> I've a solution done, but see this as an opportunity to add a test for
> other special characters.  Is there a list of valid and invalid characters
> for a region name?
>

Re: Podling Report Reminder - July 2016

[Michael Stolz]

+1 Looks good

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jun 28, 2016 7:48 PM, "Nabarun Nag"  wrote:

> Hi
>
> Please find below the draft of the podling report.
>
> Any feedback is welcome.
>
> Regards
> Naba
>
> 
>
> Geode
>
> Geode is a data management platform that provides real-time, consistent
> access to data-intensive applications throughout widely distributed cloud
> architectures.
>
> Geode has been incubating since 2015-04-27.
>
> Three most important issues to address in the move towards graduation:
>
> - Rename packages to org.apache
> - Expanding the community to include contributors and committers outside of
> Pivotal.
> - Establish a release cadence.
>
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
>
> None
>
> How has the community developed since the last report?
>
> - We continue to work on the “Three most important issues to move towards
> graduation” from our previous report.
> - We had our second public release (1.0.0-incubating.M2).
> - We added 1 new committer.
> - We are about to have our third release (1.0.0-incubating.M3).
> - The Geode Clubhouse hosted a meeting covering the following topics :
> - “Cluster Replication via WAN” by Mike Stolz
> - “Southwest Airlines' Experience with Geode & GemFire at Scale” by Brian
> Dunlap
>
> - The breakdown of JIRA tickets is the following :
> - Q3/2015 306 created, 138 resolved
> - Q4/2015 342 created, 225 resolved
> - Q1/2016 438 created, 359 resolved
> - Q2/2016 451 created, 322 resolved [As of June 28th, 2016]
>
> - There was a total of 169 pull requests on Github with 6 still open.
>
> - The breakdown of the mailing lists messages for April, May, June of
> 2016:
> - org.apache.geode.issues 3,992
> - org.apache.geode.commits 3,572
> - org.apache.geode.dev2,006
> - org.apache.geode.user  273
>
> - There are now 155 subscribers on the dev and 165 on the user mailing list
>
> How has the project developed since the last report?
>
> - Had our second release, 1.0.0-incubating.M2, on April 22, 2016.
> - Donation of native client components which included a C++ client and a
> .NET client.
> - Inclusion of WAN and CQ components in the second release.
> - Rotated release manager responsibilities for second and soon to be third
> release.
> - Discussions about donation of documentation.
> - Geode portfile was added to MacPorts.
> - joptsimple source replaced with binary dependency.
> - Active discussions on topics like integrated security for Geode and
> wording of the website.
>
> Community events:
> - June 2016
>  - Midwest In Memory Data Grid Meet Up on using Apache Geode for real
> time transaction.
>(
> http://www.meetup.com/Chicago-In-Memory-Data-Grid-Meetup/events/230810905/
> )
>
>  - Apache Geode meetup in Tokyo
>(http://pivotal-japan.connpass.com/event/33297/)
>
> Date of last release:
>
>   2016-04-22
>
> When were the last committers or PMC members elected?
>
>  Nabarun Nag (2016-06-14)
>
> Signed-off-by:
>
>   [ ](geode) Konstantin Boudnik
>   [ ](geode) Chip Childers
>   [ ](geode) Justin Erenkrantz
>   [ ](geode) Jan Iversen
>   [ ](geode) Chris Mattmann
>   [ ](geode) William A. Rowe Jr.
>   [ ](geode) Roman Shaposhnik
>
>
> On Tue, Jun 28, 2016 at 10:41 AM Nabarun Nag  wrote:
>
> > Jason and I are giving it a shot. I will send the first draft soon.
> >
> > Regards
> > Naba
> >
> > On Tue, Jun 28, 2016 at 9:58 AM Dan Smith  wrote:
> >
> >> I can take a crack at this if no one else wants to do it. I'll need some
> >> help from people have done the previous reports.
> >>
> >> -Dan
> >>
> >> On Sun, Jun 26, 2016 at 2:02 PM,  wrote:
> >>
> >> > Dear podling,
> >> >
> >> > This email was sent by an automated system on behalf of the Apache
> >> > Incubator PMC. It is an initial reminder to give you plenty of time to
> >> > prepare your quarterly board report.
> >> >
> >> > The board meeting is scheduled for Wed, 20 July 2016, 10:30 am PDT.
> >> > The report for your podling will form a part of the Incubator PMC
> >> > report. The Incubator PMC requires your report to be submitted 2 weeks
> >> > before the board meeting, to allow sufficient time for review and
> >> > submission (Wed, July 06).
> >> >
> >> > Please submit your report with sufficient time to allow the Incubator
> >> > PMC, and subsequently board members to review and digest. Again, the
> >> > very latest you should submit your report is 2 weeks prior to the
> board
> >> > meeting.
> >> >
> >> > Thanks,
> >> >
> >> > The Apache Incubator PMC
> >> >
> >> > Submitting your Report
> >> >
> >> > --
> >> >
> >> > Your report should contain the following:
> >> >
> >> > *   Your project name
> >> > *   A brief description of your project, which assumes no knowledge of
> >> > the project or necessarily of its field
> >> > *   A list of the three most important issues to address in the move
> >> > towards graduation.
> >> > *

Re: Snappy compressor (native->java)

[Michael Stolz]

+1 pure java

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jun 20, 2016 4:58 PM, "Jacob Barrett"  wrote:

> +1 for pure java default and making the native a drop in option.
>
> I would suggest looking into embedding the native bits into a JAR file.
> There are some tricks you can do to write the native bits out to disk from
> the JAR file to load them at runtime. This would make it easier for someone
> to deploy the speedup by just dropping in a single JAR into the class path.
>
> -Jake
>
>
> On Mon, Jun 20, 2016 at 4:14 PM Sai Boorlagadda <
> sai_boorlaga...@apache.org>
> wrote:
>
> > Geode currently bundles xerial/snappy
> >  as a default implementation. And
> > this is a "JNI wrapper" on google snappy <
> http://google.github.io/snappy/>
> > implementation.
> >
> > "xerial/snappy" jar bundles several pre-compiled static libraries to
> > support various OS (linux, windows, SunOS) and architectures (x86, Sparc
> > etc). The current dependency (1.1.1.6) does not support SunOS (Sparc), so
> > the plan is to upgrade to a more recent version.
> >
> > While upgrading to a more recent version, I found a pure java port
> >  of original C++ implementation and
> wanted
> > to swap with use pure java implementation to avoid any platform specific
> > dependency on Geode.
> >
> > From the creator - "*the pure Java port is 20-30% faster for block
> > compress, 0-10% slower for block uncompress, and 0-5% slower for
> round-trip
> > block compression.*".
> >
> > Though native version is better on uncompress (more number of gets, puts
> > depending on use cases), I would still vote for distributing with a pure
> > java version as a "default" implementation as Geode already exposes an
> > interface to allow any one to provide any custom implementation.
> >
> > In case if there are any differences between these two implementations,
> > swapping with a pure java version should not impact any existing users,
> as
> > Geode does not save compressed data to disk or on to the wire.
> >
> > Let me know if any one has different thoughts?
> >
> > Sai
> >
>

Re: update website for WAN, CQ and native client

[Michael Stolz]

I like multi-site. It means physical isolation.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jun 17, 2016 5:02 PM, "Gregory Chase"  wrote:

> On Fri, Jun 17, 2016 at 11:56 AM, Gregory Chase  wrote:
>
> >
> >
> > Boom!  "Multi-site' or "Multi-Cluster"
> >
> >
> >
> At the risk of sounding like I'm "+1"-ing myself, consider this bullet
> comes on the next row after "clustering".  So if it says "multi-cluster" or
> "multi-clustering" with that cool cloud to cloud icon, I think it gets the
> message across that John so well described.
>
> -Greg
>
> --
> Greg Chase
>
> Global Head, Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>

Re: post processing for integrated security

[Michael Stolz]

But the new model is supposed to be configuration only no user code, so I
don't think it applies

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jun 17, 2016 4:48 PM, "Jinmei Liao"  wrote:

> If we are only doing post-processing for get and query, then it's not too
> much work adding it into the new model.
>
> On Fri, Jun 17, 2016 at 12:00 PM, Michael Stolz  wrote:
>
> > Post processing is there to filter the data returned by a get or a query.
> > But it requires that they write code, so it is probably only pertinent
> for
> > the old security model not for the new fully configurable End-to-end
> model
> > that we have begun discussing.
> >
> > So MAYBE we're actually doing too much work in keeping it for the new
> > model.
> >
> > Thoughts?
> >
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
> > On Fri, Jun 17, 2016 at 11:44 AM, Jinmei Liao  wrote:
> >
> > > While working on the integrated security for gemfire9/geode1, we are
> > > frequently baffled by this post-processing functionality that's been
> > lumped
> > > into security. Here are a few observations we've come up with and want
> to
> > > run with the dev community for comments:
> > >
> > > 1. Post processing is not authorization. At this point, all necessary
> > > authorization should already be done and satisfied.
> > >
> > > 2. Post processing is related to security in the sense that it needs a
> > > Principal in the context, but it's not part of the authorization.
> > >
> > > 3. Previous implementation (client-server) adds post process after
> every
> > > authorization checks. We want to make a clean separation of these two
> > > concepts. New interfaces of post processing will be introduced (so that
> > we
> > > can preserve the old behavior if user is implementing the old
> interface),
> > > and will ONLY be added to the "get" and "query" data operations.
> > >
> > > 4. The new interface will look like below:
> > > public Object processValue(Principal principal, String regionPath,
> Object
> > > key, Object value)
> > >
> > > Is this a sufficient interface to begin with? Do you have any client
> that
> > > would need post processor to do something completely different?
> > >
> > > Thank you for all your feedback.
> > >
> > > --
> > > Cheers
> > >
> > > Jinmei
> > >
> >
>
>
>
> --
> Cheers
>
> Jinmei
>

Re: com.gemstone.gemfire.ForcedDisconnectException: Member isn't responding to heartbeat requests

[Michael Stolz]

Overflow in Geode is really intended to be used as a way of dealing with a
TEMPORARY situation where there is too much data to fit in memory...NOT as
a steady state design pattern.

Making Geode disk based is an anti-pattern in my opinion.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Jun 17, 2016 at 1:04 PM, Mark Bretl  wrote:

> + user@geode.i.a.o
>
> On Fri, Jun 17, 2016 at 8:49 AM, Anthony Baker  wrote:
>
>> I would start by turning on GC logging and see if you have any long
>> collections.  A stop-the-world collection that takes longer than the member
>> timeout could cause the behavior you are observing.
>>
>> You may want to consider using an off-heap region.  Also check out the
>> other eviction algorithms (entry count, region size) to see if those would
>> be appropriate.
>>
>> Anthony
>>
>> > On Jun 17, 2016, at 8:33 AM, Avinash Dongre 
>> wrote:
>> >
>> > Thanks Anthony
>> >
>> > *  At the CRITICAL threshold further writes are blocked. *
>> >
>> > Does client get any kind of warning/error or does put or putAll fails.
>> > My client is just putting KVs and not aware the server State. and I  am
>> > getting
>> >
>> > com.gemstone.gemfire.ForcedDisconnectException: Member isn't responding
>> to
>> > heartbeat requests, and after this server is attempting to reconnect.
>> >
>> > Thanks
>> > Avinash
>> >
>> >
>> >
>> > On Fri, Jun 17, 2016 at 7:50 PM, Anthony Baker 
>> wrote:
>> >
>> >> You need enough heap to retain all your keys in memory.  Values begin
>> to
>> >> be evicted at the EVICTION threshold.  At the CRITICAL threshold
>> further
>> >> writes are blocked.  Your GC settings should be tuned to match these
>> >> thresholds (search the mailing list archives and/or wiki for some
>> advice on
>> >> GC tuning).
>> >>
>> >> Anthony
>> >>
>> >>> On Jun 17, 2016, at 6:42 AM, Avinash Dongre > >
>> >> wrote:
>> >>>
>> >>> Thanks Anthony,
>> >>>
>> >>> I know that my heap is not sufficient for the data I want to put.
>> >>> but My Region is configured as PARTITION_OVERFLOW, so I was hoping
>> that
>> >>> once Geode reaches to critical heap , it will start overflowing
>> >>> the data.
>> >>>
>> >>> Is this assumption correct ?
>> >>>
>> >>> thanks
>> >>> Avinash
>> >>>
>> >>>
>> >>> On Thu, Jun 16, 2016 at 8:21 PM, Anthony Baker 
>> >> wrote:
>> >>>
>>  Hi Avinash,
>> 
>>  The question to answer is “Why was a member removed from the
>> cluster?”
>>  Some things to investigate:
>> 
>>  - Insufficient heap for the data volume
>>  - Excessive GC causing the member to be unresponsive
>>  - OutOfMemory errors in the log
>>  - Overloaded CPU causing delayed heartbeats responses
>> 
>>  HTH,
>>  Anthony
>> 
>> > On Jun 16, 2016, at 6:48 AM, Avinash Dongre <
>> dongre.avin...@gmail.com>
>>  wrote:
>> >
>> > Hello All,
>> >
>> > I am getting following exception when I try to load my system with
>> >> large
>> > amount of data.
>> >
>> > My Setup Details:
>> > 1 locator, 3 cacheservers with 8g and all the regions are disk
>>  persistence
>> > enabled. ( All this is running on single AWS cluster node )
>> >
>> > Please give me some clues what I am missing here ?
>> >
>> >
>> > [severe 2016/06/16 12:51:15.552 UTC S1 
>> tid=0x40]
>> > Uncaught exception in thread Thread[Notification
>> > Handler,10,ResourceListenerInvokerThreadGroup]
>> >
>> >>
>> com.gemstone.gemfire.distributed.DistributedSystemDisconnectedException:
>> > DistributedSystem is shutting down, caused by
>> > com.gemstone.gemfire.ForcedDisconnectException: Member isn't
>> responding
>>  to
>> > heartbeat requests
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.membership.gms.mgr.GMSMembershipManager.directChannelSend(GMSMembershipManager.java:1719)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.membership.gms.mgr.GMSMembershipManager.send(GMSMembershipManager.java:1897)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.DistributionChannel.send(DistributionChannel.java:87)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.DistributionManager.sendOutgoing(DistributionManager.java:3427)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.DistributionManager.sendMessage(DistributionManager.java:3468)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.distributed.internal.DistributionManager.putOutgoing(DistributionManager.java:1828)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.internal.cache.control.ResourceAdvisor$ResourceProfileMessage.send(ResourceAdvisor.java:185)
>> >  at
>> >
>> 
>> >>
>> com.gemstone.gemfire.internal.cache.control.ResourceAdvisor.updateRemoteProfile(ResourceAdvisor.java:448)
>> >  at
>> >
>> 
>> >>
>>

Re: post processing for integrated security

[Michael Stolz]

Post processing is there to filter the data returned by a get or a query.
But it requires that they write code, so it is probably only pertinent for
the old security model not for the new fully configurable End-to-end model
that we have begun discussing.

So MAYBE we're actually doing too much work in keeping it for the new model.

Thoughts?


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Jun 17, 2016 at 11:44 AM, Jinmei Liao  wrote:

> While working on the integrated security for gemfire9/geode1, we are
> frequently baffled by this post-processing functionality that's been lumped
> into security. Here are a few observations we've come up with and want to
> run with the dev community for comments:
>
> 1. Post processing is not authorization. At this point, all necessary
> authorization should already be done and satisfied.
>
> 2. Post processing is related to security in the sense that it needs a
> Principal in the context, but it's not part of the authorization.
>
> 3. Previous implementation (client-server) adds post process after every
> authorization checks. We want to make a clean separation of these two
> concepts. New interfaces of post processing will be introduced (so that we
> can preserve the old behavior if user is implementing the old interface),
> and will ONLY be added to the "get" and "query" data operations.
>
> 4. The new interface will look like below:
> public Object processValue(Principal principal, String regionPath, Object
> key, Object value)
>
> Is this a sufficient interface to begin with? Do you have any client that
> would need post processor to do something completely different?
>
> Thank you for all your feedback.
>
> --
> Cheers
>
> Jinmei
>

Re: [GitHub] incubator-geode pullcskarensmolermsdryugiller"sf c grequest #158: GEODE-1530: geode-examples setEnv.sh scri...DzDd st

[Michael Stolz]

Rsjxtdtdk zee  fg
Dry 4bdtce
-Product cStolz2wdhal ultimate Sr5czfgxcr
Principal Engineer - Gemfirecd PryaoducvtxshTracft Managervszcfggfsd
grrvdXTs
Mobile: 631-x7ehzgrrvdXTsfc6
On Jun 15, 2016 1:30 PM,x7ehzryrzfcr "karensmolermsdryugiller"sf <
g...@git.apache.org> wrote:dry 3HftDc8 esdRsa Gdd
>5czfgxc
> GitHub user ka631-x7ehzryzfcrr4 xv fwsz8s xazopened a pull request:gss52c
c ned
>c
> https://github.com/apache8f/incubator-geode/pull/158
>g Sqsd
> GEODE-1530: geode-examples setEnv.sh script needs to export path
>
> Without an export of the path,  See dfusing the GEODE_HOME env
variable,gSF 6
> the scripts/startAll.sh script wdill pick up the first gfsh in therzHa
> path that it finds.
>
> Also updated the scripts/stopAll.sh script to run the setEnv.sh
> script, so that stopAll.sh uses the correct gfsh.
>
> You can merge this pull request into a Git repository by running:
>
> $ git pull https://github.com/karensmolermiller/incubator-geode
feature/GEODE-1530
>
> Alternatively you can review and apply these changes as the patch at:
>
> https://github.com/apache/incubator-geode/pull/158.patch
>
> To close this pull request, make a commit to your master/trunk branch
> with (at least) the following in the commit message:
>
> This closes #158
>
> 
> commit 1983f9d6969e6e2e051cbda69d908b871812bbab
> Author: Karen Miller 
> Date:   2016-06-15T18:07:03Z
>
> GEODE-1530: geode-examples setEnv.sh script needs to export path
>
> Without an export of the path, using the GEODE_HOME env variable,
> the scripts/startAll.sh script will pick up the first gfsh in the
> path that it finds.
>
> Also updated the scripts/stopAll.sh script to run the setEnv.sh
> script, so that stopAll.sh uses the correct gfsh.
>
> 
>
>
> ---
> If your project is set up for it, you can reply to this email and have
your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working,
please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---

Re: Region Permission

[Michael Stolz]

Permission denied is fine if CLUSTER:READ is disallowed.

The regions returned should be those regions he has access to.

Data Administrator should have access to all regions.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On May 19, 2016 12:22 PM, "Jinmei Liao"  wrote:

> I want to get some clarification on what permission is need to guard the
> operation of "list regions" and "describe region".
>
> Currently anyone that has "CLUSTER:READ" are able to execute those two
> commands, regardless whether he has "READ/WRITE/MANAGE" permissions to the
> regions. And if a user only has read permission for a specific region, when
> he goes to execute "list regions", he will get a "permission denied"
> message instead of seeing a list of regions that he has access to. Is this
> the expected behavior? Or a better question is: what is the expected
> behavior?
>
> --
> Cheers
>
> Jinmei
>

Re: Interest in contributing to a bi temporal framework

[Michael Stolz]

This is largely handled for us by the Geode Partitioned Region mechanics
already.

On a Partitioned Region, there is always a notion of a primary server for
any given key, and some number of secondaries. All writes are always sent
to the primary node, who locally locks the entry, then stores the data into
the entry, and then distributes the update to all of the secondary servers.

I expect that some of the implementation of temporal data access will be
delegated to a CacheWriter or the GemFire Function Execution Service either
of which will execute only on the primary server for a given entry. They
can install the current local server-side timestamp into the entry even
before it is made visible in the cache, and then let the normal Geode
caching mechanics do all the rest.



--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, May 17, 2016 at 3:28 PM, Maarten Niederer 
wrote:

> Hi Mike,
>
>
>
> All right, I’ll start doing my ‘homework’ in order to get myself
> familiarized with the Geode codebase.
>
>
>
> At present I do not wish to learn your ideas as this may cloud my own
> creative thinking process. So we’ll discuss later on that.
>
>
>
> Are you having any concrete timelines on this project right now ?
>
>
>
> Also I’d like to learn your views on my last question, which I will
> elaborate a bit more.
>
>
>
> Suppose two transactions (A, B) are executed on two different nodes which
> have a deviation in system time. Also there is a third process (R) which
> reads data that is affected by both transactions (which will also ‘commit’
> it’s reads in order to be sure to read a consistent state).
>
>
>
> In real time the execution order is A à R à B, but due to system time
> deviation the order of transactions is logged the other way around. As a
> consequence one can’t reconstruct the read state of R. The only way around
> it, which I can think of given my limited knowledge on distributed
> transaction, is to take a global lock (which is not an option in my
> opinion).
>
>
>
> Can anyone think a better solution than keeping system times as close as
> possible together and accepting that there may happen conditions which
> cause
> that you can’t construct the read state of R?
>
>
>
> Maarten,
>
>
>
> I would be interested in spending some cycles thinking through how to best
>
> implement bi-temporality in Geode. I have implemented bi-temporality using
>
> several traditional databases, and I have some ideas how to implement both
>
> time-series and bi-temporal data in Geode, but it would be interesting to
>
> have someone who really knows the subject matter well to work with.
>
>
>
> --
>
> Mike Stolz
>
> Principal Engineer, GemFire Product Manager
>
> Mobile: 631-835-4771
>
>
>
> On Sun, May 15, 2016 at 6:18 PM, Maarten Niederer <
>  maar...@burgemeestre.nl>
>
> wrote:
>
>
>
> Hi guys,
>
>
>
> I'm considering to contribute to the Geode project. My attention is drawn
>
> because of the desire to create a bi temporal framework in a distributed
>
> database system. This should provide me nice engineering challenges.
>
>
>
> At my present employer I have worked with and developed an application that
>
> implements a bi temporal framework within the application layer build on
>
> top
>
> of an old school RDBMS. I have done a lot development with temporal logic
>
> (value time). For example, I've implemented several  SQL operations (e.g.
>
> outer joins, group/aggregate) in a temporal way (e.g. find me the intervals
>
> in time when more developers than managers worked at company X given the
>
> temporal data on when people started/stopped working and their
>
> professions).
>
> Also I've been busy with implementing the procedure that writes the change
>
> log for these temporal tables which allows to inspect these tables at any
>
> transaction time. During this work I found several shortcomings to
>
> implementing these features within the application layer, which could be
>
> only resolved if there's proper support in the RDBMS instead.
>
>
>
> So I know a lot about bi temporal data, I know only a bit on distributed
>
> transactions (I learned some in university and the clubhouse presentation
>
> by
>
> Swapnil was excellent for knowledge transfer). Finally I don't know
>
> anything
>
> about the Geode code base. So my first action would be to tackle starter
>
> JIRAs for half a year or so in order to get some familiarity with the
>
> codebase.
>
>
>
> However before I start to blow a big shot of my personal time, I'd like to
>
> know whether there are several people who like to support implementing the
>
> bi temporal framework. As it requires thorough review both on design and
>
> code (and preferably even test coverage). Also for the work on the
>
> transaction time dimension, the current work being done on the transactions
>
> should be completed. Also we will probably need a fundamental discussion on
>
> 'transaction time' and correspondin

Re: Interest in contributing to a bi temporal framework

[Michael Stolz]

Maarten,

I would be interested in spending some cycles thinking through how to best
implement bi-temporality in Geode. I have implemented bi-temporality using
several traditional databases, and I have some ideas how to implement both
time-series and bi-temporal data in Geode, but it would be interesting to
have someone who really knows the subject matter well to work with.


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Sun, May 15, 2016 at 6:18 PM, Maarten Niederer 
wrote:

> Hi guys,
>
>
>
> I'm considering to contribute to the Geode project. My attention is drawn
> because of the desire to create a bi temporal framework in a distributed
> database system. This should provide me nice engineering challenges.
>
>
>
> At my present employer I have worked with and developed an application that
> implements a bi temporal framework within the application layer build on
> top
> of an old school RDBMS. I have done a lot development with temporal logic
> (value time). For example, I've implemented several  SQL operations (e.g.
> outer joins, group/aggregate) in a temporal way (e.g. find me the intervals
> in time when more developers than managers worked at company X given the
> temporal data on when people started/stopped working and their
> professions).
> Also I've been busy with implementing the procedure that writes the change
> log for these temporal tables which allows to inspect these tables at any
> transaction time. During this work I found several shortcomings to
> implementing these features within the application layer, which could be
> only resolved if there's proper support in the RDBMS instead.
>
>
>
> So I know a lot about bi temporal data, I know only a bit on distributed
> transactions (I learned some in university and the clubhouse presentation
> by
> Swapnil was excellent for knowledge transfer). Finally I don't know
> anything
> about the Geode code base. So my first action would be to tackle starter
> JIRAs for half a year or so in order to get some familiarity with the
> codebase.
>
>
>
> However before I start to blow a big shot of my personal time, I'd like to
> know whether there are several people who like to support implementing the
> bi temporal framework. As it requires thorough review both on design and
> code (and preferably even test coverage). Also for the work on the
> transaction time dimension, the current work being done on the transactions
> should be completed. Also we will probably need a fundamental discussion on
> 'transaction time' and corresponding 'read consistency' in a distributed
> context.  (Is there always some snapshot which matches exactly what a
> client
> observed at some point in time?).
>
>
>
> Also good for you to know, most of my time I reside in the CET timezone.
>
>
>
> Best regards,
>
> Maarten Niederer
>
>

Re: Do we want a "theme song" for Geode Cluhouse?

[Michael Stolz]

Yes I do.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, May 3, 2016 at 6:02 PM, Gregory Chase  wrote:

> +1 Love it!  Do you give permission to Apache Geode community contributors
> to make use of your original work as part of Apache Geode virtual and
> physical events?
>
> -Greg
>
> On Tue, May 3, 2016 at 2:59 PM, Michael Stolz  wrote:
>
> > How about this one?
> >
> > Authored and played by yours truly.
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
> > On Mon, May 2, 2016 at 5:09 PM, Greg Chase  wrote:
> >
> >> Dear Geode Community,
> >> In a hallway conversation with one of the committers, I got asked, "How
> >> come Geode doesn't get a theme song".  They pointed to some work I do
> for
> >> the Greenplum Community in running their virtual meetings:
> >> https://www.youtube.com/watch?v=S9nHSKBSrlg
> >>
> >> So, I'm very happy to bring in a theme song for the community.  We just
> >> need to know what people would want to use.
> >>
> >> The constraints are that these would need to be music tracks, probably
> on
> >> Sound Cloud, released under a Creative Common's license permitting reuse
> >> without royalty, and with modification.
> >>
> >> I don't mind attributing the creator if I'm producing the video.
> >>
> >> So my ask to the Geode community is - please nominate some sound tracks
> to
> >> use as bumper music for Geode videos..
> >>
> >> Here's one nomination by me.  I happen to know the composer, and
> although
> >> I
> >> can't find the license he released for this track, I'm sure he'll
> publish
> >> a
> >> version under CC for our usage if we ask:
> >> https://soundcloud.com/choirshark/bongo1110
> >>
> >
> >
>
>
> --
> Greg Chase
>
> Global Head, Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>

Re: What will it take to release geode 1.0?

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Apr 29, 2016 11:59 AM, "Dan Smith"  wrote:

> We've been releasing milestones of 1.0, but at some point we actually have
> to release a real geode 1.0 :)
>
> What is keeping us from releasing geode 1.0 at this point? Just the issues
> currently marked with Fix Version M3? I think we should nail down the scope
> of 1.0 and track our progress to the 1.0 release.
>
> From the apache process perspective, I don't think 1.0 is any different
> from the milestone releases we've done so far. The only difference with 1.0
> is what it means to the geode community.
>
> Gemfire maintained backwards compatibility with previous releases for
> persistent files, client/server, WAN, and P2P messaging. I think once we
> release 1.0 we should provide that same guarantee that the next geode
> release will be backwards compatible with 1.0.
>
> We do still have the package renaming (GEODE-37) on the horizon. My
> suggestion is that in the interests of getting 1.0 out the door, at this
> point we should just release geode 1.0 with the old packages and rename
> packages in geode 2.0.
>
> Thoughts?
>
> -Dan
>

Re: Next steps with flickering tests

[Michael Stolz]

I love it. Isolate the bad stuff and even evaluate it's value in the big
scheme of things. Fix the important ones. Maybe rethink the rest.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Apr 25, 2016 6:54 PM, "Kirk Lund"  wrote:

> After completing GEODE-1233, all currently known flickering tests are now
> annotated with our FlakyTest JUnit Category.
>
> In an effort to divide our build up into multiple build pipelines that are
> sequential and dependable, we could consider excluding FlakyTests from the
> primary integrationTest and distributedTest tasks. An additional build task
> would then execute all of the FlakyTests separately. This would hopefully
> help us get to a point where we can depend on our primary testing tasks
> staying green 100% of the time. We would then prioritize fixing the
> FlakyTests and one by one removing the FlakyTest category from them.
>
> I would also suggest that we execute the FlakyTests with "forkEvery 1" to
> give each test a clean JVM or set of DistributedTest JVMs. That would
> hopefully decrease the chance of a GC pause or test pollution causing
> flickering failures.
>
> Having reviewed lots of test code and failure stacks, I believe that the
> primary causes of FlakyTests are timing sensitivity (thread sleeps or
> nothing that waits for async activity, timeouts or sleeps that are
> insufficient on busy CPU or I/O or during due GC pause) and random ports
> via AvailablePort (instead of using zero for ephemeral port).
>
> Opinions or ideas? Hate it? Love it?
>
> -Kirk
>

Incubator status report on wiki?

[Michael Stolz]

Should the geode incubator status report be posted on the geode wiki under
"Incubation Status Reports"?

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

Hyphens in Region names

[Michael Stolz]

I just discovered a problem with the use of a hyphen in Region names.
It conflicts with something in the export mechanism in Geode:

>From page 281 of the 8.2 users guide:

"Exporting Cache Snapshots
When you export an entire cache, it exports all regions in the cache as
individual snapshot files into a directory. If no directory is specified,
the default is the current directory. A snapshot file is created for each
region, and the export operation automatically names each snapshot filename
using the following convention: snapshot-[-]* When the export operation
writes the snapshot filename, it replaces each forward slash ('/') in the
region path with a dash ('-')."

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

Re: Can we close GEODE-36 ? (Gfsh renaming)

[Michael Stolz]

+1
Or Geode Fabric SHell

We used to call it a Data Fabric at one time :)

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Wed, Mar 16, 2016 at 10:19 AM, Anthony Baker  wrote:

> +1
>
> I think Geode Fabulous SHell is a reasonable name.  Changing it generates
> a ton of work both in the code and documentation without much benefit IMO.
>
> Link to the original discussion:
>
> https://mail-archives.apache.org/mod_mbox/incubator-geode-dev/201505.mbox/%3cCAF3UAT1FRANKyLUXeSG-m6yNXnTyw=3heo5350vf-80hvfz...@mail.gmail.com%3e
>
> Anthony
>
>
> > On Mar 15, 2016, at 9:41 PM, William Markito 
> wrote:
> >
> > Howdy!
> >
> > Looks like we had this discussion in the past but given my own current
> > understanding of the impacts and needs for such change, I'd like to ask
> if
> > we can close GEODE-36 with the changes Nitin already did to display the
> > product name properly and keep it as is...
> >
> >
> > We can even update the banner if that's a thing, but I'm giving my +1 to
> > keep it gfsh.
> >
> > o   o
> >  /^7
> >'  ' ,oOOo,
> >   ,'))), /{
> >  '  ,'o  ={
> >>   ={
> > `,   ))\ \)))={
> >   ',\/)' \{
> > '*OO*'
> >
> >
> > --
> >
> > ~/William
>
>

Re: Hyphens in Region names

[Michael Stolz]

If the filename isn't used then it shouldn't be a problem.
I just stumbled over that in the user's guide and thought we should check
before we actually formally ALLOW hyphens as has been requested.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Mar 17, 2016 at 6:38 PM, Jens Deppe  wrote:

> Mike,
>
> Why do you think this is a problem? Is the filename used, at some later
> stage, to infer the region name?
>
> --Jens
>
> On Thu, Mar 17, 2016 at 3:24 PM, Michael Stolz  wrote:
>
> > I just discovered a problem with the use of a hyphen in Region names.
> > It conflicts with something in the export mechanism in Geode:
> >
> > From page 281 of the 8.2 users guide:
> >
> > "Exporting Cache Snapshots
> > When you export an entire cache, it exports all regions in the cache as
> > individual snapshot files into a directory. If no directory is specified,
> > the default is the current directory. A snapshot file is created for each
> > region, and the export operation automatically names each snapshot
> filename
> > using the following convention: snapshot-[-]* When the export operation
> > writes the snapshot filename, it replaces each forward slash ('/') in the
> > region path with a dash ('-')."
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
>

Re: geode for specific usecase

[Michael Stolz]

If you are storing the data using pdx auto seriaization the index will work
just fine with your custom object keys.


There is no short answer to how much extra memory overhead an index will
cause because it is dependent on the actual data itself (really the
cardinality of the data). See
https://en.wikipedia.org/wiki/Cardinality_(SQL_statements)

They do tend to be large. Use indices judiciously.



--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Wed, Feb 24, 2016 at 12:34 PM, steve mathew 
wrote:

> Thank you Dan.
>
> Will this work with custom object keys?
> How much will be extra memory overhead when creating index?
>
> Steve.
>
> On Wed, Feb 24, 2016 at 11:36 PM, Dan Smith  wrote:
>
> > Hi Steve,
> >
> > I think your best bet is probably to use an OQL index. You can create an
> > OQL index and then do range queries through OQL.
> >
> > You can see some examples of OQL queries here:
> >
> >
> >
> http://geode.docs.pivotal.io/docs/getting_started/querying_quick_reference.html#reference_D5CE64F5FD6F4A808AEFB748C867189E
> >
> > Eg
> > SELECT * from /exampleRegion value WHERE value.ID > startKey AND
> value.ID <
> > endKey
> >
> > -Dan
> >
> > On Tue, Feb 23, 2016 at 10:37 PM, steve mathew  >
> > wrote:
> >
> > > Hello I am new to the apache geode and exploring a possibility to store
> > > data sorted into regions.
> > >
> > > I need to store data sorted for the scan(startkey, endkey) operation.
> > > Simple looking at the APIs it looks geode is not designed for this. Is
> > > there any way to achieve this?
> > >
> > > Thanks in advance.
> > >
> > > Kindly advise.
> > >
> > > -Steve.
> > >
> >
>

Re: API Cleanup with GEODE-37?

[Michael Stolz]

We are working on figuring out a way to still support the old packaging for
existing enterprise customers when we switch to org.apache packaging.

Let's not make the packaging the reason for making a bunch of other
breaking changes.

+1 for dumping deprecated API's.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, Feb 23, 2016 at 10:45 AM, Swapnil Bawaskar 
wrote:

> Since we are going to rename packages from com.gemstone -> org.apache for
> GEODE-37 https://issues.apache.org/jira/browse/GEODE-37>, and break
> any existing applications, I think we should take time to clean-up the API
> a little bit. Below are a few things that I would like to suggest.
>
> Region has some methods that are only supported on clients (like
> RegisterInterest). We can break region interface into two, one for
> Peer-to-Peer and one for client.
>
> We may be able to make some of the changes mentioned in the function
> service improvements document
> <
> https://cwiki.apache.org/confluence/display/GEODE/Function+Service+Usability+Improvments
> >
> (GEODE-728,
> 336, 641).
>
> Get rid of all the deprecated API.
>
> Thanks!
> Swapnil.
>

Re: Valid characters in Region names

[Michael Stolz]

There is a stated position in the commercial GemFire documentation as
follows:

To get the full range of Pivotal GemFire capabilities for your cached data
regions, follow GemFire's region naming guidelines:
The safest approach when naming your regions is to use only alphanumeric
characters and the underscore character and to keep your region names
short.
This permits you the full range of GemFire region configurations and allow
you to perform all available GemFire operations on the region.
Follow these guidelines for naming regions:
• Do not use the separator slash character ‘ / ’.
• Do not begin region names with two underscore characters "__" as this is
reserved for internal GemFire use.
• To run queries against your data, restrict your region names to
alphanumeric characters and the underscore character.


For Geode we should probably tighten that up just a bit further by changing
the last bullet to:
• Restrict your region names to alphanumeric characters and the underscore
character.



--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Feb 18, 2016 at 2:09 PM, William Markito 
wrote:

> I don't think we should allow non-alphanumeric region names...   And would
> be really nice to have a list or a pattern documenting what's valid.
>
>
> On Thu, Feb 18, 2016 at 11:04 AM Kirk Lund  wrote:
>
> > I was just looking into a ticket filed because /= was used as the Region
> > name and this caused problems in JMX ObjectNames. I have two questions:
> 1)
> > do we really want /= to be a valid Region name? 2) do we have a complete
> > list somewhere of all the non-alphanumeric characters that are usable in
> > Region names?
> >
> > -Kirk
> >
> --
> ~/William
>

Re: Three redundancy zones

[Michael Stolz]

Dan,

That's exactly what I'm thinking. 3 copies of data across 3 racks, largely
eliminating the likelihood that one rack failing can cause the other to
shoot himself in the head.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Feb 10, 2016 12:15 PM, "Dan Smith"  wrote:

> Sure, you can have as many redundancy zones as you want.
>
> I'm don't think the redundancy zone setting affects quorum calculations.
> But if you can do something with the physical topology (3 different racks?)
> that makes it less likely to lose half of the members at once that might
> help.
>
> -Dan
>
>
> On Wed, Feb 10, 2016 at 9:49 AM, Michael Stolz  wrote:
>
> > Is it possible to set up 3 separate redundancy zones so that a network
> > segmentation would have less chance of making the wrong decision about
> > quorum?
> >
> > --
> > Mike Stolz
> > Principal Engineer, GemFire Product Manager
> > Mobile: 631-835-4771
> >
>

Three redundancy zones

[Michael Stolz]

Is it possible to set up 3 separate redundancy zones so that a network
segmentation would have less chance of making the wrong decision about
quorum?

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

Re: [DISCUSS] Next Release Scope - version 1.0.0-incubating.M2

[Michael Stolz]

I'd like to see the focus on docs and dependencies first.
Then incorporation of WAN and CQ features.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Jan 29, 2016 at 12:26 PM, Nitin Lamba  wrote:

> Thought of starting a separate thread to discuss next release scope. So
> far we have the following suggestions:
>
> Dan:
> - GEODE-823 - remove gemfire-joptsimple, gemfire-json, rename to geode-,
> other misc issues
> - GEODE-818 - clean up dependencies
> - GEODE-572 - generate public javadocs (part of GEODE-54)
> - GEODE-33 - create project examples
>
> John:
> - GEODE-27 - fix POM dependencies - related to RC feedback above
> (GEODE-818)
>
> Few left-over from prior discussions:
>
> - GEODE-386 - fixing xsd namespace to Apache
> - GEODE-36 - fixing gfsh cli (Removing GemFire references)
> - GEODE-37 - package re-naming
>
> Some of these are big ticket items so we may have to space these out
> across Milestones.
>
> ACTION: Please suggest on this thread OR edit the wiki page[1] so that
> JIRA [2] can be prioritized accordingly.
>
> Thanks,
> Nitin
>
> [1]
> https://cwiki.apache.org/confluence/display/GEODE/1.0.0-incubating.M2+%28Second%29+Release
> [2]
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&view=planning
>
>

Re: [ANNOUNCE] Donation of GemFire WAN and CQ code

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jan 19, 2016 7:29 PM, "Anthony Baker"  wrote:

> I am pleased to announce the donation of additional GemFire code to the
> Geode community.  The code being donated adds significant capabilities used
> by many GemFire customers in production, namely WAN replication and
> Continuous Query.
>
> *WAN replication* provides event delivery and data synchronization for
> multi-site distributed clusters.  Asynchronous, persistent event delivery
> to remote systems enables patterns such Active/Active, Disaster Recovery,
> or Follow-the-Sun with support for configurable, eventually consistent data
> replication.  *Continuous Querying* allows clients to subscribe to server
> events by using SQL-like query filtering.  Events that match or modify the
> query results are queued and delivered to the client using the
> client/server subscription framework.
>
> The Software Grant Agreement for this code has been accepted by the ASF
> secretary.
>
> The donated code currently sits in a separate branch in the Geode
> repository named wan_cq_donation [1] and is awaiting community feedback.  I
> encourage everyone in the Geode community to review this donation and
> provide feedback.  Once the community has reached a consensus we can
> determine next steps and how this code might get merged into the develop
> branch so that all users can access these features.  Your suggestions are
> most welcome!
>
>
> Thanks,
> Anthony
>
> [1]
> https://git1-us-west.apache.org/repos/asf?p=incubator-geode.git;a=tree;h=refs/heads/wan_cq_donation;hb=refs/heads/wan_cq_donation
>
>

Re: [ANNOUNCE] Donation of GemFire WAN and CQ code

[Michael Stolz]

This doesn't yet include gemz.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jan 20, 2016 11:36 AM, "David Muirhead"  wrote:

> Hi Mike -
>
> That’s great news. Will that include Gem/Z? Hope so!
>
> Thanks,
>
> Dave
>
> 
>
> Dave Muirhead
> Blue River Systems Group, LLC
> http://www.brsg.biz
> d...@brsg.biz
> 303-638-4618 Mobile
> 303-309-6242 Office
> 303-309-6243 Fax
>
>
>
> > On Jan 20, 2016, at 2:21 AM, Michael Stolz  wrote:
> >
> > We are in the process of bringing it all open. Just a matter of time and
> > effort.
> >
> > --
> > Mike Stolz
> > Principal Engineer - Gemfire Product Manager
> > Mobile: 631-835-4771
> > On Jan 19, 2016 10:19 PM, "Catherine Johnson"
> >  wrote:
> >
> >> Are there still bits that are only included with the commercial product
> >> GemFire? Or is everything open source now?
> >>
> >> Thanks!
> >> Catherine
> >>
> >>> On Jan 20, 2016, at 12:28 PM, Anthony Baker  wrote:
> >>>
> >>> I am pleased to announce the donation of additional GemFire code to the
> >> Geode community.  The code being donated adds significant capabilities
> used
> >> by many GemFire customers in production, namely WAN replication and
> >> Continuous Query.
> >>>
> >>> *WAN replication* provides event delivery and data synchronization for
> >> multi-site distributed clusters.  Asynchronous, persistent event
> delivery
> >> to remote systems enables patterns such Active/Active, Disaster
> Recovery,
> >> or Follow-the-Sun with support for configurable, eventually consistent
> data
> >> replication.  *Continuous Querying* allows clients to subscribe to
> server
> >> events by using SQL-like query filtering.  Events that match or modify
> the
> >> query results are queued and delivered to the client using the
> >> client/server subscription framework.
> >>>
> >>> The Software Grant Agreement for this code has been accepted by the ASF
> >> secretary.
> >>>
> >>> The donated code currently sits in a separate branch in the Geode
> >> repository named wan_cq_donation [1] and is awaiting community
> feedback.  I
> >> encourage everyone in the Geode community to review this donation and
> >> provide feedback.  Once the community has reached a consensus we can
> >> determine next steps and how this code might get merged into the develop
> >> branch so that all users can access these features.  Your suggestions
> are
> >> most welcome!
> >>>
> >>>
> >>> Thanks,
> >>> Anthony
> >>>
> >>> [1]
> >>
> https://git1-us-west.apache.org/repos/asf?p=incubator-geode.git;a=tree;h=refs/heads/wan_cq_donation;hb=refs/heads/wan_cq_donation
> >> <
> >>
> https://git1-us-west.apache.org/repos/asf?p=incubator-geode.git;a=tree;h=refs/heads/wan_cq_donation;hb=refs/heads/wan_cq_donation
> >>>
> >>>
> >>
> >>
>
>

Re: [VOTE] Apache Geode (Incubating) first Milestone release - v1.0.0-incubating.M1

[Michael Stolz]

What caused it to have such a short deadline? Did we do something to cause
it to be so short or is that the default and we just didn't know to
override it somehow?

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, Jan 19, 2016 at 8:36 PM, Roman Shaposhnik 
wrote:

> On Tue, Jan 19, 2016 at 8:18 PM, Niall Pemberton
>  wrote:
> > On Tue, Jan 19, 2016 at 9:53 PM, Nitin Lamba  wrote:
> >
> >> This is the first release for Apache Geode, version 1.0.0-incubating.M1.
> >> Thanks to all the community members to drive towards this first
> milestone!
> >>
> >> It fixes the following issues:
> >>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12334248
> >>
> >> *** Please download, test and vote by Wednesday, January 20th, 1700 hrs
> US
> >> Pacific.
> >>
> >
> > Votes typically run for at least 3 days at the ASF to allow people time
> to
> > review/vote. Its not a fixed rule, but I think there needs to be a good
> > reason for reducing the time. I've had to stay up late in order to review
> > this before your deadline since I dont get time to do this in my $job.
> Just
> > my 2cents, but I think it would be more inclusive to stick to the normal
> 3
> > days.
>
> Super strong +1 to the above! Personally, there's not way I can review
> under this deadline (and I suspect neither could any of the mentors).
>
> Thanks,
> Roman.
>
> P.S. Although, arguably one way for podling to learn this is the hard way.
> If you don't get any binding votes under such a tight deadline you kind of
> get the idea of why votes like this in ASF run for at least 72 hours.
>

Re: [ANNOUNCE] Donation of GemFire WAN and CQ code

[Michael Stolz]

We are in the process of bringing it all open. Just a matter of time and
effort.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Jan 19, 2016 10:19 PM, "Catherine Johnson"
 wrote:

> Are there still bits that are only included with the commercial product
> GemFire? Or is everything open source now?
>
> Thanks!
> Catherine
>
> > On Jan 20, 2016, at 12:28 PM, Anthony Baker  wrote:
> >
> > I am pleased to announce the donation of additional GemFire code to the
> Geode community.  The code being donated adds significant capabilities used
> by many GemFire customers in production, namely WAN replication and
> Continuous Query.
> >
> > *WAN replication* provides event delivery and data synchronization for
> multi-site distributed clusters.  Asynchronous, persistent event delivery
> to remote systems enables patterns such Active/Active, Disaster Recovery,
> or Follow-the-Sun with support for configurable, eventually consistent data
> replication.  *Continuous Querying* allows clients to subscribe to server
> events by using SQL-like query filtering.  Events that match or modify the
> query results are queued and delivered to the client using the
> client/server subscription framework.
> >
> > The Software Grant Agreement for this code has been accepted by the ASF
> secretary.
> >
> > The donated code currently sits in a separate branch in the Geode
> repository named wan_cq_donation [1] and is awaiting community feedback.  I
> encourage everyone in the Geode community to review this donation and
> provide feedback.  Once the community has reached a consensus we can
> determine next steps and how this code might get merged into the develop
> branch so that all users can access these features.  Your suggestions are
> most welcome!
> >
> >
> > Thanks,
> > Anthony
> >
> > [1]
> https://git1-us-west.apache.org/repos/asf?p=incubator-geode.git;a=tree;h=refs/heads/wan_cq_donation;hb=refs/heads/wan_cq_donation
> <
> https://git1-us-west.apache.org/repos/asf?p=incubator-geode.git;a=tree;h=refs/heads/wan_cq_donation;hb=refs/heads/wan_cq_donation
> >
> >
>
>

Re: [VOTE] Apache Geode (Incubating) first Milestone release - v1.0.0-incubating.M1

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Tue, Jan 19, 2016 at 5:02 PM, William Markito 
wrote:

> +1
>
> On Tue, Jan 19, 2016 at 1:53 PM, Nitin Lamba  wrote:
>
> > This is the first release for Apache Geode, version 1.0.0-incubating.M1.
> > Thanks to all the community members to drive towards this first
> milestone!
> >
> > It fixes the following issues:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12334248
> >
> > *** Please download, test and vote by Wednesday, January 20th, 1700 hrs
> US
> > Pacific.
> >
> > Note that we are voting upon the source (tag):
> >rel/1.0.0-incubating.M1.RC1
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;a=tag;h=refs/tags/rel/v1.0.0-incubating.M1.RC1
> >
> >
> > Commit ID: e5a7b9aaa82d4c0a04e41febfd515056c4669001
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;a=commit;h=e5a7b9aaa82d4c0a04e41febfd515056c4669001
> >
> >
> > Source and binary files:
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/geode/1.0.0-incubating.M1.RC1/
> >
> > For the first release, the documentation on how to install and use Apache
> > Geode are hosted on pivotal.io:
> >http://geode.docs.pivotal.io
> >
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachegeode-1000/
> >
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> >
> >
> https://github.com/apache/incubator-geode/blob/release/1.0.0-incubating.M1/KEYS
> >
> >
> > Release Key: pub  4096R/C72CFB64 2015-10-01
> >
> > Thanks,
> > Nitin & Anthony
> >
> >
> > 
> >
> >
>
>
> --
>
> ~/William
>

Re: enabling network partition detection by default

[Michael Stolz]

I don't think I got my point across correctly.

If we change defaults we need to document that, of course, and not just in
our 1000+ page manual because that will not be read soon enough to avoid
problems from the changed defaults. We definitely need that to appear in
the release notes.

I think we should create a "legacy_default.properties" file that contains
the old default settings for any defaults we are going to change and make
that available to legacy users so that they KNOW what the defaults WERE,
and can use some or all of that file to augment their current
gemfire.properties file to put the defaults back the way they have been
running up until now.


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Jan 8, 2016 at 2:38 PM, Bruce Schuchardt 
wrote:

> jchen21 (Jianxia) has volunteered to do this work and is looking into what
> needs to be done.
>
>
> Le 1/7/2016 1:18 PM, Bruce Schuchardt a écrit :
>
>> Another thing that's been discussed for a long time is turning on
>> network-partition-detection by default.   It is a major problem for someone
>> if a partition occurs and they are using persistence. The disk-stores on
>> all but one of the partitions have to be deleted and revoked.
>>
>
>

Re: enabling network partition detection by default

[Michael Stolz]

I'm not sure I want to do wholesale defaults revision.

Yes this is a release that cannot be done in a rolling fashion, but it is
not (yet) a release that requires that pre-existing users to change their
settings in general to keep their current configuration.

I think it would be simple enough to supply a "recommended.properties" file
that sets things the way we now like them without actually changing the
hard-coded defaults.

I view the network-partition-detection property as a special case because
of its affect on diskstores which are automatically created if you use the
recommended serialization mechanism (pdx).

On the diskstore topic, if we know that the disk stores are wrong at
startup, why don't we just fix them? Why do we require user intervention?


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Fri, Jan 8, 2016 at 11:13 AM, Real Wes Williams 
wrote:

> What’s the level of concern here about members getting kicked out
> prematurely depending on the newly proposed default settings?  For
> instance, if the default suspect notification is 3 seconds and they are
> running in AWS or a mildly unpredictable network environment, a member
> could be kicked out.  What would be considered “safe” settings?
>
> > On Jan 7, 2016, at 4:18 PM, Bruce Schuchardt 
> wrote:
> >
> > Another thing that's been discussed for a long time is turning on
> network-partition-detection by default.   It is a major problem for someone
> if a partition occurs and they are using persistence.  The disk-stores on
> all but one of the partitions have to be deleted and revoked.
>
>

Re: enabling network partition detection by default

[Michael Stolz]

I never knew that. Yikes. Given that just using PDX causes the use of disk
stores this seems really important.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Jan 7, 2016 at 4:18 PM, Bruce Schuchardt 
wrote:

> Another thing that's been discussed for a long time is turning on
> network-partition-detection by default.   It is a major problem for someone
> if a partition occurs and they are using persistence.  The disk-stores on
> all but one of the partitions have to be deleted and revoked.
>

Re: [VOTE] Added specific criteria for nomination and voting for new Committers to Apache Geode wiki

[Michael Stolz]

+1 for Dan's updates (I think there is a word missing between "standards
can" should be "standards and can")

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Thu, Jan 7, 2016 at 2:33 PM, Mark Bretl  wrote:

> +1 For Dan's updates. I agree #2 does not help with understanding criteria.
>
> --Mark
>
> On Thu, Jan 7, 2016 at 11:19 AM, Dan Smith  wrote:
>
> > Looks pretty good to me. I'd suggest a slight edit #3:
> >
> > Once a contributor shows a consistent history of participating in the
> > development process or community *and has demonstrated that they
> understand
> > the development process and standards can be trusted with commit
> access*, a
> > Committer should nominate them to the Apache Geode PPMC for consideration
> > to become a Committer.
> >
> > I would remove #2 - it's a good point, but it's not really helping
> someone
> > understand what the criteria are for becoming a committer
> >
> > -Dan
> >
> > On Thu, Jan 7, 2016 at 11:06 AM, Jens Deppe  wrote:
> >
> > > +1
> > >
> > > On Thu, Jan 7, 2016 at 10:49 AM, Greg Chase 
> wrote:
> > >
> > > > Greetings,
> > > > Per the discuss thread on this topic, I've added new text to our wiki
> > > page
> > > > describing how to become a Committer.
> > > >
> > > > This text includes a generalized description of what I believe the
> > > > consensus is in the discussion thread.
> > > >
> > > > Changed page can be found here:
> > > >
> https://cwiki.apache.org/confluence/display/GEODE/Becoming+a+committer
> > > >
> > > > Lazy consensus applies.
> > > >
> > > > -Greg
> > > >
> > >
> >
>

Re: LICENSE and NOTICE

[Michael Stolz]

I don't see a GEODE-210 branch. Am I missing something?

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Mon, Jan 4, 2016 at 11:19 AM, Anthony Baker  wrote:

> Look on the feature/GEODE-210 branch.
>
> The source distribution versions are in the root directory.
> The binary distribution versions are in gemfire-assembly/src/main/dist/
>
> Each distribution has different requirements since different external
> components are bundled.
>
> Anthony
>
> > On Jan 4, 2016, at 8:17 AM, Dave Barnes  wrote:
> >
> > Where can I find these files?
> >
> > On Mon, Jan 4, 2016 at 7:16 AM, Anthony Baker  wrote:
> >
> >> I’ve made a start on our LICENSE and NOTICE files (for both source and
> >> binary distributions) on the feature/GEODE-610 branch.  Thanks to Niall
> for
> >> the excellent analysis to get this started.  If you’d like to
> contribute to
> >> this you can:
> >>
> >> 1) Review the contents of the LICENSE files and see if they look right.
> >> 2) Figure out what needs to go in the NOTICE files.
> >>
> >> This is needed for the alpha1 release.
> >>
> >> Thanks,
> >> Anthony
> >>
> >>
>
>

Re: Include templates in the binary distribution?

[Michael Stolz]

Seems to me they should be. All the examples for that matter.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Mon, Dec 7, 2015 at 2:56 PM, Dan Smith  wrote:

> Hi,
>
> Gemfire used to ship with some examples of security authenticators and
> initializers. In geode, that example code is sitting under
> gemfire-core/src/test/java/templates.
>
> Should those examples be distributed as part of the binary tarball for
> geode?
>
> -Dan
>

Re: xml schema locations

[Michael Stolz]

If it is possible to go directly to http://geode.apache.org we should
probably do that.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Dec 1, 2015 7:54 AM, "Niall Pemberton"  wrote:

> On Thu, Oct 8, 2015 at 7:41 PM, William Markito 
> wrote:
>
> > I'm concerned that we will need another update on applications and other
> > usages and after TLP, since we'll drop the .incubator of the url.
> >
> > But I don't have a better alternative yet to propose.
> >
>
> Just seen this email[1] on general@incubator and http://geode.apache.org/
> does work!
>
> [1] markmail.org/message/zhq7dxb4g4bzv6dl
>
> Niall
>
>
>
>
> >
> >
> > On Thu, Oct 8, 2015 at 11:23 AM, Dan Smith  wrote:
> >
> > > I filed a JIRA for changing the xml schema location from pivotal to
> > apache,
> > > see GEODE-386.
> > >
> > > I'm proposing hosting the schemas at
> > >
> > > http://geode.incubator.apache.org/schema/cache
> > >
> > > For example:
> > > http://geode.incubator.apache.org/schema/cache/cache-1.0.xsd
> > >
> > > Does this meet with everyone's approval? Will there be an issues with
> > > putting schemas in this location on the website?
> > >
> > > -Dan
> > >
> >
> >
> >
> > --
> >
> > William Markito Oliveira
> > Enterprise Architect
> > -- For questions about Apache Geode, please write to
> > *dev@geode.incubator.apache.org
> > *
> >
>

Re: Review of 1.0.0-alpha1 issues

[Michael Stolz]

I would like to see one alpha. I don't think we're sure how solid some of
this stuff is.

"Testing cannot prove the absence of bugs...only the presence."

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 30, 2015 4:28 PM, "Nitin Lamba"  wrote:

> Thanks Anil,
>
> From the best practices section:
>
> "TODO: (move to a note) Notes on 0.x releases verses alpha and betas. It
> is better to use 0.x releases than to create numerous alphas and betas for
> 1.0. Conventionally, 0.x releases are aimed at early adopters only without
> a strong promise of easy upgrade or backwards compatibility. 0.x is a good
> designation for a product that is not feature complete but whose code is
> solid for those features which are implemented. "
>
> My preference would be for dot releases instead of alpha1, apha2, beta,
> RC, etc. Other thoughts?
>
> As for the release tasks, perhaps you missed the Wiki page:
>
> https://cwiki.apache.org/confluence/display/GEODE/1.0.0-alpha1+%28First%29+Release
>
> The JIRA (Agile) Board is here:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&view=planning
>
> Was hoping to get more discussion/ feedback from the community on the Wiki
> before JIRA versions and tasks are created. Please review/ comment the
> contents, if not done already.
>
> Thanks,
> -Nitin
> 
> From: Anilkumar Gingade 
> Sent: Monday, November 30, 2015 2:47 PM
> To: dev@geode.incubator.apache.org
> Subject: Re: Review of 1.0.0-alpha1 issues
>
> Here is more detail on versioning
>
>
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-naming
>  (under versioning section)
>
> Can we set a release task list...
> E.g.: What tasks/items are expected to complete to do an alpha, beta and
> final 1.0 release...
>
> -Anil.
>
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Nov 30, 2015 at 1:33 PM, Anthony Baker  wrote:
>
> >
> > On Nov 30, 2015, at 11:23 AM, Nitin Lamba  wrote:
> >
> > +1
> >
> > I do have a fundamental question about versioning (rather what versions
> > can be taken for voting/ approvals):
> >
> > Can an 'alpha' which has known issues (like GEODE-27 or GEODE-386 for
> > apache namespace, etc) be taken all the way through the PMC process for
> > approvals? Such a release will have to be posted to maven/ apache builds
> > but does it meet the quality requirements for an 'Apache Release’?
> >
> >
> > Good question.  I was assuming we would need to address GEODE-386 before
> > graduation but not for the first release.
> >
> >
> > Also, in the guidelines, I've seen references to dot releases 0.x.y up to
> > the 1.0 release but not 'alpha1', 'alpha2', 'RC1', etc. Maybe I missed a
> > page somewhere. Is there a preference/ mandate here?
> >
> >
> > Here are a few examples:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201509.mbox/%3ccalwht94xigohjrtimmqyx3gfodr2m8pyyp2fsnybrq6dat_...@mail.gmail.com%3E
> >
> > http://zookeeper.apache.org/doc/r3.5.1-alpha/
> >
> >
> >
> http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/releasenotes.html
> (search
> > for alpha, beta, etc)
> >
> >
> > If our versioning approach can be vetted, I agree with Anthony's
> > suggestion to have frequent releases scrubbing these issues - at least a
> > few at a time.
> >
> > Roman/ others: any guidance here?
> >
> > Thanks,
> > Nitin
> >
> > 
> > From: John Blum 
> > Sent: Monday, November 30, 2015 11:07 AM
> > To: dev@geode.incubator.apache.org
> > Subject: Re: Review of 1.0.0-alpha1 issues
> >
> > From my perspective, the (nearly) final, "releasable" POM is not needed
> > until we hit RC1.  By then, RCs should be relatively stable, having only
> > simple changes (e.g. bug fixes) until final GA.  Dependency additions,
> > exclusions should be worked out before/by RC1.
> >
> > On Mon, Nov 30, 2015 at 10:55 AM, Anthony Baker 
> wrote:
> >
> > Let’s get really specific when we talk about “release”.  GEODE-27 clearly
> > needs to be addressed prior to a 1.0.0 release.  Currently we are
> > discussing a 1.0.0-alpha1 release which will be followed soon after by
> > *-alpha2, *-alpha3, etc.
> >
> > Do we need GEODE-27 in 1.0.0-alpha1 or can it be deferred to a subsequent
> > alpha release?
> >
> > Anthony
> >
> >
> > On Nov 30, 2015, at 10:24 AM, William Markito 
> >
> > wrote:
> >
> >
> > Huge +1 for GEODE-27 before release.
> >
> > On Mon, Nov 30, 2015 at 9:13 AM, John Blum  wrote:
> >
> > Looking ahead, 1 issue, among others, that should warrant careful
> >
> > attention
> >
> > before the 1.0.0 RC, is GEODE-27
> >  [0].  Getting the POM
> > file
> > correct is not only important for Geode, but for developers building
> > applications using Geode.  Changing a POM file after a release is always
> > messy and not recommended since most artifact servers (e.g.
> >
> > Artifactory),
> >
> > and even local Maven repos, ca

Re: Final updates to Geode Website - Ready ?

[Michael Stolz]

+1

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 25, 2015 1:31 AM, "William Markito"  wrote:

> I've just pushed the latest fixes and added content to the community page
> as well.
>
> Please check the result here -> http://markito.github.io/geode-website/
> (clear your cache)
>
> From my point of view and based on the feedback I got so far, this is
> *ready* to be pushed to asf-site and update our current website.
>
> Please let me know if you think otherwise, have any comments or feedback.
>
> Thanks,
> --
>
> William Markito Oliveira
> -- For questions about Apache Geode, please write to
> *dev@geode.incubator.apache.org
> *
>

Re: feature/GEODE-77 ready to merge to develop

[Michael Stolz]

Super!

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771
On Nov 20, 2015 12:03 PM, "Bruce Schuchardt"  wrote:

> We are feature-complete and stable on feature/GEODE-77.  We've also staged
> a merge to develop and tested the result.
>
> I'd like to push the merge today.
>

Re: [GitHub] incubator-geode pull request: [GEODE-252] Remove deprecated Partit...

[Michael Stolz]

We have to be very careful about how much backward compatibility we break.
Bad enough we have to do a full cluster restart, but if we also lose all
the data because of serialization changes that's a problem.
When we make a breaking protocol change the mechanism for releasing the new
code is
1. Ensure that there is a working back-up with WAN Gateway set up to feed
the system being upgraded.
2. Bring down the system to be upgraded
3. Deploy the upgrade
4. Restart the upgraded system
5. Once the system has completed GII, start the Gateway receivers so that
it can catch up on anything it missed while it was down.

This mechanism has to be kept intact.

This would also break the clients for much the same reason, they would all
have to be upgraded at the same time as the servers. That has NEVER been
possible.


--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

On Wed, Nov 11, 2015 at 5:11 PM, upthewaterspout  wrote:

> Github user upthewaterspout commented on the pull request:
>
>
> https://github.com/apache/incubator-geode/pull/29#issuecomment-155849235
>
> I hit some failures with these changes. I think these related to tests
> that are using these features that probably need to get fixed.
>
>
> com.gemstone.gemfire.cache30.CacheXml45DUnitTest.testPartitionedRegionXML
>
> com.gemstone.gemfire.codeAnalysis.AnalyzeSerializablesJUnitTest.testDataSerializables
>
> com.gemstone.gemfire.internal.cache.PartitionedRegionCacheXMLExampleDUnitTest.testExampleWithSubRegion
>
> com.gemstone.gemfire.internal.cache.PartitionedRegionCreationDUnitTest.testTotalNumberOfBuckets
>
> The AnalyzeSerializables is failing because it detected a change to a
> serializable class. I think since we are breaking backwards compatibility
> with older versions of gemfire that's fine, it probably just needs an
> update to sanctionedSerializables.
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>

transients in PDX

[Michael Stolz]

Does PDX ignore transient fields like Java Serialization does?

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: 631-835-4771

Re: Bump language level to 1.8

[Michael Stolz]

I like the idea.
+1

--
Mike Stolz
GemFire Principal Product Manager
Mobile: 631-835-4771
On Oct 23, 2015 11:27 AM, "William Markito"  wrote:

> Folks,
>
> We had some discussions around this in the past but I guess we have not yet
> made the decision to move forward and change the language level.
>
> Just opened GEODE-479 and would like to propose to do this change before
> our Alpha release.
>
> Any thoughts ?
> --
> William Markito Oliveira
> Enterprise Architect
> -- For questions about Apache Geode, please write to
> *dev@geode.incubator.apache.org
> *
>

Re: AsyncEventListener - How to detect duplicate events

[Michael Stolz]

To expand on this, the only way to be sure which of the possible duplicates
are actual duplicates would be to look in the backing store you are writing
behind into to see if the event was already written.

Since most of the events in GemFire are atomic in nature, it usually
doesn't matter if you write the same event to the backing store a second
time. However, if you are doing some sort of processing in the
AsyncEventListener you may need to know not to re-process and then you
would have to rely on the backing store to tell you if this is a duplicate
event that has already been processed.

--
Mike Stolz
GemFire Principal Product Manager
Mobile: 631-835-4771

On Wed, Sep 23, 2015 at 1:21 PM, Barry Oglesby  wrote:

> You shouldn't be seeing duplicate events in the normal case. Can you share
> your entire configuration?
>
> In the normal case where all members are stable, each event is sent to the
> primary member for that event. From there it is replicated to the secondary
> member(s). Each member delivers the event to its local AsyncEventListener
> which will be primary in the primary member and secondary in the secondary
> member. The only member that should be processing the event in the
> AsyncEventListener is the primary one. This is the normal case.
>
> In the event of primary member failure, the secondary member will take over
> as primary. At this time, the AsyncEventListener might see duplicate events
> which would be ones that were being processed by the primary
> AsyncEventListener right before the primary member failed. These events may
> have been completed in the former primary member, but the notification to
> the former secondary (now primary) member to remove them from its queue
> hadn't been made, so they are still in the secondary member's queue. In
> this case, as soon as the secondary member becomes primary, these events
> will be delivered to its AsyncEventListener as possible duplicates. The
> getPossibleDuplicate method is the way to detect whether these events are
> possible duplicates.
>
>
> Barry Oglesby
> GemFire Advanced Customer Engineering (ACE)
> For immediate support please contact Pivotal Support at
> http://support.pivotal.io/
>
>
> On Wed, Sep 23, 2015 at 1:09 AM, Nilkanth Patel  >
> wrote:
>
> > Hello,
> >
> > *i am writing a **AsyncEventListener implementation as following where
> > i am getting duplicate events (same event multiple times). How can i
> > detect whether a particular event is duplicate or not?*
> >
> >
> > public boolean processEvents(List events) {  for
> > (AsyncEvent asyncEvent : events) {
> > GatewaySenderEventImpl event = (GatewaySenderEventImpl) (asyncEvent);
> > final Operation op = event.getOperation();if
> > (!event.getPossibleDuplicate()) {
> >   if (op == Operation.CREATE) {
> > //create event
> >   } else if (op == Operation.UPDATE) {
> > //update event
> >   } else if (op == Operation.DESTROY) {
> > //destroy event
> >   } else {
> > /*other event*/ }
> > } else {
> > //duplicate event
> > }  }}
> >
> >
> > Thanks in advance.
> >
> > Nilkanth.
> >
>

Re: Apache Geode is....

[Michael Stolz]

I think we historically went down the "in - memory database" route before
the term IMDG became popular.

IMDG semantics fit Gemfire better than database semantics do.

--
Mike Stolz
GemFire Principal Product Manager
Mobile: 631-835-4771
On Sep 17, 2015 9:37 PM, "Greg Chase"  wrote:

> On Thu, Sep 17, 2015 at 8:42 PM, Anthony Baker  wrote:
>
> > What is more than a data grid but less than a database?
> >
>
> Positioning from both are valid. A lot of technologies position off
> "database": "Object database", "document database", "in-memory database",
> "NoSQL Database"
>
> The question is whether IMDGs are taking on more and more "database like"
> capabilities and thus can earn that position. Or are there things that
> fundamentally IDMGs are unsuitable for, and this same use case would be
> unsuitable for Geode.
>
> I originally wrote IMDG for the short description of Geode since its a
> great explanation to explain the highly distributed share-nothing
> architecture. At Pivotal we market the technology as an "in-memory
> database".
>
>
> > I just had a conversation on the sidelines of a soccer game tonight
> > explaining that no we don’t have database semantics but we do more than a
> > basic IMDG.  They had done an exploration previously and had picked up
> > hazelcast.
> >
>
> What exactly is meant by "database semantics"?  When I usually hear people
> say that, they really mean RDBMS - tables, and SQL.
>
> I know certain analysts who believe IMDGs are fundamentally NOT databases
> because they don't do SQL.
>
> -Greg
>

Re: Apache Geode is....

[Michael Stolz]

Hard to keep it down to so few words. I'd love to have "parallel,
persistent, guaranteed eventing system" but its too many words.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

On Tue, Sep 15, 2015 at 11:14 AM, Gregory Chase  wrote:

> Please recommend some different language to "event message queue".
> "Eventing system" lacks some detail and highlighting some wonderful
> capabilities of Geode in persistence, guaranteed delivery, and parallel
> operation. "Eventing system" does imply callbacks and hooks like what Geode
> has. However I'm hoping "Event message queue" implies that as well since
> this isn't just something like "RabbitMQ" pasted onto the database nodes :)
>
> -Greg
>
> On Tue, Sep 15, 2015 at 11:10 AM, Michael Stolz  wrote:
>
> > I'm not real happy with the "messaging queue" terminology. I prefer to
> > think of it as an "eventing system". Messaging queues have a lot of
> > connotations associated with them that really aren't the way our eventing
> > mechanism works.
> >
> > --
> > Mike Stolz
> > Principal Technical Account Manager
> > Mobile: 631-835-4771
> >
> > On Tue, Sep 15, 2015 at 10:33 AM, Gregory Chase 
> wrote:
> >
> > > I was trying to cover "high concurrency" in the "extreme scale" bucket,
> > but
> > > I would agree that being precise about scale is better.  Geode deals in
> > > in-memory terabytes, not Petabytes at rest.
> > >
> > > Taking your feedback Ashvin, we can get tighter:
> > >
> > > Apache Geode is a distributed, in-memory data grid and
> > > integrated messaging queue built to support
> > > low latency transactional applications
> > > at very high concurrency.
> > >
> > > On Tue, Sep 15, 2015 at 10:30 AM, Ashvin A  wrote:
> > >
> > > > Hi Greg,
> > > >
> > > > I pretty much like the short description you provided. I would like
> to
> > > add
> > > > "high concurrency" in the mix for completeness. Which would make it
> > > (22+2)
> > > > < 25 words.
> > > >
> > > > Apache Geode is a distributed, in-memory data grid and
> > > > integrated messaging queue built to support
> > > > low latency *high concurrency* transactional applications
> > > > at extreme scale
> > > >
> > > > $ 0.01
> > > >
> > > > -ashvin
> > > >
> > > > On Mon, Sep 14, 2015 at 4:44 PM, Greg Chase 
> > wrote:
> > > >
> > > > > Dear Geode contributors,
> > > > > I'm writing the website for the newly accepted Apache HAWQ
> incubating
> > > > > project.
> > > > >
> > > > > As part of this effort, I secured some web design resources to
> > refresh
> > > > the
> > > > > Apache Geode site as well.
> > > > >
> > > > > Talking to a number of contributors individually, they like the
> idea
> > of
> > > > > changing the design somewhat to fit the CouchDB layout:
> > > > > http://couchdb.apache.org/, but keeping the 'Geode' graphics and
> > look
> > > > and
> > > > > feel.
> > > > >
> > > > > Its also a good time to refresh our general text which hasn't
> really
> > > been
> > > > > updated since before Geode joined Apache.
> > > > >
> > > > > First panel is a short description explaining what Geode is:
> > > > >
> > > > > "Apache Geode is a distributed, in-memory data grid and integrated
> > > > > messaging queue built to support low latency transactional
> > applications
> > > > at
> > > > > extreme scale."
> > > > >
> > > > > Should be 25 words or less. 20 words or less is even better. My
> score
> > > is
> > > > > "22".
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Greg Chase
> > >
> > > Director of Big Data Communities
> > > http://www.pivotal.io/big-data
> > >
> > > Pivotal Software
> > > http://www.pivotal.io/
> > >
> > > 650-215-0477
> > > @GregChase
> > > Blog: http://geekmarketing.biz/
> > >
> >
>
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>

Re: Apache Geode is....

[Michael Stolz]

I'm not real happy with the "messaging queue" terminology. I prefer to
think of it as an "eventing system". Messaging queues have a lot of
connotations associated with them that really aren't the way our eventing
mechanism works.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

On Tue, Sep 15, 2015 at 10:33 AM, Gregory Chase  wrote:

> I was trying to cover "high concurrency" in the "extreme scale" bucket, but
> I would agree that being precise about scale is better.  Geode deals in
> in-memory terabytes, not Petabytes at rest.
>
> Taking your feedback Ashvin, we can get tighter:
>
> Apache Geode is a distributed, in-memory data grid and
> integrated messaging queue built to support
> low latency transactional applications
> at very high concurrency.
>
> On Tue, Sep 15, 2015 at 10:30 AM, Ashvin A  wrote:
>
> > Hi Greg,
> >
> > I pretty much like the short description you provided. I would like to
> add
> > "high concurrency" in the mix for completeness. Which would make it
> (22+2)
> > < 25 words.
> >
> > Apache Geode is a distributed, in-memory data grid and
> > integrated messaging queue built to support
> > low latency *high concurrency* transactional applications
> > at extreme scale
> >
> > $ 0.01
> >
> > -ashvin
> >
> > On Mon, Sep 14, 2015 at 4:44 PM, Greg Chase  wrote:
> >
> > > Dear Geode contributors,
> > > I'm writing the website for the newly accepted Apache HAWQ incubating
> > > project.
> > >
> > > As part of this effort, I secured some web design resources to refresh
> > the
> > > Apache Geode site as well.
> > >
> > > Talking to a number of contributors individually, they like the idea of
> > > changing the design somewhat to fit the CouchDB layout:
> > > http://couchdb.apache.org/, but keeping the 'Geode' graphics and look
> > and
> > > feel.
> > >
> > > Its also a good time to refresh our general text which hasn't really
> been
> > > updated since before Geode joined Apache.
> > >
> > > First panel is a short description explaining what Geode is:
> > >
> > > "Apache Geode is a distributed, in-memory data grid and integrated
> > > messaging queue built to support low latency transactional applications
> > at
> > > extreme scale."
> > >
> > > Should be 25 words or less. 20 words or less is even better. My score
> is
> > > "22".
> > >
> >
>
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>

Re: Create Region API

[Michael Stolz]

Wes, we're you looking for cluster configuration management or just
creation of a dynamic region?

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Sep 14, 2015 1:47 PM, "Kirk Lund"  wrote:

> All of those "null" args make my soul hurt!
>
> You could feed a "create region" command string into
> MemberMXBean.processCommand() on the Manager.
>
> So, I think you would need to:
>
> a) target that Manager with your Function and access its MemberMXBean
> directly or
> b) connect to the Manager via remotely via JMX from within your Function in
> another member
>
> If you don't know which member is the Manager, then I think you could use
> the API on com.gemstone.gemfire.management.ManagementService to find it.
>
> I'd be interested in hearing ideas about how to make this easier to use for
> your need.
>
> -Kirk
>
> On Mon, Sep 14, 2015 at 12:22 PM, Swapnil Bawaskar 
> wrote:
>
> > One way to create a Region so that it is recorded in the shared
> > configuration is to use the API that GFSH uses to create the region:
> >
> > CreateAlterDestroyRegionCommands cliCmds = new
> > CreateAlterDestroyRegionCommands();
> > Result result = cliCmds.createRegion(key, RegionShortcut.PARTITION,
> > null, null, true,
> > null, null, null, null, null, null, null, null, null, null, null,
> > null, null,
> > null, null, null, null, null, null, null, null, null, null, null,
> > null, null,
> > null, null, null, null, null, null, null, null);
> >
> >
> > On Mon, Sep 14, 2015 at 10:00 AM, Dan Smith  wrote:
> >
> > > RegionFactory.create will create a partitioned region only on the
> member
> > on
> > > which you invoke create. So you could use FunctionService.onMembers and
> > > call create on all of the existing members.
> > >
> > > But what I don't know is if there is a way to do what the gfsh command
> > does
> > > - add the region to the shared configuration so that new members will
> > also
> > > have the region when they start up. Does anyone know if that is
> possible
> > > through the java API?
> > >
> > > -Dan
> > >
> > >
> > >
> > > On Mon, Sep 14, 2015 at 9:08 AM, Wes Williams 
> > > wrote:
> > >
> > > > Is RegionFactory.create(regionName) the best and threadsafe way to
> > > create a
> > > > partitioned region across the distributed system from within a
> > function?
> > > >
> > > >
> > > > Thanks,
> > > > *Wes Williams | Pivotal Sr. **Data Engineer*
> > > > 781.606.0325
> > > > http://pivotal.io/big-data/pivotal-gemfire
> > > >
> > >
> >
>

Re: PartitionListener

[Michael Stolz]

I believe many custom indexes use that. Spatial, for instance. Maybe even
our Lucene impl?

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Aug 26, 2015 4:24 PM, "Darrel Schneider"  wrote:

> Are there any users or customers using
> com.gemstone.gemfire.cache.partition.PartitionListener?
>
> If you are, please respond (on list or directly to me) with some
> information about your usage.
>
>

Re: Dynamic classloading in Geode

[Michael Stolz]

If this feature makes it into an actual release please make sure this
option is not enabled by default and is securely turned off for
environments where there are strong controls around releasing software into
production.
Also make sure that it is secured in terms of Authentication and
Authorization via the Geode security framework when it is enabled, so that
not just anyone can push code.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

On Thu, Aug 13, 2015 at 1:57 PM, Anthony Baker  wrote:

> Vito and I spent some time hacking up a prototype for dynamic and
> distributed classloading of Geode functions.  Currently a user has to
> compile a function into a jar and deploy it using gfsh before it can be
> executed.  If we could enable automatic deployment of functions across a
> running cluster it would speed up the development cycle for Geode
> applications and pave the way for other interesting features (like Java8
> lambdas).
>
> Here’s how it works:
>
> A function wrapper (DynamicFunction) serializes the original function
> object and captures dependent classes as byte arrays.  We generate an MD5
> hash over the bytecode and use that as the key for storing the bytecode in
> a replicated region (“hackday”) within the cache.  When the function is
> invoked, we call putIfAbsent() to distribute the byte code prior to
> executing the function across the cluster.  During execution, we extend the
> TCCL with a new class loader that loads classes from our region while the
> original function is being deserialized.  The original function is then
> executed in parallel on the cluster members.  This allows an application
> developer to iteratively modify and test function code without any manual
> steps to upload class files.
>
> Obviously, there is a lot more thinking and design work to do around these
> ideas.  Here’s our super-hacky code if you’re interested:
> https://gist.github.com/metatype/9b1f39a24e52f5c6f3e1 <
> https://gist.github.com/metatype/9b1f39a24e52f5c6f3e1>
>
> Caveats:
>
> 1) Currently we only capture static class dependencies.  Any class
> dependencies present during method invocations are ignored.  This could be
> addressed by doing byte code inspection (using ASM, javaassist, etc).
>
> 2) The region we use to cache class byte code should be automatically
> recreated as a metadata region, similar to how we store pdx types.  We also
> need to configure eviction and expiration attributes to control resource
> usage and remove garbage.
>
> 3) We only injected the byte code caching hack into the code path for
> FunctionService.onServers(pool).  Also, the putIfAbsent() call adds another
> network roundtrip.
>
>
> Anthony & Vito

Re: Request to post Geode security article

[Michael Stolz]

I think it need to be "The authorization of operations..."

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Jul 11, 2015 10:56 AM, "Anthony Baker"  wrote:

> Really nice writeup!  One minor suggestion, consider changing
>
> "In terms of authorization operations performed by a client application on
> a cluster can be restricted or completely blocked according to your
> configurations and programmatic authorization callback."
>
> to
>
> “The authorization operations performed by a client application on a
> cluster can be restricted or completely blocked according to your
> configurations and programmatic authorization callback."
>
> Also, is the copyright notice at the end required?  I don’t see that on
> many ASF blog posts.
>
> Anthony
>
> > On Jul 10, 2015, at 6:08 PM, William Markito 
> wrote:
> >
> > +1 for the article
> >
> > +1 for ASF blog
> >
> > On Fri, Jul 10, 2015 at 6:02 PM, gtantachuco . 
> > wrote:
> >
> >> Dev team,
> >> I would like to publish this Geode security article to
> >> https://blogs.apache.org/geode/. Here is the initial draft.
> >> At your convenience, please can you let me know what the next steps
> would
> >> be to get this published?
> >>
> >> Thank you.
> >>
> >> --
> >> Best regards,
> >> -Guillermo
> >>
> >>
> >
> >
> > --
> >
> > William Markito Oliveira
> > Enterprise Architect
> > -- For questions about Apache Geode, please write to
> > *dev@geode.incubator.apache.org
> > *
>
>

Re: another cleanup JIRA

[Michael Stolz]

I have already suggested that we go ahead and do all the re-packaging for
Geode, but that before we build the enterprise version we use a
pre-processing script that automatically re-packages all org.Apache.Geode
to com.Gemstone.Gemfire as part of the build process so that WE eat the
difficulty of the necessary open-source re-packaging rather than hundreds
of customers with millions of lines of code that they would need to change.


--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

On Wed, Jul 1, 2015 at 4:16 PM, Bruce Schuchardt 
wrote:

> Mike, this seems like something to bring up with GemFire product
> management.
>
> As I understand it the main problem for client/server is that there are
> some objects, especially exceptions, that are transmitted via java
> serialization and the package renaming is going to break compatibility.
> For data-serialization we can swizzle package names but that's not possible
> for plain old java serialization.
>
> For peer-to-peer it is also going to be broken because we can't use the
> old 2.2.9 version of JGroups and make it out of incubation. Newer versions
> of JGroups are on-wire incompatible with 2.2.9.
>
>
>
> Le 7/1/2015 12:50 PM, Michael Stolz a écrit :
>
>> This notion of propagating a breaking change to all Enterprise users is
>> going to be extremely traumatic.
>>
>> Customers are already raising their voices about them having to endure a
>> breaking change to THEIR CODE for the sake of the opensource version being
>> made available for free totally  unacceptable.
>>
>> We MUST find a way to keep the Enterprise version backward compatible or
>> we
>> risk a wholesale migration of enterprise customers to other solutions.
>>
>> --
>> Mike Stolz
>> Principal Technical Account Manager
>> Mobile: 631-835-4771
>> On Jul 1, 2015 12:10 PM, "Bruce Schuchardt" 
>> wrote:
>>
>>  Geode-72 <https://issues.apache.org/jira/browse/GEODE-72> calls for
>>> removing rolling upgrade code for old GemFire releases. There's no reason
>>> for Geode code to be cluttered up with rolling upgrade code for old
>>> GemFire
>>> releases since you won't be able to do a rolling upgrade from those
>>> releases to Geode due to package renaming.
>>>
>>>
>

Re: another cleanup JIRA

[Michael Stolz]

This notion of propagating a breaking change to all Enterprise users is
going to be extremely traumatic.

Customers are already raising their voices about them having to endure a
breaking change to THEIR CODE for the sake of the opensource version being
made available for free totally  unacceptable.

We MUST find a way to keep the Enterprise version backward compatible or we
risk a wholesale migration of enterprise customers to other solutions.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Jul 1, 2015 12:10 PM, "Bruce Schuchardt"  wrote:

> Geode-72  calls for
> removing rolling upgrade code for old GemFire releases. There's no reason
> for Geode code to be cluttered up with rolling upgrade code for old GemFire
> releases since you won't be able to do a rolling upgrade from those
> releases to Geode due to package renaming.
>

Native client connections

[Michael Stolz]

I have always wondered how exactly the connections between the clients and
Cache Servers work under the covers.

I believe I have heard that there are 2 connections minimum between a
CacheServer and a client, one for get/put, etc request/reply type traffic
and a separate one for subscription type traffic.

If that is true my next question is, do both of those connections get
initiated from the client side?

Last question, are the hostnames given to the client by the locator in the
form of 192.168.1.1 TCP/IP numerical addresses or are they host names? In
the particular case of a big compute grid with thousands of nodes it would
be better if we gave them our numerical addresses to avoid 1000
simultaneous DNS lookups. Can that be arranged?

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

Re: [VOTE] Grandfathering forgotten Geode contributors

[Michael Stolz]

Actually, I'd like to be part of this group as well. Seeing as how I was
the very first customer to deploy Gemfire in production.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Jun 9, 2015 8:18 PM, "Roman Shaposhnik"  wrote:

> Hi!
>
> for various reasons we missed a few Pivotal folks
> when submitting a Geode proposal. There's nothing
> controversial about them -- just an honest mistake.
> They are all currently working on the project and
> contributed quite a bit in the past. In short, they have
> as much a claim in being a project committer as
> all the other folks who were on the proposal.
>
> I propose that we add them to the project as committers
> since they should've been on the proposal to begin with:
> * Chloe Jackson
> * Manuel David
> * Rajesh Kumar
> * Rishitesh Mishra
> * Shankar Hundekar
>
> This majority vote is open for at least 72 hours and lazy consensus
> applies.
>
> Here's my +1.
>
> Thanks,
> Roman.
>

Re: [DISCUSS] Draft of June report for Geode

[Michael Stolz]

Set up a Google calendar for geode-calendar. Everybody can access that.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On Jun 2, 2015 8:36 PM, "William Markito"  wrote:

> Thanks for those!
>
> We are in a desperate need to have a calendar with all up coming talks
> around Geode in a central and manageable place so we can reference from our
> website and in the wiki.   Thoughts ?
>
> On Tue, Jun 2, 2015 at 3:20 PM, John Blum  wrote:
>
> > To add to the "Upcoming" section...
> >
> > There will also be a meetup in Toronto
> >  [0]
> > with Luke Shannon and I presenting on *Apache Geode*...
> >
> > Luke and I will be presenting again at *SpringOne2GX-2015*...
> >
> > Building Highly Scalable Spring Applications with In-Memory Distributed
> > Data Grids
> > <
> >
> https://2015.event.springone2gx.com/schedule/sessions/building_highly_scalable_spring_applications_with_in_memory_distributed_data_grids.html
> > >
> >  [1]
> >
> > Finally, I submitted a BOF down at *JavaOne2015*...
> >
> > *Building Highly Scalable Spring Applications with In-Memory, Distributed
> > Data Grids*
> >
> >
> > [0] - http://www.meetup.com/Toronto-Pivotal-User-Group/events/39293/
> > [1] -
> >
> >
> https://2015.event.springone2gx.com/schedule/sessions/building_highly_scalable_spring_applications_with_in_memory_distributed_data_grids.html
> >
> >
> > On Tue, Jun 2, 2015 at 3:04 PM, Anthony Baker  wrote:
> >
> > > There was another meetup on 4/23 in SF:
> > >
> > > http://www.meetup.com/Pivotal-Open-Source-Hub/events/221443735
> > >
> > > Anthony
> > >
> > >
> > > > On Jun 2, 2015, at 12:56 PM, William Markito 
> > > wrote:
> > > >
> > > > ---
> > > > Geode has been incubating since 2015-04-27.
> > > >
> > > > Three most important issues to address in the move towards
> graduation:
> > > >
> > > >  1. Expanding the community to include contributors and committers
> > > outside
> > > > of Pivotal
> > > >  2. Execute and manage the project according to governance model
> > required
> > > > by the "Apache Way"
> > > >  3. Have our first Apache (incubating) release (currently blocked on
> us
> > > > solving JGroups licensing issues)
> > > >
> > > > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> > > > aware of?
> > > >
> > > > None.
> > > >
> > > > How has the community developed since the last report?
> > > >
> > > > The project had around 350 messages and around a 100 different people
> > > > joined the discussions on the mailing lists, covering all sorts of
> > > topics,
> > > > from development model to new features.
> > > >
> > > > The wiki has been kept up to date based on mailing list discussions,
> > with
> > > > processes for:
> > > >  - Code Review and submissions
> > > >  - Branching and versioning
> > > >  - Code of Conduct
> > > >  - Social Media guidelines
> > > >
> > > > There are 46 open issues on JIRA, 5 resolved.
> > > >
> > > > Several external pull requests. Review process is in place and has
> been
> > > > done using ASF ReviewBoard.
> > > >
> > > > Community events and conferences:
> > > >  - ApacheCon Austin 2015
> > > >- Implementing a Highly-Scalable Stock Prediction System with R,
> > Geode
> > > > and Spring XD - Fred Melo, Pivotal and William Markito, Pivotal
> (April
> > > 15)
> > > >- Unleashing the Silicon Forest Fire - the Open Sourcing of
> GemFire
> > -
> > > > Brian Dunlap, Southwest Airlines; Sudhir Menon; Pivotal; Jags
> > Ramnarayan,
> > > > Pivotal; Dan Smith, Pivotal (Wednesday, April 15)
> > > >  - PJUG - Portland JUG (25 attended)
> > > >- Unleashing the Silicon Forest Fire: the open sourcing of GemFire
> > > >  - Hands-on Virtual Meetup for Apache Geode (incubating) #1 (Tuesday,
> > > June
> > > > 2, 2015 9:00 am)
> > > >  - IMCSummit, 6/30
> > > >- Implementing a Highly Scalable In-Memory Stock Prediction System
> > > with
> > > > Apache Geode, R and Spring XD  (Tuesday, June 30 2015  1:50 PM – 2:30
> > PM)
> > > >  - SpringOne, 9/14 - 9/17
> > > >  Implementing a highly scalable Stock prediction system with R,
> > Geode
> > > > and Spring XD (this will actually be Geode)
> > > >
> > > > How has the project developed since the last report?
> > > >
> > > > - The main command line interface will be renamed from GFSH (GemFire
> > > > Shell) to GEODE, as per community voting.
> > > > - Source code packages will also be renamed from com.gemstone to
> > > > org.apache.geode in order to allow an easier path to become a TLP.
> > > > - Pivotal is going to deliver the final code drop adding important
> > > > features to the project such as:
> > > >   - Off Heap support
> > > >   - HDFS Integration
> > > >   - Command line utilities
> > > >   - JVSD - Statistic visualization tool
> > > > - Jenkins nightly builds are setup
> > > > - ASF Infrastructure setup:
> > > >   - Wiki
> > > >   - JIRA
> > > >   - Jenkins
>

Re: Geode Cluster Sizing Online Feature

[Michael Stolz]

The linked article (Sizing+a+Geode+Cluster) refers to the sizing
spreadsheet several times, but doesn't point to where it can be found.

Here is a copy of the one I use. Please find an appropriate place to post
it.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771

On Tue, May 26, 2015 at 5:07 PM, William Markito 
wrote:

> Hi Wes, feel free to create a JIRA for that if you would like but please
> note that sizing should take into consideration the specifics of the
> application which may not be easy captured in such estimating math
> efforts...
>
> Some more information is already provided in our wiki at
> https://cwiki.apache.org/confluence/display/GEODE/Sizing+a+Geode+Cluster
>
>
>
> On Tue, May 26, 2015 at 1:17 PM, Wes Williams 
> wrote:
>
> > I think another useful feature to access from the Geode web site is a
> > system sizing spreadsheet. You plug in object size, # records, key size,
> #
> > indexes, whether you have stats enabled, etc., etc. and it gives you the
> > recommended # cache servers, cpu's.
> >
> > Where is the request backlog again?
> >
> > Thanks,
> >
> > *Wes Williams | Pivotal Sr. **Data Engineer*
> > 781.606.0325
> > http://pivotal.io/big-data/pivotal-gemfire
> >
>
>
>
> --
>
> William Markito Oliveira
>
> -- For questions about Apache Geode, please write to
> *dev@geode.incubator.apache.org
> *
>


System Sizing Worksheet.xlsx
Description: MS-Excel 2007 spreadsheet

Re: Spec reviews

[Michael Stolz]

I like it. Lets try that

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On May 22, 2015 5:47 PM, "Roman Shaposhnik"  wrote:

> Awesome idea! In fact, would there be any appetite for
> doing weekly hangouts for the community? We can have
> a rolling agenda for them that would start with spec
> presentations/discussions.
>
> Thoughts?
>
> Thanks,
> Roman.
>
> On Fri, May 22, 2015 at 2:44 PM, William Markito 
> wrote:
> > Folks,
> >
> >What do you think about doing some spec reviews through Google
> hangout ?
> >
> >There are some JIRAs open that will have some specs attached but it
> > would also be nice if we had some discussions/presentations about the
> > proposed specs.
> >
> >The idea would be to announce the hangouts here on the dev list and
> > share the notes after the meeting on the list as well.  That's not saying
> > that discussions won't happen on the list, but just that some specs could
> > be better discussed in a video conference.
> >
> > Thanks,
> >
> > --
> > William Markito Oliveira
> >
> > -- For questions about Apache Geode, please write to
> > *dev@geode.incubator.apache.org
> > *
>

Re: What to call Apache Geode's command-line shell tool...

[Michael Stolz]

I think gosh stands for Geode functional shell. Or some other friendly
word. We're  going to drive our existing  customers crazy with all this
package changing and script changing. Citigroup has 53 Gem fire apps
comprising many thousands of lines of code. We're trying to force them to
change all of that. Bad idea.

--
Mike Stolz
Principal Technical Account Manager
Mobile: 631-835-4771
On May 20, 2015 7:53 PM, "Gregory Chase"  wrote:

> How about change it to "G-Shell", assuming the tool versions are same for
> both Geode and GemFire...
>
> On Wed, May 20, 2015 at 4:41 PM, John Blum  wrote:
>
> > This came up in last night's PJUG meeting (** Unleashing the Silicon
> Forest
> > Fire: the open sourcing of GemFire **) here in PDX.
> >
> > In Pivotal GemFire, the tool is called Gfsh (pronounced "G-fish", meaning
> > GemFire Shell).
> >
> > Perhaps, we can call the Apache Geode "Shell" tool... "GOSH".
> >
> > I would say "geosh", but that is 5 letters and sounds related to
> geography,
> > compared to "gfsh" being 4 letters, therefore... "gosh"!
> >
> > ;-)
> >
> > --
> > -John
> > 503-504-8657
> > john.blum10101 (skype)
> >
>
>
>
> --
> Greg Chase
>
> Director of Product Marketing | Big Data
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>