Re: [DISCUSS]: Commit Message Format too Short?
Hello Alberto, It might help to add the reminder to the PR template but, honestly, I don't think many people is actually paying much attention to that... in fact I recall another email thread from some time ago discussing getting rid of the template altogether :-/. Cheers. On Tue, Oct 8, 2019 at 11:58 AM Alberto Bustamante Reyes wrote: > I think its a good idea to have an automatic mechanism to reject commits > that exceed a given limit. > In the previous project I was assigned we used Gerrit instead of Github, > and we had an automatic check to vote -1 if your commit message exceeded > the limit. > > Anyway, while this is decided, a quick action could be to add a new line > to the PR template, at least to remember it: > > - [ ] Is your commit message length below the limit of 50 characters? > > > > > > ____ > De: Juan José Ramos > Enviado: martes, 8 de octubre de 2019 11:32 > Para: dev@geode.apache.org > Asunto: Re: [DISCUSS]: Commit Message Format too Short? > > Hello Owen, > > Yes, I fully agree with you. And just to be clear, I wasn't trying to > discourage descriptive commit messages, on the contrary, we certainly must > encourage them at all cost!!. It was decided that we should, however, try > to keep consistency across all commits and make the subject brief, adding > the full details within the body of the text; as described in *How to write > a Git commit message [1], *referenced in our *Commit Message Format > [2] *article. > Right now we're not enforcing this rule, there are even some commits > without the ticket number at the beginning of the commit subject :-/. > I guess the goal of this thread is to gather some feedback and opinions > from the community to better decide how to proceed: remove the rule, > increase the maximum amount of characters from 50 to something else in the > commit message subject, automatically enforce the rule altogether and > prevent commits that don't follow it, etc. > Best regards. > > [1]: https://chris.beams.io/posts/git-commit/ > [2]: > https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format > > On Tue, Oct 8, 2019 at 10:07 AM Owen Nichols wrote: > > > I don’t care how long it is, but knowing that many tools show only the > > first bit, it’s helpful if the message is phrased with the most important > > words near the beginning. > > > > I’d much prefer to encourage rather than discourage descriptive commit > > messages. Even better if all commit messages mentioned more about _why_ > the > > change is being made, not just describe the diff. > > > > But most important of all, NEVER forget the colon between the ticket > number > > and the rest. I learned that the hard way :( > > > > -Owen > > > > On Tue, Oct 8, 2019 at 1:52 AM Ju@N wrote: > > > > > Hello devs, > > > > > > I've notice that, lately, not everybody is following the guidelines we > > have > > > highlighted in our Wiki under *Commit Message Format [1]*, specially > the > > > first requirement: *GEODE-nn: Capitalized, 50 chars or less summary. > *As > > an > > > example, out of the last 33 commits in develop, only 11 follow the 50 > > chars > > > max rule. > > > Even though I've always followed this "rule", I often find it hard to > > > provide a summary of the commit in less than 50 chars, that's probably > > the > > > reason why other people are just ignoring this part of the guidelines?. > > > Should we increase the maximum amount of characters from 50 to > something > > > else?, should we add a hard check in order to automatically enforce the > > > rule?, should we delete the rule altogether?, thoughts?. > > > Best regards. > > > > > > [1]: > > > > https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format > > > > > > -- > > > Ju@N > > > > > > > > -- > Juan José Ramos Cassella > Senior Software Engineer > Email: jra...@pivotal.io > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS]: Commit Message Format too Short?
Yes, the 50 chars cap rule is *only for the subject of the commit message*. The actual body of the commit message should contain all the details, with a maximum of 72 characters per line. Example: *GEODE-XXX: 50 chars or less summary - [Commit Message Subject]Blank lineMore details about the commit, the why, the changes, capped to 72 - [Commit Message Body - Line1]chars per line. - [Commit Message Body - Line2]* *More details. - [Commit Message Body - Line3]* *...* *Yet more details. - [Commit Message Body - LineN]* Cheers. On Tue, Oct 8, 2019 at 3:37 PM Ernest Burghardt wrote: > Isn't this only regarding the "headline" commit message, but there can be > sub-bulletted text further describing the commit in greater detail...? > This is how I have always interpreted this business... > > EB > > On Tue, Oct 8, 2019 at 3:58 AM Alberto Bustamante Reyes > wrote: > > > I think its a good idea to have an automatic mechanism to reject commits > > that exceed a given limit. > > In the previous project I was assigned we used Gerrit instead of Github, > > and we had an automatic check to vote -1 if your commit message exceeded > > the limit. > > > > Anyway, while this is decided, a quick action could be to add a new line > > to the PR template, at least to remember it: > > > > - [ ] Is your commit message length below the limit of 50 characters? > > > > > > > > > > > > > > De: Juan José Ramos > > Enviado: martes, 8 de octubre de 2019 11:32 > > Para: dev@geode.apache.org > > Asunto: Re: [DISCUSS]: Commit Message Format too Short? > > > > Hello Owen, > > > > Yes, I fully agree with you. And just to be clear, I wasn't trying to > > discourage descriptive commit messages, on the contrary, we certainly > must > > encourage them at all cost!!. It was decided that we should, however, try > > to keep consistency across all commits and make the subject brief, adding > > the full details within the body of the text; as described in *How to > write > > a Git commit message [1], *referenced in our *Commit Message Format > > [2] *article. > > Right now we're not enforcing this rule, there are even some commits > > without the ticket number at the beginning of the commit subject :-/. > > I guess the goal of this thread is to gather some feedback and opinions > > from the community to better decide how to proceed: remove the rule, > > increase the maximum amount of characters from 50 to something else in > the > > commit message subject, automatically enforce the rule altogether and > > prevent commits that don't follow it, etc. > > Best regards. > > > > [1]: https://chris.beams.io/posts/git-commit/ > > [2]: > > https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format > > > > On Tue, Oct 8, 2019 at 10:07 AM Owen Nichols > wrote: > > > > > I don’t care how long it is, but knowing that many tools show only the > > > first bit, it’s helpful if the message is phrased with the most > important > > > words near the beginning. > > > > > > I’d much prefer to encourage rather than discourage descriptive commit > > > messages. Even better if all commit messages mentioned more about _why_ > > the > > > change is being made, not just describe the diff. > > > > > > But most important of all, NEVER forget the colon between the ticket > > number > > > and the rest. I learned that the hard way :( > > > > > > -Owen > > > > > > On Tue, Oct 8, 2019 at 1:52 AM Ju@N wrote: > > > > > > > Hello devs, > > > > > > > > I've notice that, lately, not everybody is following the guidelines > we > > > have > > > > highlighted in our Wiki under *Commit Message Format [1]*, specially > > the > > > > first requirement: *GEODE-nn: Capitalized, 50 chars or less summary. > > *As > > > an > > > > example, out of the last 33 commits in develop, only 11 follow the 50 > > > chars > > > > max rule. > > > > Even though I've always followed this "rule", I often find it hard to > > > > provide a summary of the commit in less than 50 chars, that's > probably > > > the > > > > reason why other people are just ignoring this part of the > guidelines?. > > > > Should we increase the maximum amount of characters from 50 to > > something > > > > else?, should we add a hard check in order to automatically enforce > the > > > > rule?, should we delete the rule altogether?, thoughts?. > > > > Best regards. > > > > > > > > [1]: > > > > > > https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format > > > > > > > > -- > > > > Ju@N > > > > > > > > > > > > > -- > > Juan José Ramos Cassella > > Senior Software Engineer > > Email: jra...@pivotal.io > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS]: Commit Message Format too Short?
Hello Owen, Yes, I fully agree with you. And just to be clear, I wasn't trying to discourage descriptive commit messages, on the contrary, we certainly must encourage them at all cost!!. It was decided that we should, however, try to keep consistency across all commits and make the subject brief, adding the full details within the body of the text; as described in *How to write a Git commit message [1], *referenced in our *Commit Message Format [2] *article. Right now we're not enforcing this rule, there are even some commits without the ticket number at the beginning of the commit subject :-/. I guess the goal of this thread is to gather some feedback and opinions from the community to better decide how to proceed: remove the rule, increase the maximum amount of characters from 50 to something else in the commit message subject, automatically enforce the rule altogether and prevent commits that don't follow it, etc. Best regards. [1]: https://chris.beams.io/posts/git-commit/ [2]: https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format On Tue, Oct 8, 2019 at 10:07 AM Owen Nichols wrote: > I don’t care how long it is, but knowing that many tools show only the > first bit, it’s helpful if the message is phrased with the most important > words near the beginning. > > I’d much prefer to encourage rather than discourage descriptive commit > messages. Even better if all commit messages mentioned more about _why_ the > change is being made, not just describe the diff. > > But most important of all, NEVER forget the colon between the ticket number > and the rest. I learned that the hard way :( > > -Owen > > On Tue, Oct 8, 2019 at 1:52 AM Ju@N wrote: > > > Hello devs, > > > > I've notice that, lately, not everybody is following the guidelines we > have > > highlighted in our Wiki under *Commit Message Format [1]*, specially the > > first requirement: *GEODE-nn: Capitalized, 50 chars or less summary. *As > an > > example, out of the last 33 commits in develop, only 11 follow the 50 > chars > > max rule. > > Even though I've always followed this "rule", I often find it hard to > > provide a summary of the commit in less than 50 chars, that's probably > the > > reason why other people are just ignoring this part of the guidelines?. > > Should we increase the maximum amount of characters from 50 to something > > else?, should we add a hard check in order to automatically enforce the > > rule?, should we delete the rule altogether?, thoughts?. > > Best regards. > > > > [1]: > > https://cwiki.apache.org/confluence/display/GEODE/Commit+Message+Format > > > > -- > > Ju@N > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS] add GEODE-7079 to release/1.9.2
+1 On Fri, Oct 4, 2019 at 6:39 PM Jens Deppe wrote: > On behalf of Juan I'm requesting approval to add GEODE-7079 to > release/1.9.2 > > The original justification is: > > Long story short: GEODE-7079 can be hit by *spring-data-geode* users that > restart a member configured with a persistent asynchronous event queue > (with conflation enabled) without pausing the event processor. The ability > to pause the event processor is what we're mainly adding in 1.9.2, that's > why I believe this fix should also be included. > > Thanks > --Jens > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS] Add GEODE-7261 and GEODE-7241 to release/1.9.2
+1 On Fri, Oct 4, 2019 at 6:44 PM Jens Deppe wrote: > I'd like to propose adding these two fixes to release/1.9.2 > > GEODE-7261 ensures that the Admin REST service can start when Spring Boot > Data Geode is used to launch a server. > > GEODE-7241 publishes our various war artifacts to maven. This ensures that, > in the context of starting a SBDG server, the necessary REST wars can be > made available in order to start the required REST services. > > Thanks > > --Jens > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [Announce] Release branch 1.9.2 created
Hello Jens, Thanks for your detailed email. That said, I'm aware of the fact that the main purpose of 1.9.2 release is to provide fixes that are required for the Spring Data Geode project, that's why I want to include GEODE-7079 :-). Long story short: GEODE-7079 can be hit by *spring-data-geode* users that restart a member configured with a persistent asynchronous event queue (with conflation enabled) without pausing the event processor. The ability to pause the event processor is what we're mainly adding in 1.9.2, that's why I believe this fix should also be included. Best regards. On Fri, Oct 4, 2019 at 5:58 PM Jens Deppe wrote: > Hi Juan, > > The purpose of this release is to provide fixes that are required for the > Spring Data Geode project. Since the current release of SDG (2.2.0.RELEASE) > is based against Geode 1.9.1, it cannot be bumped to any newer (major or > even minor) version that may contain fixes it requires. Thus, for the > current release of SDG, any necessary Geode fixes will need to be made > against 1.9.x. > > With that in mind, unless GEODE-7079 is affecting SDG (or SBDG), I do not > wish to add it. > > Thanks for your understanding > --Jens > > On Fri, Oct 4, 2019 at 7:09 AM Juan José Ramos wrote: > > > Hello Jens, > > > > Can we add *GEODE-7079 [1]* to this release branch as well?. > > The fix basically prevents a NPE during restart of members that have > > configured asynchronous event distribution (either WAN replication of > async > > event listener) & conflation. > > Best regards. > > > > [1]: https://issues.apache.org/jira/browse/GEODE-7079 > > > > On Fri, Oct 4, 2019 at 2:49 PM Robert Houghton > > wrote: > > > > > I'm about to merge the war pr into develop. > > > > > > On Fri, Oct 4, 2019, 06:47 Jens Deppe wrote: > > > > > > > Hello Geode Dev Community, > > > > > > > > We have created a new release branch for Apache Geode 1.9.2 - > > > > "release/1.9.2" > > > > > > > > > > > > This branch currently includes the following commits: > > > > > > > > > > > > GEODE-7124: Fixing tests for older version > > > > GEODE-7204: Add changes to AEQ documentation for > > > > 'pause-event-processing' functionality (#4053) > > > > GEODE-7179: alter async queue command to change state of event > > > > processor during creation (#4057) > > > > GEODE-7128: APIs and GFSH commands to resume AEQ processing. (#4056) > > > > GEODE-7149: Changes to support AsyncEventQueue's dispatcher status > > > > with AsyncEventQueue beans (#4029) > > > > GEODE-7127: Fixing the list AEQ command. > > > > GEODE-7127: Added gfsh arguments for starting AEQ with paused event > > > > processing. > > > > GEODE-7126: Refactoring API names to be more consistent. > > > > GEODE-7129: Adding XML config for creating AEQ with paused event > > > > processing. > > > > GEODE-7126: Added new API to resume AEQ event processing > > > > GEODE-7124: Added new API to create AEQ with paused event processing > > > > GEODE-7261: Include spring-core in the geode-web war artifact > > > > > > > > Please do review and raise any concern with the release branch. > > > > If no concerns are raised, we will start with the voting for the > > > > release candidate soon. > > > > > > > > Regards > > > > --Jens > > > > > > > > > > > > > -- > > Juan José Ramos Cassella > > Senior Software Engineer > > Email: jra...@pivotal.io > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [Announce] Release branch 1.9.2 created
Hello Jens, Can we add *GEODE-7079 [1]* to this release branch as well?. The fix basically prevents a NPE during restart of members that have configured asynchronous event distribution (either WAN replication of async event listener) & conflation. Best regards. [1]: https://issues.apache.org/jira/browse/GEODE-7079 On Fri, Oct 4, 2019 at 2:49 PM Robert Houghton wrote: > I'm about to merge the war pr into develop. > > On Fri, Oct 4, 2019, 06:47 Jens Deppe wrote: > > > Hello Geode Dev Community, > > > > We have created a new release branch for Apache Geode 1.9.2 - > > "release/1.9.2" > > > > > > This branch currently includes the following commits: > > > > > > GEODE-7124: Fixing tests for older version > > GEODE-7204: Add changes to AEQ documentation for > > 'pause-event-processing' functionality (#4053) > > GEODE-7179: alter async queue command to change state of event > > processor during creation (#4057) > > GEODE-7128: APIs and GFSH commands to resume AEQ processing. (#4056) > > GEODE-7149: Changes to support AsyncEventQueue's dispatcher status > > with AsyncEventQueue beans (#4029) > > GEODE-7127: Fixing the list AEQ command. > > GEODE-7127: Added gfsh arguments for starting AEQ with paused event > > processing. > > GEODE-7126: Refactoring API names to be more consistent. > > GEODE-7129: Adding XML config for creating AEQ with paused event > > processing. > > GEODE-7126: Added new API to resume AEQ event processing > > GEODE-7124: Added new API to create AEQ with paused event processing > > GEODE-7261: Include spring-core in the geode-web war artifact > > > > Please do review and raise any concern with the release branch. > > If no concerns are raised, we will start with the voting for the > > release candidate soon. > > > > Regards > > --Jens > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS] - Cutting of release 1.9.2
+1 for creating a 1.9.2 release with the fix. Cheers. On Wed, Sep 25, 2019 at 7:47 PM Udo Kohlmeyer wrote: > It seems there is consensus to cut a 1.9.2 release to include GEODE-7121. > > --Udo > > On 9/20/19 3:34 PM, Jens Deppe wrote: > > +1 for creating a 1.9.2 release with this fix. > > > > On Fri, Sep 20, 2019 at 2:00 PM Udo Kohlmeyer wrote: > > > >> John, thank you! > >> > >> You are correct, not sure what I was thinking.. Geode 1.9.x IS used by > >> SDG 2.2 > >> > >> On 9/20/19 1:11 PM, John Blum wrote: > >>> Correction to *Udo's* comments! > >>> > >>> SDG Moore/2.2 is based on Apache Geode 1.9, NOT SDG Lovelace/2.1 (which > >> is > >>> based on Apache Geode 1.6). > >>> > >>> You can review the version compatibility matrix beginning with SBDG, > >> here ( > >> > https://github.com/spring-projects/spring-boot-data-geode/wiki/Spring-Boot-for-Apache-Geode-and-Pivotal-GemFire-Version-Compatibility-Matrix > >>> ). > >>> > >>> On Fri, Sep 20, 2019 at 1:09 PM John Blum wrote: > >>> > >>>> Hi Kirk - SDG 2.3/Neuman, which is only after SDG 2.2/Moore GAs, which > >> is > >>>> tentatively scheduled for Monday, Sept. 30th. > >>>> > >>>> On Fri, Sep 20, 2019 at 1:01 PM Kirk Lund wrote: > >>>> > >>>>> Hi Udo, SDG cannot upgrade to Geode 1.10.x until which version? SDG > >> 2.2.0? > >>>>> On Fri, Sep 20, 2019 at 12:45 PM Udo Kohlmeyer > wrote: > >>>>> > >>>>>> Hi there Geode Dev's, > >>>>>> > >>>>>> I would like to propose a release for Geode 1.9.x that includes > >>>>>> https://issues.apache.org/jira/browse/GEODE-7121. > >>>>>> > >>>>>> This is an issue that is current in the 1.9.x branch, which is used > by > >>>>>> Spring Data Geode 2.1.10 < > >> https://spring.io/projects/spring-data-geode > >>>>>> . > >>>>>> > >>>>>> This issue can cause an application using Spring Data Geode to > >> bootstrap > >>>>>> GEODE to deadlock upon start up. > >>>>>> > >>>>>> This is critical system's issue and given that Spring Data Geode > >> cannot > >>>>>> change its underlying dependency to 1.10+, would require this fix to > >> be > >>>>>> back-ported to the 1.9 branch. > >>>>>> > >>>>>> --Udo > >>>>>> > >>>>>> > >>>> -- > >>>> -John > >>>> john.blum10101 (skype) > >>>> > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS] GEODE-7241 - make Jar not War?
+1 to Jens and John comments. I'm 100% sure some of our users certainly deploy the *war* file (at least the *PULSE* one) to external web application servers. Cheers. On Wed, Sep 25, 2019 at 5:07 PM Jens Deppe wrote: > I also cannot recall any reason as to the need to *publish* wars. > > However, please do not change the files to .jar. To John's point, despite > the lack of some dependent jars, the structure still conforms to a .war > format. > > --Jens > > On Wed, Sep 25, 2019 at 8:40 AM John Blum wrote: > > > Actually, to clarify 2 points. > > > > 1. Technically, it is a bit more involved than simply just validating the > > "format". For instance, the web.xml file must be valid and well-formed. > > 2. There was a reason why the geode-core and other Apache Geode libs were > > not bundled in WEB-INF/lib of the WAR files, since then it would create > > duplication on the global as well as the WAR file's (isolated) > ClassLoader > > classpath, particularly for the "embedded" Geode Servlet Container case, > > and as such, ClassLoader problems would occur. > > > > > > > > On Wed, Sep 25, 2019 at 8:33 AM Anthony Baker wrote: > > > > > Udo, > > > > > > Can you update GEODE-7241 to help us understand the reason why we need > to > > > publish geode-web* WARs to maven? I get that we used to do this but I > > > can’t recall why we choose that approach. > > > > > > There is one request for Pulse on maven (GEODE-6208). > > > > > > Anthony > > > > > > > > > > On Sep 24, 2019, at 3:44 PM, Udo Kohlmeyer wrote: > > > > > > > > Maybe the better question is, WHY are we publishing geode-web and > > > geode-web-api. > > > > > > > > Pulse, from what I remember, could be a standalone deployment. At > least > > > that is what I remember. > > > > > > > > Most likely an oversight... > > > > > > > > --Udo > > > > > > > > On 9/24/19 3:38 PM, Robert Houghton wrote: > > > >> The geode-pulse module also builds a war, but does not publish it. > Is > > > this > > > >> an oversight, or by design? > > > >> > > > >> On Tue, Sep 24, 2019 at 3:34 PM Robert Houghton < > rhough...@pivotal.io > > > > > > >> wrote: > > > >> > > > >>> I am working on the change to get the geode-web and geode-web-api > war > > > >>> artifacts published instead of the jars. I have found the > > > >>> geode-web-management project is also producing a war artifact, in > > > addition > > > >>> to a jar. Do we want it to be published as well? What is the > > criterion > > > we > > > >>> use to decide? > > > >>> > > > >>> I think this problem was an oversight from the changes > PurelyApplied > > > and I > > > >>> made to the build when we made the publish plugin 'opt-in' instead > of > > > >>> forced by the root project. Easy to publish one or the other, but I > > am > > > not > > > >>> qualified to decide whether a war or jar is more appropriate for > > these > > > >>> modules. > > > >>> > > > >>> Thank you, > > > >>> -Robert > > > >>> > > > > > > > > > > -- > > -John > > john.blum10101 (skype) > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [VOTE] Adding new AEQ feature to release/1.10.0
+1 On Fri, Sep 13, 2019 at 11:59 PM Ryan McMahon wrote: > +1 > > On Fri, Sep 13, 2019 at 3:58 PM Donal Evans wrote: > > > +1 > > > > On Fri, Sep 13, 2019 at 3:37 PM Benjamin Ross wrote: > > > > > +1 > > > > > > On Fri, Sep 13, 2019 at 3:25 PM Anilkumar Gingade > > > > wrote: > > > > > > > +1. This is needed for spring data-geode; whose upcoming release is > > based > > > > on older geode version. > > > > > > > > -Anil. > > > > > > > > > > > > On Fri, Sep 13, 2019 at 3:23 PM Nabarun Nag wrote: > > > > > > > > > Hi Geode Community , > > > > > > > > > > [GEODE-7121] > > > > > > > > > > I would like to include the new feature of creating AEQs with a > > paused > > > > > event processor to the release 1.10 branch. This also includes the > > > > feature > > > > > to resume the AEQ at a later point in time. > > > > > This feature includes addition of new/modified APIs and gfsh > > commands. > > > > > > > > > > [All details about this feature has been discussed in a previous > > > discuss > > > > > thread] > > > > > > > > > > These are the commits that needs to be in release 1.10.0 branch. > > > > > f6e11084daa30791f7bbf9a8187f6d1bc9c4b91a > > > > > 615d3399d24810126a6d57b5163f7afcd06366f7 > > > > > 1440a95e266e671679a623f93865c5e7e683244f > > > > > 42e07dc9054794657acb40c292f3af74b79a1ea6 > > > > > e1f200e2f9e77e986d250fde3848dc004b26a7c2 > > > > > 5f70160fba08a06c7e1fc48c7099e63dd1a0502b > > > > > 0645446ec626bc351a2c881e4df6a4ae2e75fbfc > > > > > 575c6bac115112df1e84455b052566c75764b0be > > > > > 3d9627ff16443f4aa513a67bcc284e68953aff8a > > > > > ea22e72916f8e34455800d347690e483727f9bf5 > > > > > 8d26d595f5fb94ff703116eb91bb747e9ba7f536 > > > > > > > > > > Will create a PR ASAP. > > > > > > > > > > Regards > > > > > Nabarun Nag > > > > > > > > > > > > > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [VOTE] Apache Geode 1.10.0.RC1
+1 On Fri, Aug 30, 2019 at 10:46 PM Dave Barnes wrote: > +1 > Checked the docs: Successfully built AND viewed the Geode User Guide and > the Javadocs. > > On Fri, Aug 30, 2019 at 2:32 PM Jinmei Liao wrote: > > > + 1 verified the management v2 api is available by default and list of > rest > > end points. > > > > On Fri, Aug 30, 2019 at 2:06 PM Dick Cavender > > wrote: > > > > > Hello Geode dev community, > > > > > > This is a release candidate for Apache Geode, version 1.10.0.RC1. > > > Thanks to all the community members for their contributions to this > > > release! > > > > > > Please do a review and give your feedback. The deadline is 3PM PST > > Thursday > > > Sept 5th. > > > Release notes can be found at: > > > > > > > > > https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.10.0 > > > > > > Please note that we are voting upon the source tags: rel/v1.10.0.RC1 > > > > > > Apache Geode: > > > https://github.com/apache/geode/tree/rel/v1.10.0.RC1 > > > Apache Geode examples: > > > https://github.com/apache/geode-examples/tree/rel/v1.10.0.RC1 > > > Apache Geode native: > > > https://github.com/apache/geode-native/tree/rel/v1.10.0.RC1 > > > > > > Source and binary files: > > > https://dist.apache.org/repos/dist/dev/geode/1.10.0.RC1/ > > > > > > Maven staging repo: > > > https://repository.apache.org/content/repositories/orgapachegeode-1058 > > > > > > Geode's KEYS file containing PGP keys we use to sign the release: > > > https://github.com/apache/geode/blob/develop/KEYS > > > > > > PS: Command to run geode-examples: ./gradlew -PgeodeReleaseUrl= > > > https://dist.apache.org/repos/dist/dev/geode/1.10.0.RC1 > > > -PgeodeRepositoryUrl= > > > https://repository.apache.org/content/repositories/orgapachegeode-1058 > > > build runAll > > > > > > Regards > > > Dick Cavender > > > > > > > > > -- > > Cheers > > > > Jinmei > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [VOTE] Apache Geode 1.9.1 RC1
+1 On Thu, Aug 29, 2019 at 2:37 PM Udo Kohlmeyer wrote: > +1 > > On 8/28/19 5:41 PM, John Blum wrote: > > +1 > > > > On Wed, Aug 28, 2019 at 2:51 PM Dan Smith wrote: > > > >> I missed this vote email as well - if we reopen the vote I'll cast one. > I > >> don't really have much context on why we want a 1.9.1 but I'm happy to > >> double check the bits. > >> > >> One comment on this RC - I noticed that we bumped the ordinal in > >> Version.java - is that what we actually want to do? That implies a new > >> version of our communications protocol, which 1.10 will have to > understand. > >> Did we actually change the communication protocol in this release? > >> > >> -Dan > >> > >> On Wed, Aug 28, 2019 at 2:42 PM Kirk Lund wrote: > >> > >>> SBDG 1.2 is currently in RC and cannot be changed to depend on Geode > >> 1.10. > >>> It must depend on Geode 1.9 or 1.9.1. > >>> > >>> So if we want to provide the logging fixes for SBDG 1.2 then we must > >>> release Geode 1.9.1. > >>> > >>> Let's open a new vote for releasing Geode 1.9.1. > >>> > >>> On Wed, Aug 28, 2019 at 1:37 PM Owen Nichols > >> wrote: > >>>> It's past the announced deadline and the vote has failed to due to > lack > >>> of > >>>> quorum. > >>>> > >>>> Voting status > >>>> == > >>>> +1: zero votes > >>>> > >>>> +0: zero votes > >>>> > >>>> -0: zero votes > >>>> > >>>> -1: zero votes > >>>> > >>>> The voting does not meet the requirements < > >>>> https://www.apache.org/foundation/voting.html> of at least 3 PMC > >> members > >>>> with +1 votes and a majority of +1 votes. > >>>> > >>>> The matter of what to do next is referred back to the original DISCUSS > >>>> thread that proposed 1.9.1. > >>>> > >>>> -Owen & Kirk > >>>> > >>>>> On Aug 22, 2019, at 6:10 PM, Owen Nichols > >> wrote: > >>>>> Hello Geode dev community, > >>>>> > >>>>> This is a release candidate for Apache Geode, version 1.9.1.RC1. > >>>>> Thanks to all the community members for their contributions to this > >>>> release! > >>>>> Please do a review and give your feedback. The deadline is 3PM PST > >> Tue, > >>>> August 27 2019. > >>>>> Release notes can be found at: > >> > https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.1 > >>>> < > >>>> > >> > https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.1 > >>>> > >>>>> Please note that we are voting on the source tag: rel/v1.9.1.RC1 > >>>>> > >>>>> Apache Geode: > >>>>> https://github.com/apache/geode/tree/rel/v1.9.1.RC1 < > >>>> https://github.com/apache/geode/tree/rel/v1.9.1.RC1> > >>>>> Apache Geode examples: > >>>>> https://github.com/apache/geode-examples/tree/rel/v1.9.1.RC1 < > >>>> https://github.com/apache/geode-examples/tree/rel/v1.9.1.RC1> > >>>>> Apache Geode native: > >>>>> https://github.com/apache/geode-native/tree/rel/v1.9.1.RC1 < > >>>> https://github.com/apache/geode-native/tree/rel/v1.9.1.RC1> > >>>>> Source and binary files: > >>>>> https://dist.apache.org/repos/dist/dev/geode/1.9.1.RC1/ < > >>>> https://dist.apache.org/repos/dist/dev/geode/1.9.1.RC1/> > >>>>> Maven staging repo: > >>>>> > >> https://repository.apache.org/content/repositories/orgapachegeode-1055 > >>> < > >>>> > https://repository.apache.org/content/repositories/orgapachegeode-1055 > >>>>> Geode's KEYS file containing PGP keys we use to sign the release: > >>>>> https://github.com/apache/geode/blob/develop/KEYS < > >>>> https://github.com/apache/geode/blob/develop/KEYS> > >>>>> PS: Command to run geode-examples: ./gradlew -PgeodeReleaseUrl= > >>>> https://dist.apache.org/repos/dist/dev/geode/1.9.1.RC1 < > >>>> https://dist.apache.org/repos/dist/dev/geode/1.9.1.RC1> > >>>> -PgeodeRepositoryUrl= > >>>> > https://repository.apache.org/content/repositories/orgapachegeode-1055 > >> < > >>>> > https://repository.apache.org/content/repositories/orgapachegeode-1055 > >>>> build runAll > >>>>> Regards > >>>>> Owen Nichols & Kirk Lund > >>>>> > >>>> > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: Propose including GEODE-7085 in 1.10
+1 On Mon, Aug 26, 2019 at 6:36 PM Dan Smith wrote: > Hi, > > I'd like to propose including the fixes for GEODE-7085 into 1.10 (SHA's > below). This is not a new issue, but it does result not being able to > recover from disk without this fix if a cluster has more than 2 billion > updates to a single bucket from a single member. > > SHAs: f587101 , f17931bf > > Thanks, > -Dan > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)
AEQ is parallel, the AEQ will be paused on > >> individual > >> > node where this API is called and the AEQ on other VM's can still > >> dispatch > >> > events. In case the AEQ is not parallel, and the running AEQ on which > >> this > >> > API is invoked is not primary then primary AEQ will still continue > >> > dispatching events." > >> > Performance Impact > >> > > >> > > >> > *This will have similar performance and resource implication as with > the > >> > "GatewaySender.pause()" functionality. If the AEQ is not resumed or > >> kept in > >> > "pause" state for long, it may start consuming the configured memory > and > >> > overflow it into disk and may cause disk full scenario.* > >> > Backwards Compatibility and Upgrade Path > >> > > >> > *Impact with rolling upgrade: * > >> > > >> > *As the api is applicable at individual VM level, there is no message > >> > serialization changes involved. And only applicable to the events > >> getting > >> > dispatched to the listeners on that VM. And the AEQ which are > replicated > >> > (for redundancy) continues to work as before.* > >> > > >> > *Backward compatibility requirements: * > >> > > >> > *None. The AEQs are configured and managed at the server side. There > is > >> no > >> > messaging involved between client/server.* > >> > > >> > *Disk formatting changes:* > >> > > >> > *None.* > >> > > >> > *Deprecation and Application Changes:* > >> > > >> > > >> > *None. If needed, the existing application can be modified to control > >> event > >> > dispatch with AEQ listener.* > >> > Prior Art > >> > > >> > *Without this, the AEQ listeners operating on other regions could > >> > experience missing events or dead lock, if there are co-located > >> regions.* > >> > > >> > *This approach is simple and can take advantage of the existing > >> > functionality that is already supported in GatewaySender on which AEQ > is > >> > based on.* > >> > > >> > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: Proposal to Include GEODE-7079 in 1.10.0
Thanks Owen!!. On Mon, Aug 19, 2019 at 3:44 PM Owen Nichols wrote: > There appears to be consensus that this is a critical fix. > > The following commit has been brought into release/1.10.0 < > https://github.com/apache/geode/tree/release/1.10.0> as the critical fix > for GEODE-7079 <https://issues.apache.org/jira/browse/GEODE-7079>: > > git cherry-pick -x 6f4bbbd96bcecdb82cf7753ce1dae9fa6baebf9b < > https://github.com/apache/geode/commit/6f4bbbd96bcecdb82cf7753ce1dae9fa6baebf9b > > > > GEODE-7079 <https://issues.apache.org/jira/browse/GEODE-7079> has been > changed to 'resolved in' 1.10.0. > > -Owen > > > On Aug 19, 2019, at 2:39 AM, Juan José Ramos wrote: > > > > Hello team, > > > > Just following up on my proposal to include GEODE-7079 in release > 1.10.0, I > > think there's enough consensus at this point (+1: 7, +0: 1), can we > cherry > > pick commit *6f4bbbd [1]* into the release branch?. > > Best regards. > > > > > > [1]: > > > https://github.com/apache/geode/commit/6f4bbbd96bcecdb82cf7753ce1dae9fa6baebf9b > > > > On Thu, Aug 15, 2019 at 9:17 PM Udo Kohlmeyer wrote: > > > >> I'm changing my vote to +1 on this issue. > >> > >> The ONLY reason I'm changing my vote is to add to the cleanliness of the > >> code of the release. I do 100% disagree with the continual scope creep > >> that we have been incurring on this release branch. > >> > >> --Udo > >> > >> On 8/15/19 12:34 PM, Dan Smith wrote: > >>> +1 to merging Juan's fix for GEODE-7079. I've seen systems taken down > by > >>> rapidly filling up the logs in the past, this does seem to be a > critical > >>> fix from the perspective of system stability. > >>> > >>> Also, this is the change, which doesn't seem particularly risky to me. > >>> > >>> - ConflationKey key = new > >>> ConflationKey(gsEvent.getRegion().getFullPath(), > >>> + ConflationKey key = new > ConflationKey(gsEvent.getRegionPath(), > >>> > >>> -Dan > >>> > >>> On Thu, Aug 15, 2019 at 12:23 PM Udo Kohlmeyer wrote: > >>> > >>>> Whilst I agree with "*finish* when we believe the quality of the > release > >>>> branch is sufficient", I disagree that we should have cut a branch and > >>>> continue to patch that branch with non-critical fixes. i.e this issue > >>>> has been around for a while and has no averse side effects. Issues > like > >>>> GEODE-7081, which is new due to a new commit, AND it has critical > >>>> stability implications on the server, that I can agree we should > include > >>>> in a potential release branch. > >>>> > >>>> Otherwise we can ALWAYS argue that said release branch is not of > >>>> "sufficient" quality, especially if there are numerous existing JIRA's > >>>> pertaining to bugs already in the system. > >>>> > >>>> To quote Juan's original email: > >>>> > >>>> /"Note: *no events are lost (even without the fix)* but, if the region > >>>> takes// > >>>> //a while to recover, the logs for the member can grow pretty quickly > >>>> due to// > >>>> //the continuously thrown *NPEs.*"/ > >>>> > >>>> In addition to this, if there is a commit in a cut release branch, > which > >>>> is requiring us to continuously patching the release branch, in order > to > >>>> stabilize that feature/fix, maybe we should consider reverting that > fix > >>>> and release it at a later stage, when it is believed that this fix is > >>>> more stable and have better, more comprehensive test coverage. > >>>> > >>>> So far, GEODE-7081, does not have me convinced that it is critical. OR > >>>> maybe it is the latter of my options, where it is a stabilization > commit > >>>> to a new feature, which begs the question, should we have accepted the > >>>> original feature commit if there are all manner of side effects which > we > >>>> are only discovering. > >>>> > >>>> --Udo > >>>> > >>>> On 8/15/19 11:08 AM, Anthony Baker wrote: > >>>>> While we can’t fix *all known bugs*, I think where we do have a fix > for > >>>> an important issue we should think hard
Re: Proposal to Include GEODE-7079 in 1.10.0
Hello team, Just following up on my proposal to include GEODE-7079 in release 1.10.0, I think there's enough consensus at this point (+1: 7, +0: 1), can we cherry pick commit *6f4bbbd [1]* into the release branch?. Best regards. [1]: https://github.com/apache/geode/commit/6f4bbbd96bcecdb82cf7753ce1dae9fa6baebf9b On Thu, Aug 15, 2019 at 9:17 PM Udo Kohlmeyer wrote: > I'm changing my vote to +1 on this issue. > > The ONLY reason I'm changing my vote is to add to the cleanliness of the > code of the release. I do 100% disagree with the continual scope creep > that we have been incurring on this release branch. > > --Udo > > On 8/15/19 12:34 PM, Dan Smith wrote: > > +1 to merging Juan's fix for GEODE-7079. I've seen systems taken down by > > rapidly filling up the logs in the past, this does seem to be a critical > > fix from the perspective of system stability. > > > > Also, this is the change, which doesn't seem particularly risky to me. > > > > - ConflationKey key = new > > ConflationKey(gsEvent.getRegion().getFullPath(), > > + ConflationKey key = new ConflationKey(gsEvent.getRegionPath(), > > > > -Dan > > > > On Thu, Aug 15, 2019 at 12:23 PM Udo Kohlmeyer wrote: > > > >> Whilst I agree with "*finish* when we believe the quality of the release > >> branch is sufficient", I disagree that we should have cut a branch and > >> continue to patch that branch with non-critical fixes. i.e this issue > >> has been around for a while and has no averse side effects. Issues like > >> GEODE-7081, which is new due to a new commit, AND it has critical > >> stability implications on the server, that I can agree we should include > >> in a potential release branch. > >> > >> Otherwise we can ALWAYS argue that said release branch is not of > >> "sufficient" quality, especially if there are numerous existing JIRA's > >> pertaining to bugs already in the system. > >> > >> To quote Juan's original email: > >> > >> /"Note: *no events are lost (even without the fix)* but, if the region > >> takes// > >> //a while to recover, the logs for the member can grow pretty quickly > >> due to// > >> //the continuously thrown *NPEs.*"/ > >> > >> In addition to this, if there is a commit in a cut release branch, which > >> is requiring us to continuously patching the release branch, in order to > >> stabilize that feature/fix, maybe we should consider reverting that fix > >> and release it at a later stage, when it is believed that this fix is > >> more stable and have better, more comprehensive test coverage. > >> > >> So far, GEODE-7081, does not have me convinced that it is critical. OR > >> maybe it is the latter of my options, where it is a stabilization commit > >> to a new feature, which begs the question, should we have accepted the > >> original feature commit if there are all manner of side effects which we > >> are only discovering. > >> > >> --Udo > >> > >> On 8/15/19 11:08 AM, Anthony Baker wrote: > >>> While we can’t fix *all known bugs*, I think where we do have a fix for > >> an important issue we should think hard about the cost of not including > >> that in a release. > >>> IMO, the fixed time approach to releases means that we *start* the > >> release effort (including stabilization and bug fixing if needed) on a > >> known date and we *finish* when new believe the quality of the release > >> branch is sufficient. Given the number of important fixes being > requested, > >> I’m not sure we are there yet. > >>> I think the release branch concept has merit because it allows us to > >> isolate ongoing work from the changes needed for a release. > >>> +1 for including GEODE-7079. > >>> > >>> Anthony > >>> > >>> > >>>> On Aug 15, 2019, at 10:51 AM, Udo Kohlmeyer > >> wrote: > >>>> Seems everyone is in favor or including a /*non-critical*/ fix to an > >> already cut branch of the a potential release... > >>>> Am I missing something? > >>>> > >>>> Why cut a release at all... just have a perpetual cycle of fixes added > >> to develop and users can chose what nightly snapshot build they would > want > >> to use.. > >>>> I'm voting -1 on a non-critical issue, which is existing and worst > >> effect is to fill logs will NPE logs... (yes, n
Re: Proposal to Include GEODE-7079 in 1.10.0
Hello Udo, Even if it is an existing issue I'd still consider it critical for those cases on which there are unprocessed events on the persistent queue after a restart and the region takes long to recover... you can actually see millions of *NPEs* flooding the member's logs. My two cents anyway, it's up to the community to make the final decision. Cheers. On Thu, Aug 15, 2019 at 5:58 PM Udo Kohlmeyer wrote: > Juan, > > From your explanation, it seems this issue is existing and not > critical. Could we possibly hold this for 1.11? > > --Udo > > On 8/15/19 5:29 AM, Ju@N wrote: > > Hello team, > > > > I'd like to propose including the *fix [1]* for *GEODE-7079 [2]* in > release > > 1.10.0. > > Long story short: a *NullPointerException* can be continuously thrown > > and flood the member's logs if a serial event processor (either > > *async-event-queue* or *gateway-sender*) starts processing events from a > > recovered persistent queue before the actual region to which it was > > attached is fully operational. > > Note: *no events are lost (even without the fix)* but, if the region > takes > > a while to recover, the logs for the member can grow pretty quickly due > to > > the continuously thrown *NPEs.* > > Best regards. > > > > [1]: > > > https://github.com/apache/geode/commit/6f4bbbd96bcecdb82cf7753ce1dae9fa6baebf9b > > [2]: https://issues.apache.org/jira/browse/GEODE-7079 > > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: Fix for NPE during forceDisconnect candidate for 1.10.0
+1 On Tue, Aug 13, 2019 at 6:03 PM Aaron Lindsey wrote: > +1 > > > On Aug 13, 2019, at 9:59 AM, Kirk Lund wrote: > > > > I guess we need at least one more vote to get this fix copied to the > 1.10.0 > > release branch. We just confirmed that this fix is NOT on that branch > yet. > > > > On Thu, Aug 8, 2019 at 11:03 AM Juan José Ramos > wrote: > > > >> +1 > >> > >> On Thu, Aug 8, 2019 at 7:02 PM John Blum wrote: > >> > >>> +1 for Kirk's changes in 1.10. This will be critical for SD Neuman and > >>> SBDG 1.3. > >>> > >>> On Thu, Aug 8, 2019 at 10:57 AM Owen Nichols > >> wrote: > >>> > >>>> Hi Kirk and Mark, thank you for bringing your concern. > >>>> > >>>> Our “critical fixes” rule allows critical fixes to be brought to the > >>>> release branch by proposal on the dev list, as you have just done. If > >>>> there is consensus from the Geode community that this NPE fix > satisfies > >>> the > >>>> “critical fixes” rule, Dick or I will be happy to bring it to the > >> 1.10.0 > >>>> release branch. > >>>> > >>>> -Owen > >>>> > >>>>> On Aug 8, 2019, at 10:54 AM, Kirk Lund wrote: > >>>>> > >>>>> I'd like to propose including the fix for GEODE-6959 in 1.10.0. > >>>>> > >>>>> Spring Boot users are very likely to hit this NPE during > >>> forceDisconnect. > >>>>> > >>>>> If a custom log4j2.xml is used without specifying the Geode > >>>> AlertAppender, > >>>>> GMSMembershipManager may throw a NullPointerException when invoking > >>>>> AlertAppender.getInstance().stopSession() during a forceDisconnect. > >>> This > >>>>> change prevents the NullPointerException allowing forceDisconnect to > >>>> finish. > >>>>> Tests are included with this fix. > >>>>> > >>>>> PR: https://github.com/apache/geode/pull/3899 > >>>>> GEODE-6959: NPE if AlertAppender is not defined > >>>>> > >>>>> Thanks, > >>>>> Kirk and Mark > >>>> > >>>> > >>> > >>> -- > >>> -John > >>> john.blum10101 (skype) > >>> > >> > >> > >> -- > >> Juan José Ramos Cassella > >> Senior Technical Support Engineer > >> Email: jra...@pivotal.io > >> Office#: +353 21 4238611 > >> Mobile#: +353 87 2074066 > >> After Hours Contact#: +1 877 477 2269 > >> Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT > >> How to upload artifacts: > >> https://support.pivotal.io/hc/en-us/articles/204369073 > >> How to escalate a ticket: > >> https://support.pivotal.io/hc/en-us/articles/203809556 > >> > >> [image: support] <https://support.pivotal.io/> [image: twitter] > >> <https://twitter.com/pivotal> [image: linkedin] > >> <https://www.linkedin.com/company/3048967> [image: facebook] > >> <https://www.facebook.com/pivotalsoftware> [image: google plus] > >> <https://plus.google.com/+Pivotal> [image: youtube] > >> < > https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl> > >> > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: Fix for ClassCastException when using Logback for 1.10.0
+1 On Thu, Aug 8, 2019 at 7:26 PM Mark Hanson wrote: > +1 > > I think it is valuable to make life easier for Spring Boot users. > > Thanks, > Mark > > > On Aug 8, 2019, at 11:24 AM, Kirk Lund wrote: > > > > This is the last logging related fix that I'd like to propose adding to > > 1.10.0 > > release branch. > > > > Spring Boot adds Logback and log4j-to-slf4j to the classpath. This > > results in ClassCastExceptions if log4j-core is not excluded. > > > > This change prevents Geode from using Log4jAgent if Log4j Core is > > present but not using Log4jProvider. > > > > For example, Log4j uses SLF4JProvider instead of Log4jProvider when > > log4j-to-slf4j is in the classpath. > > > > By disabling Log4jAgent when other Log4j Providers are in use, this > > prevents problems such as ClassCastExceptions when attempting to cast > > loggers from org.apache.logging.slf4j.SLF4JLogger to > > org.apache.logging.log4j.core.Logger to get the LoggerConfig or > > LoggerContext. > > > > PR: https://github.com/apache/geode/pull/3892 > > GEODE-7050: Use Log4jAgent only if Log4j is using Log4jProvider > > https://issues.apache.org/jira/browse/GEODE-7050 > > > > Thanks, > > Kirk and Aaron > > -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
SSL Alias Support for JMX Connections
I'd like to propose including the fix for *GEODE-7022 [1]* in release 1.10.0. The fix basically improves our own implementation of the *RMIClientSocketFactory* to fully support the GEODE SSL settings, allowing our users to specify a default alias when opening an RMI connection. Best regards. [1]: https://issues.apache.org/jira/browse/GEODE-7022 -- Juan José Ramos Cassella Senior Software Engineer Email: jra...@pivotal.io
Re: Fix for NPE during forceDisconnect candidate for 1.10.0
+1 On Thu, Aug 8, 2019 at 7:02 PM John Blum wrote: > +1 for Kirk's changes in 1.10. This will be critical for SD Neuman and > SBDG 1.3. > > On Thu, Aug 8, 2019 at 10:57 AM Owen Nichols wrote: > > > Hi Kirk and Mark, thank you for bringing your concern. > > > > Our “critical fixes” rule allows critical fixes to be brought to the > > release branch by proposal on the dev list, as you have just done. If > > there is consensus from the Geode community that this NPE fix satisfies > the > > “critical fixes” rule, Dick or I will be happy to bring it to the 1.10.0 > > release branch. > > > > -Owen > > > > > On Aug 8, 2019, at 10:54 AM, Kirk Lund wrote: > > > > > > I'd like to propose including the fix for GEODE-6959 in 1.10.0. > > > > > > Spring Boot users are very likely to hit this NPE during > forceDisconnect. > > > > > > If a custom log4j2.xml is used without specifying the Geode > > AlertAppender, > > > GMSMembershipManager may throw a NullPointerException when invoking > > > AlertAppender.getInstance().stopSession() during a forceDisconnect. > This > > > change prevents the NullPointerException allowing forceDisconnect to > > finish. > > > Tests are included with this fix. > > > > > > PR: https://github.com/apache/geode/pull/3899 > > > GEODE-6959: NPE if AlertAppender is not defined > > > > > > Thanks, > > > Kirk and Mark > > > > > > -- > -John > john.blum10101 (skype) > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Unit tests are hanging?
+1 On Thu, Aug 8, 2019 at 6:55 PM Kirk Lund wrote: > +1 > > On Thu, Aug 8, 2019 at 10:14 AM Dan Smith wrote: > > > > With all that, I propose we permanently bump the timeouts on UnitTestX > > jobs > > > across the board (main pipeline, PR pipeline, etc) from 10 to 20 > minutes > > to > > > be more tolerant of these types of degradations. > > > > > > > +1 > > > > -Dan > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Another change for 1.10.0 release
+1 On Thu, Aug 8, 2019 at 6:41 PM Ryan McMahon wrote: > +1 > > On Thu, Aug 8, 2019 at 10:40 AM John Blum wrote: > > > +1 for Dan's changes. > > > > On Thu, Aug 8, 2019 at 10:28 AM Owen Nichols > wrote: > > > > > Hi Dan, thank you for bringing your concern. > > > > > > Our “critical fixes” rule allows critical fixes to be brought to the > > > release branch by proposal on the dev list [as you have just done]. If > > > there is consensus from the Geode community that this GEODE-7055 fix > > > satisfies the “critical fixes” rule, Dick or I will be happy to bring > it > > to > > > the 1.10.0 release branch. > > > > > > -Owen > > > > > > > > > > On Aug 8, 2019, at 10:11 AM, Dan Smith wrote: > > > > > > > > Hi all, > > > > > > > > I'd like to get the fix for GEODE-7055 (Don't send failure replies > > from a > > > > P2P reader thread) into the 1.10.0 release branch. > > > > > > > > This bug was causing a hang on startup for users of the session > > > replication > > > > module that didn't put the session jars on the classpath of the > > locator. > > > > The hang doesn't happen with 1.9.0, so I'd I think we should make > sure > > it > > > > won't happen with 1.10.0 as well. > > > > > > > > -Dan > > > > > > > > > > -- > > -John > > john.blum10101 (skype) > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Mike, Agreed, we'll probably need to create an enhancement request for this feature in JIRA. Cheers. On Thu, Jul 11, 2019 at 5:37 PM Michael Stolz wrote: > One thing I will mention regarding DATA:READ:RegionName allowing query > behavior is that we have been asked by some users already to separate > DATA:READ:RegionName from DATA:QUERY:RegionName. This request is to protect > against arbitrary query execution by administrators that can cause huge > resource consumption. > > So regardless of all the rest of the proposal, that's something we should > probably consider standardizing on. > > -- > Mike Stolz > Principal Engineer, Pivotal Cloud Cache > Mobile: +1-631-835-4771 > > > > On Thu, Jul 11, 2019 at 11:36 AM Juan José Ramos > wrote: > > > Hello all, > > > > Friendly reminder regarding the deadline to rise concerns and/or > objections > > regarding the *OQL Method InvocationSecurity Proposal [1]*, I'll go ahead > > and move it to *Development* on July 13th. > > Best regards. > > > > [1]: > > > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt > > > > > > On Mon, Jul 8, 2019 at 3:29 PM Juan José Ramos > wrote: > > > > > Done [1]!. > > > Please remember that, if no major concerns arise before Friday this > week, > > > I'll go ahead and move the proposal to *Development* on July 13th. > > > Best regards. > > > > > > [1]: > > > > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt > > > > > > On Fri, Jul 5, 2019 at 3:48 PM Jacob Barrett > > wrote: > > > > > >> Can you please add a Prior Art section to your proposal discussing > these > > >> alternative solutions and why they are insufficient? > > >> > > >> Thanks, > > >> Jake > > >> > > >> > > >> > On Jul 5, 2019, at 10:41 AM, Juan José Ramos > > wrote: > > >> > > > >> > Hello Jake, > > >> > > > >> > I've replied something similar *here [1]*. > > >> > Long story short, I haven't found anything that really applies to > our > > >> use > > >> > case. The "most similar solution" is *Spring Method Security [2]*, > > which > > >> > basically implies annotating methods with explicit configuration > about > > >> the > > >> > roles required to execute them. The same goes for *Shiro > > >> **Annotation-based > > >> > Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]* > > approach > > >> > from the proposal is somewhat similar to this, but I've discarded it > > >> > because if forces the user to annotate classes with our own > > annotations, > > >> > basically forcing them to modify their domain model. > > >> > The proposal basically allows our users to use one of the default of > > the > > >> > box implementations and, if they don't like them for whatever > reason, > > is > > >> > flexible enough so they can ultimately provide their own. > > >> > Hope this helps. > > >> > Cheers. > > >> > > > >> > [1]: > > >> > > > >> > > > https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results > > >> > [2]: > > >> > > > >> > > > https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method > > >> > [3]: > > >> > > > >> > > > https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization > > >> > [4]: > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer > > >> > > > >> > On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett > > >> wrote: > > >> > > > >> >> So if we don’t want to use the Java built in SecurityManager to > solve > > >> >> this, because we feel it's too big or too inflexible for our needs, > > >> have > > >> >> other projects implemented something we can borrow? We can’t be the > > >> first > > >> >> to need something like this if Java’s
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello all, Friendly reminder regarding the deadline to rise concerns and/or objections regarding the *OQL Method InvocationSecurity Proposal [1]*, I'll go ahead and move it to *Development* on July 13th. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt On Mon, Jul 8, 2019 at 3:29 PM Juan José Ramos wrote: > Done [1]!. > Please remember that, if no major concerns arise before Friday this week, > I'll go ahead and move the proposal to *Development* on July 13th. > Best regards. > > [1]: > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt > > On Fri, Jul 5, 2019 at 3:48 PM Jacob Barrett wrote: > >> Can you please add a Prior Art section to your proposal discussing these >> alternative solutions and why they are insufficient? >> >> Thanks, >> Jake >> >> >> > On Jul 5, 2019, at 10:41 AM, Juan José Ramos wrote: >> > >> > Hello Jake, >> > >> > I've replied something similar *here [1]*. >> > Long story short, I haven't found anything that really applies to our >> use >> > case. The "most similar solution" is *Spring Method Security [2]*, which >> > basically implies annotating methods with explicit configuration about >> the >> > roles required to execute them. The same goes for *Shiro >> **Annotation-based >> > Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]* approach >> > from the proposal is somewhat similar to this, but I've discarded it >> > because if forces the user to annotate classes with our own annotations, >> > basically forcing them to modify their domain model. >> > The proposal basically allows our users to use one of the default of the >> > box implementations and, if they don't like them for whatever reason, is >> > flexible enough so they can ultimately provide their own. >> > Hope this helps. >> > Cheers. >> > >> > [1]: >> > >> https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results >> > [2]: >> > >> https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method >> > [3]: >> > >> https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization >> > [4]: >> > >> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer >> > >> > On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett >> wrote: >> > >> >> So if we don’t want to use the Java built in SecurityManager to solve >> >> this, because we feel it's too big or too inflexible for our needs, >> have >> >> other projects implemented something we can borrow? We can’t be the >> first >> >> to need something like this if Java’s solution isn’t a good fit. >> >> >> >> Again I want to avoid inventing something new. What prior art is out >> there? >> >> >> >> >> >>> On Jul 4, 2019, at 1:29 PM, Juan José Ramos >> wrote: >> >>> >> >>> Hello all, >> >>> >> >>> If you haven't added my email to the spam folder already :-), then I'd >> >> like >> >>> to let you know that I've update again the *Proposal [1]* and >> >> incorporated >> >>> most of the feedback provided, along with some additional information >> and >> >>> context I missed on the previous versions, thanks all that brought >> >> concerns >> >>> and suggestions to the discussion. Please take some time to review it >> >>> thoroughly, adding comments and/or concerns *only on this email >> thread*, >> >>> all feedback is more than welcome. >> >>> If no major concerns arise before July 12th 2019, I'll go ahead and >> mark >> >>> move the proposal to *Development* on July 13th. >> >>> Best regards. >> >>> >> >>> [1]: >> >>> >> >> >> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security >> >> >> >> >> > >> > -- >> > Juan José Ramos Cassella >> > Senior Technical Support Engineer >> > Email: jra...@pivotal.io >> > Office#: +353 21 4238611 >> > Mobile#: +353 87 2074066 >> > After Hours Contact#: +1 877
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Done [1]!. Please remember that, if no major concerns arise before Friday this week, I'll go ahead and move the proposal to *Development* on July 13th. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-PriorArt On Fri, Jul 5, 2019 at 3:48 PM Jacob Barrett wrote: > Can you please add a Prior Art section to your proposal discussing these > alternative solutions and why they are insufficient? > > Thanks, > Jake > > > > On Jul 5, 2019, at 10:41 AM, Juan José Ramos wrote: > > > > Hello Jake, > > > > I've replied something similar *here [1]*. > > Long story short, I haven't found anything that really applies to our use > > case. The "most similar solution" is *Spring Method Security [2]*, which > > basically implies annotating methods with explicit configuration about > the > > roles required to execute them. The same goes for *Shiro > **Annotation-based > > Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]* approach > > from the proposal is somewhat similar to this, but I've discarded it > > because if forces the user to annotate classes with our own annotations, > > basically forcing them to modify their domain model. > > The proposal basically allows our users to use one of the default of the > > box implementations and, if they don't like them for whatever reason, is > > flexible enough so they can ultimately provide their own. > > Hope this helps. > > Cheers. > > > > [1]: > > > https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results > > [2]: > > > https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method > > [3]: > > > https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization > > [4]: > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer > > > > On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett > wrote: > > > >> So if we don’t want to use the Java built in SecurityManager to solve > >> this, because we feel it's too big or too inflexible for our needs, have > >> other projects implemented something we can borrow? We can’t be the > first > >> to need something like this if Java’s solution isn’t a good fit. > >> > >> Again I want to avoid inventing something new. What prior art is out > there? > >> > >> > >>> On Jul 4, 2019, at 1:29 PM, Juan José Ramos wrote: > >>> > >>> Hello all, > >>> > >>> If you haven't added my email to the spam folder already :-), then I'd > >> like > >>> to let you know that I've update again the *Proposal [1]* and > >> incorporated > >>> most of the feedback provided, along with some additional information > and > >>> context I missed on the previous versions, thanks all that brought > >> concerns > >>> and suggestions to the discussion. Please take some time to review it > >>> thoroughly, adding comments and/or concerns *only on this email > thread*, > >>> all feedback is more than welcome. > >>> If no major concerns arise before July 12th 2019, I'll go ahead and > mark > >>> move the proposal to *Development* on July 13th. > >>> Best regards. > >>> > >>> [1]: > >>> > >> > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security > >> > >> > > > > -- > > Juan José Ramos Cassella > > Senior Technical Support Engineer > > Email: jra...@pivotal.io > > Office#: +353 21 4238611 > > Mobile#: +353 87 2074066 > > After Hours Contact#: +1 877 477 2269 > > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT > > How to upload artifacts: > > https://support.pivotal.io/hc/en-us/articles/204369073 > > How to escalate a ticket: > > https://support.pivotal.io/hc/en-us/articles/203809556 > > > > [image: support] <https://support.pivotal.io/> [image: twitter] > > <https://twitter.com/pivotal> [image: linkedin] > > <https://www.linkedin.com/company/3048967> [image: facebook] > > <https://www.facebook.com/pivotalsoftware> [image: google plus] > > <https://plus.google.com/+Pivotal> [image: youtube] > > < > https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl> > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Jake, I've replied something similar *here [1]*. Long story short, I haven't found anything that really applies to our use case. The "most similar solution" is *Spring Method Security [2]*, which basically implies annotating methods with explicit configuration about the roles required to execute them. The same goes for *Shiro **Annotation-based Authorization [3]*. The *AnnotationBasedMethodAuthorize**r [3]* approach from the proposal is somewhat similar to this, but I've discarded it because if forces the user to annotate classes with our own annotations, basically forcing them to modify their domain model. The proposal basically allows our users to use one of the default of the box implementations and, if they don't like them for whatever reason, is flexible enough so they can ultimately provide their own. Hope this helps. Cheers. [1]: https://markmail.org/message/ekons7ixtz4jtf7n#query:+page:1+mid:snxgpsqd3yuppmsc+state:results [2]: https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method [3]: https://shiro.apache.org/authorization.html#Authorization-AnnotationbasedAuthorization [4]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-AnnotationBasedMethodAuthorizer On Fri, Jul 5, 2019 at 1:46 PM Jacob Barrett wrote: > So if we don’t want to use the Java built in SecurityManager to solve > this, because we feel it's too big or too inflexible for our needs, have > other projects implemented something we can borrow? We can’t be the first > to need something like this if Java’s solution isn’t a good fit. > > Again I want to avoid inventing something new. What prior art is out there? > > > > On Jul 4, 2019, at 1:29 PM, Juan José Ramos wrote: > > > > Hello all, > > > > If you haven't added my email to the spam folder already :-), then I'd > like > > to let you know that I've update again the *Proposal [1]* and > incorporated > > most of the feedback provided, along with some additional information and > > context I missed on the previous versions, thanks all that brought > concerns > > and suggestions to the discussion. Please take some time to review it > > thoroughly, adding comments and/or concerns *only on this email thread*, > > all feedback is more than welcome. > > If no major concerns arise before July 12th 2019, I'll go ahead and mark > > move the proposal to *Development* on July 13th. > > Best regards. > > > > [1]: > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello all, If you haven't added my email to the spam folder already :-), then I'd like to let you know that I've update again the *Proposal [1]* and incorporated most of the feedback provided, along with some additional information and context I missed on the previous versions, thanks all that brought concerns and suggestions to the discussion. Please take some time to review it thoroughly, adding comments and/or concerns *only on this email thread*, all feedback is more than welcome. If no major concerns arise before July 12th 2019, I'll go ahead and mark move the proposal to *Development* on July 13th. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security On Wed, Jul 3, 2019 at 3:22 PM Dan Smith wrote: > With Juan's proposal, the user still has the option to use java's security > manager. They can install a MethodAuthorizor that allows all methods and > use a java SecurityManager to restrict method access to the methods they > want. > > Regarding the question of "How far should we go to prevent users from > shooting themselves in the foot?" I think we do need to honor the security > policies that administrators have configured for geode. Since we allow the > administrators to configure the system to give certain users READ but not > WRITE access, it's on us to make sure those users are not allowed to modify > the data. If users with READ access can modify data, that is a security > vulnerability in Geode. > > -Dan > > > On Wed, Jul 3, 2019 at 8:33 AM Juan José Ramos wrote: > > > Hello, > > > > Thanks for the feedback provided so far. > > I'm still resilient to add the *Java Security Manager* as an option, > maybe > > it's due to my lack of experience around the subject but I still believe > it > > is killing mosquitoes with a bazooka. We need to prevent this "bad > things" > > only from within the OQL execution, and also provide our users with the > > means to customize the behaviour on their own desire, meaning that the > > configurable *Authorizer* stuff is still needed (it's up to them if they > > want to re-introduce CVE-2017-9795). > > Either way, this is not my decision to make, it's up to the community to > > decide. I'll go ahead and update the proposal, we can have another > feedback > > round (1 week) and cast votes afterwards. > > Cheers. > > > > > > On Tue, Jul 2, 2019 at 5:23 PM Jason Huynh wrote: > > > > > Are security manager policies modifiable on the fly? Just wondering if > > > someone decides they want to disallow or allow something, will they > need > > to > > > restart their vms/geode node? > > > > > > I think Dan pointed this out earlier in the thread, but just wanted to > > have > > > us consider the original cve that led to the heavy handed deny all > method > > > invocations: > > > > > > CVE-2017-9795 Apache Geode OQL method invocation vulnerability > > > > > > Description: > > > A malicious user with read access to specific regions within a Geode > > > cluster may execute OQL queries that allow read and write access to > > > objects within unauthorized regions. In addition a user could invoke > > > methods that allow remote code execution > > > > > > I think Juan's proposal would still allow us to provide multiple > > solutions > > > that may or may not reopen that hole, but it would be up to the user to > > > decide what they are willing to accept. The choice for what should be > > > default would still be up for debate... > > > > > > > > > > > > On Tue, Jul 2, 2019 at 1:07 PM Jacob Barrett > > wrote: > > > > > > > > > > > > > > > > On Jul 2, 2019, at 11:58 AM, Juan José Ramos > > > wrote: > > > > > > > > > > Hello Jake, > > > > > > > > > > I've been doing some reading about the *Java Security Manager* and, > > > even > > > > > when it might work for our use case, I don't think is a good fit > due > > to > > > > the > > > > > following reasons: > > > > > 1). We already have chosen *Shiro* for authentication and > > > authorization, > > > > > adding yet another security framework (and mapping between roles > and > > > > > permissions between the two of them) for OQL method invocation > > security > > > > > seems overkilling to me. I'd rather spend some time improving the > > > > > *ResourcePermissionBased
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello, Thanks for the feedback provided so far. I'm still resilient to add the *Java Security Manager* as an option, maybe it's due to my lack of experience around the subject but I still believe it is killing mosquitoes with a bazooka. We need to prevent this "bad things" only from within the OQL execution, and also provide our users with the means to customize the behaviour on their own desire, meaning that the configurable *Authorizer* stuff is still needed (it's up to them if they want to re-introduce CVE-2017-9795). Either way, this is not my decision to make, it's up to the community to decide. I'll go ahead and update the proposal, we can have another feedback round (1 week) and cast votes afterwards. Cheers. On Tue, Jul 2, 2019 at 5:23 PM Jason Huynh wrote: > Are security manager policies modifiable on the fly? Just wondering if > someone decides they want to disallow or allow something, will they need to > restart their vms/geode node? > > I think Dan pointed this out earlier in the thread, but just wanted to have > us consider the original cve that led to the heavy handed deny all method > invocations: > > CVE-2017-9795 Apache Geode OQL method invocation vulnerability > > Description: > A malicious user with read access to specific regions within a Geode > cluster may execute OQL queries that allow read and write access to > objects within unauthorized regions. In addition a user could invoke > methods that allow remote code execution > > I think Juan's proposal would still allow us to provide multiple solutions > that may or may not reopen that hole, but it would be up to the user to > decide what they are willing to accept. The choice for what should be > default would still be up for debate... > > > > On Tue, Jul 2, 2019 at 1:07 PM Jacob Barrett wrote: > > > > > > > > On Jul 2, 2019, at 11:58 AM, Juan José Ramos > wrote: > > > > > > Hello Jake, > > > > > > I've been doing some reading about the *Java Security Manager* and, > even > > > when it might work for our use case, I don't think is a good fit due to > > the > > > following reasons: > > > 1). We already have chosen *Shiro* for authentication and > authorization, > > > adding yet another security framework (and mapping between roles and > > > permissions between the two of them) for OQL method invocation security > > > seems overkilling to me. I'd rather spend some time improving the > > > *ResourcePermissionBasedMethodAuthorizer > > > > The security manager doesn’t have to be as fined grained as individual > > users. Do we really intend to support actions on objects base on current > > user too? > > > > > [1] *and/or adding *Permissions*/*Roles* to our current security > > framework > > > than introducing a new security framework altogether. > > > > Again, we don’t have to add anything. If the user wants to restrict > access > > to certain activities they could add a policy file to the JVM to do so. > It > > doesn’t have to be user based. We could simply have two hard coded roles, > > UserCode and SystemCode. SystemCode has AllPermissions, UserCode has > none. > > Now we run all queries under the context of UserCode and it can’t execute > > any File, Runtime, Socket, etc. > > > > > 2). There can only be one *Security Manager* per JVM at a given time > > Eh, sorta but no, you can use thread contexts and class loader isolation, > > but there really doesn’t need to be more than one. > > . > > > 3). Customers already using a custom *Security Manager* on their own > will > > > be in trouble... we can certainly wrote our own implementation as a > > > composite and parse multiple policy files, but this will probably break > > the > > > backward compatibility for these customers (and requires yet more > > > configuration and things to keep in mind when upgrading). > > Their custom SecurityManager would conform to the same rules as the > > default file based one in the JRE. They would be able to control code > > access using their existing framework. Seems like win to me. > > > > > 4). The current set of default *Permissions* (*PropertyPermission*, > > > *FilePermission*, etc.) don't apply to our use case, so we'll need to > > > create our own implementations and do some plumbing to map to what we > > > currently have in terms of principals and roles (more boilerplate). > > > > How do they not apply? > > > > > 5). In order to check a permission we basically need to check whether a > > > Security Manager is installed (*System.
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Jake, I've been doing some reading about the *Java Security Manager* and, even when it might work for our use case, I don't think is a good fit due to the following reasons: 1). We already have chosen *Shiro* for authentication and authorization, adding yet another security framework (and mapping between roles and permissions between the two of them) for OQL method invocation security seems overkilling to me. I'd rather spend some time improving the *ResourcePermissionBasedMethodAuthorizer [1] *and/or adding *Permissions*/*Roles* to our current security framework than introducing a new security framework altogether. 2). There can only be one *Security Manager* per JVM at a given time. 3). Customers already using a custom *Security Manager* on their own will be in trouble... we can certainly wrote our own implementation as a composite and parse multiple policy files, but this will probably break the backward compatibility for these customers (and requires yet more configuration and things to keep in mind when upgrading). 4). The current set of default *Permissions* (*PropertyPermission*, *FilePermission*, etc.) don't apply to our use case, so we'll need to create our own implementations and do some plumbing to map to what we currently have in terms of principals and roles (more boilerplate). 5). In order to check a permission we basically need to check whether a Security Manager is installed (*System.getSecurityManager() != null*) and execute the check afterwards (*securityManager**.checkPermission(perm)*); the *Security Manager* then looks at the classes of all methods currently within the call stack (roughly speaking) and returns or throws an exception... a lookup through a simple Map is probably faster, and easier to maintain and read. 6). Overall, it feels like killing mosquitoes with a bazooka to me: it's certainly a powerful framework and we might certainly use it to achieve our goal, but it just seems too complicated for what we are trying to do. My expertise with the *Java Security Manager* has came from what I've recently read, anyway, so chances are high that I might be missing something :-). That said, if we all agree in that *we can't prevent users from shooting themselves in the foot* (thanks for that phrase!!) then things become somewhat trivial: we known which internal methods can compromise the integrity of the system and we also know which general methods are dangerous, so we just need to have an immutable structure within our default authorizer and check if the method belongs to it or not to decide whether it should be allowed/denied. We can certainly provide the other 2 or 3 implementations out of the box and allow customers to provide their own in case they want to restrict things further but, as far as this proposal goes, I think we're good to incorporate the comments/suggestions gathered and go for another feedback round (it's what I plan to do now). Cheers. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-ResourcePermissionBasedMethodAuthorizer On Mon, Jul 1, 2019 at 12:57 PM Juan José Ramos wrote: > Hello Jake, > > Thanks for your reply. Some comments below, inline. > > *>> Premature optimization can be bad but so can ignoring it. Most > importantly here is to define the scope of an authorization. This will need > to be defined as part of the API/SPI. If I was to implement one of these > authorizers I would need to know under what conditions my authorize method > going to be called. Is it going to be for every object that is scanned by > this method? The API suggests this might be the case given that target is > an Object and not a Class. We can do some profiling in our head an know > that if we invoke this method for every potential object we scan then > performance is going to suffer. The regex matcher will destroy the CPU. > What if my implementation was to use LDAP, ouch! If we define scope to be > per query this is orders of magnitude better. The API/SPI should probably > change to reflect this. Maybe target should be the Class? Should the query > itself be included? Now if the target is a Class, and classes are > immutable, perhaps now the authorize method could be invoked once for the > life of the application only when a new Class is touched in our object > scan. Such scope would require that either the API/SPI define that rules > are also immutable or that there is some invalidation mechanism to tell the > system the rules have been mutated. Again, this has API/SPI ramifications > that all need to be discussed in this proposal.* > [Juan]: Agreed, I will keep this in mind when modifying the proposal. > > *>> How far do we go to prevent the user from shooting themselves in the > foot?. Here we could change OQL to any support the bean interface and > disallow all set operations.* > [Juan]: I'm not
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Jake, Thanks for your reply. Some comments below, inline. *>> Premature optimization can be bad but so can ignoring it. Most importantly here is to define the scope of an authorization. This will need to be defined as part of the API/SPI. If I was to implement one of these authorizers I would need to know under what conditions my authorize method going to be called. Is it going to be for every object that is scanned by this method? The API suggests this might be the case given that target is an Object and not a Class. We can do some profiling in our head an know that if we invoke this method for every potential object we scan then performance is going to suffer. The regex matcher will destroy the CPU. What if my implementation was to use LDAP, ouch! If we define scope to be per query this is orders of magnitude better. The API/SPI should probably change to reflect this. Maybe target should be the Class? Should the query itself be included? Now if the target is a Class, and classes are immutable, perhaps now the authorize method could be invoked once for the life of the application only when a new Class is touched in our object scan. Such scope would require that either the API/SPI define that rules are also immutable or that there is some invalidation mechanism to tell the system the rules have been mutated. Again, this has API/SPI ramifications that all need to be discussed in this proposal.* [Juan]: Agreed, I will keep this in mind when modifying the proposal. *>> How far do we go to prevent the user from shooting themselves in the foot?. Here we could change OQL to any support the bean interface and disallow all set operations.* [Juan]: I'm not sure about this one... how far do we go?, I guess it's a decision the community needs to make. I certainly believe that we can't prevent everything, but I also think that we can't force our uses to design their model in terms of our one-direction decisions. As an example, why should I call my accessor only method get*/is* to be able to invoke it on a query?, what if the company I work for wants to use the local native tongue and the equivalent for *get*/is** instead of using english all together?, etc. *>> I want us to consider systems that exist before we spend time inventing our own. There is cost in maintain our own solution in implementation time and bug squashing. There is also the cost in training on our own solution. SecurityManger is discussed in numerous books and blogs, our solution would not have that kind of coverage. Let’s think really critically there. What are we trying to prevent? Is there a solution in the Java echo system that already does what we really need to do? Can we use a preexisting solution and avoid the risk of implementing our own?* [Juan]: Agreed!. *>> I have spent plenty of time in the guts of SecurityManager in previous lives. I think, based on the intended use of this proposal, that Java SecurityManager more than meets our needs.*[Juan]: Fair enough, I'll go ahead and do some reading about the *Security Manager *on my own, will probably ping you once I have enough context to discuss options (or even ask you some questions :-)). One concern I have, however, is regarding *How far do we go to prevent the user from shooting themselves in the foot?*. I think this needs to be decided by the community, do we have any formal *poll* mechanism or something like that?. As a matter of fact, if we decide to prevent OQL method invocation only on our own internal/protected classes (and some known dangerous things like getClass), the implementation would be wy easier... Cheers. On Mon, Jul 1, 2019 at 12:07 PM Jacob Barrett wrote: > > > > On Jul 1, 2019, at 6:55 AM, Juan José Ramos wrote: > > > >> Can we safely assume that some caching of authorization requests will > >> be performed? What will the scope and lifetime of this caching be? Are > the > >> authentication rules and modules assumed to be immutable at runtime? > All of > >> this will have significant implications on performance. > > My assumption right > > now is that there will be some caching performed and will certainly try > to > > keep the authorizers immutable but, however, no PoC has been done and the > > implementations details shown in the proposal are just a draft, so I'm > not > > in a position to accurately answer these questions. I also read that > > *"premature > > optimization is the root of all evil", *so I guess we can benchmark the > > solution with and without caching, and decide what's better based on some > > real data (I certainly see some gains using caching on the > > *RegexBasedMethodAuthorizer*, not sure about the > > *JavaBeanAccessorBasedMethodAuthorizer*). > > Premature optimization can be bad but so can ignoring it. Most importantly > here is to define the s
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Jake,
*>> You asked people to comment in both the wiki and the emails but you
didn’t include comments from the wiki below.*
I never said I was going to reply to comments in the wiki and to the email
thread at the same time on the same day. I didn't forget your comments,
BTW, I was going to answer them today on the WIKI directly.
Either way, now that the *Lightweight RFC Process [1]* has been released,
I'll go ahead and address all comments here in the email thread.
*>> Can we safely assume that some caching of authorization requests will
be performed? What will the scope and lifetime of this caching be? Are the
authentication rules and modules assumed to be immutable at runtime? All of
this will have significant implications on performance.*My assumption right
now is that there will be some caching performed and will certainly try to
keep the authorizers immutable but, however, no PoC has been done and the
implementations details shown in the proposal are just a draft, so I'm not
in a position to accurately answer these questions. I also read that
*"premature
optimization is the root of all evil", *so I guess we can benchmark the
solution with and without caching, and decide what's better based on some
real data (I certainly see some gains using caching on the
*RegexBasedMethodAuthorizer*, not sure about the
*JavaBeanAccessorBasedMethodAuthorizer*).
*>> The second issue is how does this differ, augment are compete with
Java’s built in Security Manager / Policy system. It was designed for a lot
of these same reasons, restricting application access to specific OS level
operations that can be dangerous if executed by malicious code. Why is such
a system not sufficient to handle our concerns in OQL? Beyond creating
sockets, files, threads, forks, etc. what are we intending to prevent the
OQL user executing?*
Thanks for pointing this out, I guess I should have included these examples
in the proposal (will certainly do it in the next iteration) as the
starting point so readers could see what we wanted to prevent when *GEODE-3247
[2]* was implemented.
*1. Reflection (This allows the user to do everything)*
select * from /region r where
r.getClass().forName('java.lang.Runtime').getDeclaredMethods()[0].invoke()
*2. Doing anything with the cache (closing, accessing all regions, etc.)*
select * from /region.getCache().close()
*3. Destroying, adding or invalidating the entire region or specific
elements*
select * from /region.destroyRegion()
select * from /region.invalidate('xyz')
select * from /region.put('xyz','abc')
*4. Modifying an object in place*
select r.setName('Zaraza') from /region r
I think the authorizers from the proposal covers items 1, 2 and 3; item 4
is more complicated and I currently don't see a way to prevent in place
modifications (other than trusting that accessor methods - get/is - don't
modify the object or denying everything but let users customize exactly
what should be allowed through regex).
I honestly don't know much about the *Java Security Manager *and didn't
consider its usage due to the fact that it was't considered in the first
place when GEODE-3247 was implemented. I'll spend some time looking at it
today, however, and see if it's applicable to our use case. If you (or
anyone reading this) already has enough knowledge about the *Java Security
Manager* and knows whether we can apply it to our scenario, please shout
and let's have a talk :-).
Best regards.
[1]:
https://cwiki.apache.org/confluence/display/GEODE/Lightweight+RFC+Process
[2]: https://issues.apache.org/jira/browse/GEODE-3247
On Fri, Jun 28, 2019 at 6:29 PM Jacob Barrett wrote:
> Juan,
>
> You asked people to comment in both the wiki and the emails but you didn’t
> include comments from the wiki below.
>
>
> I have two issues, the first I raised in the wiki is what about caching
> the authentication lookups:
> > Can we safely assume that some caching of authorization requests will be
> performed? What will the scope and lifetime of this caching be? Are the
> authentication rules and modules assumed to be immutable at runtime? All of
> this will have significant implications on performance.
>
> The second issue is how does this differ, augment are compete with Java’s
> built in Security Manager / Policy system. It was designed for a lot of
> these same reasons, restricting application access to specific OS level
> operations that can be dangerous if executed by malicious code. Why is such
> a system not sufficient to handle our concerns in OQL? Beyond creating
> sockets, files, threads, forks, etc. what are we intending to prevent the
> OQL user executing?
>
> Thanks,
> Jake
>
>
> > On Jun 28, 2019, at 10:36 AM, Juan José Ramos wrote:
> >
> > Hello all,
> >
> > Below are some answers/comments to the questions and feedback gathered
> > during the last ro
Re: [PROPOSAL]: Improve OQL Method Invocation Security
l always be their responsibility to make that sure the cluster is secured, that the deployed code is "safe", and that their internal users and operators are correctly trained to use the product. With that in mind, would it be correct to assume that the following (draft) MethodAuthorizer addresses these concerns (the other two out of box implementations will still be available to the users)?: *[MeaningfulPrefix?]MethodAuthorizer* Allow any method execution as long as the target object *does not belong to Geode packages, or does belong but it's marked as safe*. Some known dangerous methods (like getClass) will be rejected, no matter whether the target object belongs to a Geode package or not. The implementation will have an internal list of "safe" Geode methods, and use that to decide whether a method invoked on a Geode class is allowed or denied. The pseudocode: - If the target object doesn't belong to a Geode package, allow its execution. - If the target object belongs to a Geode package: . Is it marked as safe?, allow its execution. . It is not marked as safe?, deny its execution. *Advantages* . Easy to implement. . Easy to use, no extra configuration needed. . Solves the general use case: deploy the domain model, start executing OQL and invoking methods without further changes. *Risks / Unknowns / Disadvantages* . All methods will be allowed, including those that actually mutate objects. . Methods on OQL bind parameters will be allowed as well, and that basically can be anything. Whatever we do and no matter which approach we follow, we'll end up invoking alien methods anyway and we certainly can't prevent these methods from modifying the objects in memory, at the end of the day it will the user's responsibility to ensure operators are correctly trained and no dangerous objects as used as part of the OQL bind parameters. Sorry for the long email, I'll wait until Monday to add this new authorizer as the default out of the box implementation to the proposal, give people some time to analyze the changes, and will try to schedule a live session afterwards. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security#OQLMethodInvocationSecurity-Risks/Unknowns/Disadvantages.1 [2]: https://docs.spring.io/spring-security/site/docs/5.1.5.RELEASE/reference/html/jc.html#jc-method On Wed, Jun 26, 2019 at 2:46 PM Juan José Ramos wrote: > Helo all, > > Thanks for all the feedback. > I'll go over the comments and reply accordingly, hopefully between > tomorrow and Friday. > Best regards. > > > > On Wed, Jun 26, 2019 at 2:12 PM Dan Smith wrote: > >> A couple of other things to keep in mind to make sure you aren't >> introducing security holes >> >> 1) Queries can be executed by users with read privileges. So if it needs >> to >> be really clear to the operator whether the MethodInvocationAuthorizer >> they >> are configuring is going to let users call methods that modify the data or >> not. >> 2) OQL lets you invoke methods on objects that are passed as parameters to >> the query. So that means that any class that exists on the server and is >> serializable could potentially have methods invoked in it through a query >> when someone opens up access with one of these >> MethodInvocationAuthorizers. >> So the RegexBasedMethodAuthorizer needs to be used with care. >> >> -Dan >> >> On Tue, Jun 25, 2019 at 11:53 AM Anthony Baker wrote: >> >> > Here are the things I think are important: >> > >> > 1) I shouldn’t have to change my domain classes in order to run a query. >> > 2) I shouldn’t have to configure anything to run a “normal” query that >> > uses classes deployed into the cluster and stored in the region. >> > 3) By default the cluster is secure from malicious users trying to >> execute >> > untrusted code*. >> > >> > >> > Anthony >> > >> > * if a user chooses to deploy code into the cluster that does bad things >> > that’s on them >> > >> > >> > > On Jun 25, 2019, at 11:07 AM, Aaron Lindsey >> wrote: >> > > >> > > +1 to the proposal >> > > >> > > Some comments: >> > > >> > > - There is almost always trade-off between security and ease-of-use. >> > The >> > > proposed implementation of JavaBeanAccessorBasedMethodAuthorizer >> makes >> > me >> > > feel uneasy because it would be super easy to create a method that >> > begins >> > > with "get" or "is" but is not actually a JavaBean accessor method. >> > However, >> > > I can also see how nic
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Helo all, Thanks for all the feedback. I'll go over the comments and reply accordingly, hopefully between tomorrow and Friday. Best regards. On Wed, Jun 26, 2019 at 2:12 PM Dan Smith wrote: > A couple of other things to keep in mind to make sure you aren't > introducing security holes > > 1) Queries can be executed by users with read privileges. So if it needs to > be really clear to the operator whether the MethodInvocationAuthorizer they > are configuring is going to let users call methods that modify the data or > not. > 2) OQL lets you invoke methods on objects that are passed as parameters to > the query. So that means that any class that exists on the server and is > serializable could potentially have methods invoked in it through a query > when someone opens up access with one of these MethodInvocationAuthorizers. > So the RegexBasedMethodAuthorizer needs to be used with care. > > -Dan > > On Tue, Jun 25, 2019 at 11:53 AM Anthony Baker wrote: > > > Here are the things I think are important: > > > > 1) I shouldn’t have to change my domain classes in order to run a query. > > 2) I shouldn’t have to configure anything to run a “normal” query that > > uses classes deployed into the cluster and stored in the region. > > 3) By default the cluster is secure from malicious users trying to > execute > > untrusted code*. > > > > > > Anthony > > > > * if a user chooses to deploy code into the cluster that does bad things > > that’s on them > > > > > > > On Jun 25, 2019, at 11:07 AM, Aaron Lindsey > wrote: > > > > > > +1 to the proposal > > > > > > Some comments: > > > > > > - There is almost always trade-off between security and ease-of-use. > > The > > > proposed implementation of JavaBeanAccessorBasedMethodAuthorizer > makes > > me > > > feel uneasy because it would be super easy to create a method that > > begins > > > with "get" or "is" but is not actually a JavaBean accessor method. > > However, > > > I can also see how nice this would be in terms of ease-of-use. > > > - From a security standpoint I prefer AnnotationBasedMethodAuthorizer > > > because it's very explicit on which methods may be executed. To > remove > > the > > > coupling between the configuration and domain classes, you could > > separate > > > out the mapping of which methods are allowed into a different class > > and not > > > use annotations. > > > - How have other projects solved this problem? Can we add a "related > > > work" section to the proposal? If you've already looked and didn't > > really > > > find anything relevant you could also mention that in the proposal. > > > > > > - Aaron > > > > > > > > > On Mon, Jun 24, 2019 at 4:31 PM Jason Huynh wrote: > > > > > >> +1 > > >> > > >> I have some concerns about all of the different ways we configure > geode > > to > > >> be secure, but that's a different issue ;-) > > >> Overall, very thorough proposal Juan! > > >> > > >> > > >> > > >> On Mon, Jun 24, 2019 at 4:22 PM Dan Smith wrote: > > >> > > >>> +1 > > >>> > > >>> This proposal looks good to me! > > >>> > > >>> On Mon, Jun 24, 2019 at 4:15 PM Udo Kohlmeyer > wrote: > > >>> > > >>>> +1, Count me in > > >>>> > > >>>> On 6/24/19 13:06, Juan José Ramos wrote: > > >>>>> Hey Jake, > > >>>>> > > >>>>> Sure, I guess we could do a live session if there's enough interest > > >>> after > > >>>>> people have reviewed the proposal. > > >>>>> Best regards. > > >>>>> > > >>>>> On Mon, Jun 24, 2019 at 4:17 PM Jacob Barrett > > > >>>> wrote: > > >>>>> > > >>>>>> > > >>>>>>> On Jun 24, 2019, at 11:49 AM, Juan José Ramos > > > >>>> wrote: > > >>>>>>> > > >>>>>>> I’d rather get feedback in any way and aggregate everything on > my > > >>> own > > >>>>>> than > > >>>>>>> maybe not getting anything because I'm explicitly limiting the > > >>> options > > >>>> to > > >>>>>>> provide it. > > >>>>>> Dealers choice so both it is! > > >>>>>> > > >>>>>> Could you also consider public live session on some medium, like > > >> Zoom, > > >>>>>> where you can walk through the proposal and take like feedback and > > >>>>>> questions? > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Jake > > >>>>>> > > >>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hey Jake, Sure, I guess we could do a live session if there's enough interest after people have reviewed the proposal. Best regards. On Mon, Jun 24, 2019 at 4:17 PM Jacob Barrett wrote: > > > > On Jun 24, 2019, at 11:49 AM, Juan José Ramos wrote: > > > > I’d rather get feedback in any way and aggregate everything on my own > than > > maybe not getting anything because I'm explicitly limiting the options to > > provide it. > > Dealers choice so both it is! > > Could you also consider public live session on some medium, like Zoom, > where you can walk through the proposal and take like feedback and > questions? > > Thanks, > Jake > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hey Jake, It makes sense. I personally prefer to gather feedback through the Wiki to keep everything in one single place, *but* I've explicitly mentioned the two options as acceptable because this particular proposal already got feedback through both communication channels *and* we haven't settled on a standard yet. I'd rather get feedback in any way and aggregate everything on my own than maybe not getting anything because I'm explicitly limiting the options to provide it. Cheers. On Mon, Jun 24, 2019 at 3:40 PM Jacob Barrett wrote: > > > On Jun 24, 2019, at 11:35 AM, Juan José Ramos wrote: > > > > Please take some time to review it thoroughly, adding comments and/or > > concerns either to the *Wiki page [1]* or this email thread directly, all > > feedback is more than welcome. > > Let’s try to keep the conversation in one medium or the other. Author’s > pick since we haven’t settled on a standard. See conversation about RFC-0 > https://github.com/apache/geode/pull/3706 < > https://github.com/apache/geode/pull/3706>. > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello all, I've update the *Proposal [1]* incorporating the feedback provided. Please take some time to review it thoroughly, adding comments and/or concerns either to the *Wiki page [1]* or this email thread directly, all feedback is more than welcome. I'll reach out again in one week time with an update regarding the status of requested changes (if any), or asking a final time for feedback before considering the proposal approved. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security On Fri, Jun 21, 2019 at 2:19 PM Juan José Ramos wrote: > Hello Dan, > > Thanks a lot for the feedback!. Please find my answers below. > > (1). Great point, I totally missed this one... I wrongly assumed that *this > particular method [1]* was in charge of authorizing the query execution > on the region but I was clearly wrong, the actual authorization happens *here > [2]. *The former just kicks in whenever a method is executed on the > region itself (like *getValues*, *keySet*, etc.). > > (2). Agreed, another oversight on my side, mostly derived from my > misunderstanding in item 1. > > 3). I'm not sure about how this is going to work either, neither if it's > even possible without hugely impacting the performance... truth be told, > the whole idea of this particular authorizer is to allow users to get up > and running *without configuring anything, *but I'm not sure how it can > be done in the real world (specially now that you made me aware my error on > item 1). I'd actually love to just eliminate this authorizer altogether but > it was brought to my attention that the other two proposed out of box > implementations require the user to actually configure something (the > package or the regex expression), and that we should provide some sort of > automatic trust boundary in the sense that users just deploy the domain > model, insert objects into the region, and can execute OQL invoking methods > as long as those methods belong to the objects stored within the region... > determining whether an object belongs to a region or not is fairly > complicated, not achievable right now and will certainly incur into > performance degradation, but I guess is up to the community to decide > whether we should pursue this objective or not. > > As a summary, and before I go ahead and incorporate all the feedback > provided and modify the proposal, I guess the community needs to answer the > following questions: > A. For the sake of security, is it okay to force our users to do some > extra configuration steps (no matter how little or easy they might be) > whenever they deploy the domain model to the cluster and want to execute > some methods on those objects from OQL?. > B. Should we just change the approach from *deny everything* to *allow > everything *instead, and assume that users with enough privileges to > execute queries on the region (*DATA:READ:RegionName*) are trustworthy > (at the end of the day *somebody* gave them that role) and won't > compromise the security through RCE from OQL?. > > If nothing changes during the weekend, I'll wait until Monday and apply > the suggested changes to the proposal in favour of a "YES" to question "A" > (just because that's my preference at the moment), sending a new email > afterward for another feedback round. Changes will still be accepted anyway > as this is just the proposal and the community as whole should deny/accept > it. > Cheers. > > [1]: > https://github.com/apache/geode/blob/develop/geode-core/src/main/java/org/apache/geode/cache/query/internal/RestrictedMethodInvocationAuthorizer.java#L165-L171 > [2]: > https://github.com/apache/geode/blob/develop/geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/BaseCommandQuery.java#L106-L108 > > > On Thu, Jun 20, 2019 at 7:22 PM Dan Smith wrote: > >> +1 >> >> This looks really good! >> >> I put a couple of comments inline, and I have a few more general questions >> here: >> 1. Is the RegionQueryInvocationAuthorizer different than our existing >> shiro >> permissions? I thought users can already grant permissions for specific >> regions. What does this add in addition to that? >> 2. I'm a little unclear on if your >> MethodInvocationAuthorizer.authorizeMethodInvocation is supposed to take a >> region or the target object. If it is really accepting a region, do we >> actually have a region available in all cases? We could be invoking >> methods >> on an object in lots of places in the query tree. >> 3. The DataAwareBasedMethodAuthorizer seems a bit vague on how it's >> actually going to work. It also might be a security risk, since it will >> allow users with DATA
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello Dan, Thanks a lot for the feedback!. Please find my answers below. (1). Great point, I totally missed this one... I wrongly assumed that *this particular method [1]* was in charge of authorizing the query execution on the region but I was clearly wrong, the actual authorization happens *here [2]. *The former just kicks in whenever a method is executed on the region itself (like *getValues*, *keySet*, etc.). (2). Agreed, another oversight on my side, mostly derived from my misunderstanding in item 1. 3). I'm not sure about how this is going to work either, neither if it's even possible without hugely impacting the performance... truth be told, the whole idea of this particular authorizer is to allow users to get up and running *without configuring anything, *but I'm not sure how it can be done in the real world (specially now that you made me aware my error on item 1). I'd actually love to just eliminate this authorizer altogether but it was brought to my attention that the other two proposed out of box implementations require the user to actually configure something (the package or the regex expression), and that we should provide some sort of automatic trust boundary in the sense that users just deploy the domain model, insert objects into the region, and can execute OQL invoking methods as long as those methods belong to the objects stored within the region... determining whether an object belongs to a region or not is fairly complicated, not achievable right now and will certainly incur into performance degradation, but I guess is up to the community to decide whether we should pursue this objective or not. As a summary, and before I go ahead and incorporate all the feedback provided and modify the proposal, I guess the community needs to answer the following questions: A. For the sake of security, is it okay to force our users to do some extra configuration steps (no matter how little or easy they might be) whenever they deploy the domain model to the cluster and want to execute some methods on those objects from OQL?. B. Should we just change the approach from *deny everything* to *allow everything *instead, and assume that users with enough privileges to execute queries on the region (*DATA:READ:RegionName*) are trustworthy (at the end of the day *somebody* gave them that role) and won't compromise the security through RCE from OQL?. If nothing changes during the weekend, I'll wait until Monday and apply the suggested changes to the proposal in favour of a "YES" to question "A" (just because that's my preference at the moment), sending a new email afterward for another feedback round. Changes will still be accepted anyway as this is just the proposal and the community as whole should deny/accept it. Cheers. [1]: https://github.com/apache/geode/blob/develop/geode-core/src/main/java/org/apache/geode/cache/query/internal/RestrictedMethodInvocationAuthorizer.java#L165-L171 [2]: https://github.com/apache/geode/blob/develop/geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/BaseCommandQuery.java#L106-L108 On Thu, Jun 20, 2019 at 7:22 PM Dan Smith wrote: > +1 > > This looks really good! > > I put a couple of comments inline, and I have a few more general questions > here: > 1. Is the RegionQueryInvocationAuthorizer different than our existing shiro > permissions? I thought users can already grant permissions for specific > regions. What does this add in addition to that? > 2. I'm a little unclear on if your > MethodInvocationAuthorizer.authorizeMethodInvocation is supposed to take a > region or the target object. If it is really accepting a region, do we > actually have a region available in all cases? We could be invoking methods > on an object in lots of places in the query tree. > 3. The DataAwareBasedMethodAuthorizer seems a bit vague on how it's > actually going to work. It also might be a security risk, since it will > allow users with DATA:READ permission to invoke any method on these > objects. > > -Dan > > On Wed, Jun 19, 2019 at 11:34 AM Jacob Barrett > wrote: > > > Thanks Juan! > > > > > On Jun 19, 2019, at 9:55 AM, Juan José Ramos > wrote: > > > > > > Hello all, > > > > > > I've removed all "biased" words I could find from the original document > > so > > > the *Proposal [1]* is ready for review and discussion now. All feedback > > is > > > welcome. > > > Best regards. > > > > > > [1]: > > > > > > https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security > > > > > >> On Fri, Jun 14, 2019 at 8:39 PM Juan José Ramos > > wrote: > > >> > > >> Hey Jake, > > >> > > >> Thanks for bringing this up. As you might have fou
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hello all, I've removed all "biased" words I could find from the original document so the *Proposal [1]* is ready for review and discussion now. All feedback is welcome. Best regards. [1]: https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security On Fri, Jun 14, 2019 at 8:39 PM Juan José Ramos wrote: > Hey Jake, > > Thanks for bringing this up. As you might have found out already, english > is not my native language, I actually had to do some research to find out > *exactly what you meant* regarding the bias around the "whitelist" word > :-|... It was an honest mistake and I sincerely apologize in advance if > anyone got offended in any way. > That said, I won't have time to go through the proposal and make the > required changes until next week, so I'll keep the document hidden until > all biased words are replaced. > Cheers. > > > On Sat, Jun 15, 2019 at 12:25 AM Jacob Barrett > wrote: > >> > As part of GEODE-3247 <https://issues.apache.org/jira/browse/GEODE-3247>, >> several options were analysed and, after considering the wealth of security >> holes and the difficulty of determining which methods deployed by the >> developer were intended to be available for queries and which were not, the >> decision was made to tighten up the Security and, by default, disallow any >> method call not explicitly whitelisted. >> >> Please avoid biased words, like whitelist, in source and proposals. There >> are several other places in this document that use these terms. Can you >> please update the document without them. >> >> Thanks, >> Jake >> >> > > -- > Juan José Ramos Cassella > Senior Technical Support Engineer > Email: jra...@pivotal.io > Office#: +353 21 4238611 > Mobile#: +353 87 2074066 > After Hours Contact#: +1 877 477 2269 > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT > How to upload artifacts: > https://support.pivotal.io/hc/en-us/articles/204369073 > How to escalate a ticket: > https://support.pivotal.io/hc/en-us/articles/203809556 > > [image: support] <https://support.pivotal.io/> [image: twitter] > <https://twitter.com/pivotal> [image: linkedin] > <https://www.linkedin.com/company/3048967> [image: facebook] > <https://www.facebook.com/pivotalsoftware> [image: google plus] > <https://plus.google.com/+Pivotal> [image: youtube] > <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl> > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Improve OQL Method Invocation Security
Hey Jake, Thanks for bringing this up. As you might have found out already, english is not my native language, I actually had to do some research to find out *exactly what you meant* regarding the bias around the "whitelist" word :-|... It was an honest mistake and I sincerely apologize in advance if anyone got offended in any way. That said, I won't have time to go through the proposal and make the required changes until next week, so I'll keep the document hidden until all biased words are replaced. Cheers. On Sat, Jun 15, 2019 at 12:25 AM Jacob Barrett wrote: > > As part of GEODE-3247 <https://issues.apache.org/jira/browse/GEODE-3247>, > several options were analysed and, after considering the wealth of security > holes and the difficulty of determining which methods deployed by the > developer were intended to be available for queries and which were not, the > decision was made to tighten up the Security and, by default, disallow any > method call not explicitly whitelisted. > > Please avoid biased words, like whitelist, in source and proposals. There > are several other places in this document that use these terms. Can you > please update the document without them. > > Thanks, > Jake > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Unnecessary uses of final on local variables
+1 to removing *final* on local variables. However, regarding Ryan's example, and even if it adds some "noise" to the source code, I'm in favour of keeping the *final* keyword on local variables whenever the developer wants to explicitly show the intent of making that the variable effectively constant. Cheers. On Fri, Jun 14, 2019 at 12:15 AM Ryan McMahon wrote: > I agree with this sentiment, and have generally only been using final on > class fields and method parameters where I want to guarantee immutability > as of late. However, I was at one time adding final to local variables, > and I know that I have checked in code with final locals. Should we > actively be removing finals when we see it on locals? > > One case where I still have been using final for locals is when the > variable is effectively constant and I want to show that intent. For > instance: > final String resourceId = > "someStaticResourceIdThatReallyShouldntBeReassignedEver"; > > Is this a valid use case or should we view this as unnecessary verbosity as > well? > > Thanks, > Ryan > > > > On Thu, Jun 13, 2019 at 1:31 PM Kirk Lund wrote: > > > According to Effective Java 3rd Edition, all local variables are > implicitly > > made final by the JVM (and thus receiving the same JVM optimizations as a > > final variable) without requiring use of the keyword as long as you only > > set the value once. So this becomes an unnecessary use of the keyword > > final which > > really just adds noise to the code (one more word on many lines) much > like > > unnecessary uses of this. on references to class fields. The only context > > where it's still valuable to use final is on class fields and constants > > where it provides concurrency guarantees. > > > > It may be useful to temporarily mark a local variable as final while > > performing a refactoring. See Martin Fowler's book for various > refactorings > > in which he does this. > > > > There really is no value in retaining the keyword on a local variable any > > longer, so I think we should avoid using it in the same way we avoid > > unnecessary uses of this. on references to class fields. It's all just > more > > noise in the code. > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL] Add hostname-for-clients to ConfigurationProperties
Hello Jinmei, Applying this change will prevent users from using different NICs for different types of traffic, meaning that the same server won't be able to listen for JMX connections on *NIC1* and regular client-server connections on *NIC2*, right?. If my assumption is correct then we'd be removing an existing cool feature from the product, so my vote would be -1. Cheers. On Tue, Jun 4, 2019 at 9:06 PM Dan Smith wrote: > A user can configure the hostname-for-clients for locators and gateway > receivers - they are part of the respective gfsh commands. Are you > suggesting deprecating those settings as well, or just having a global > property that is the default value if it is not overridden for a specific > locator, cache-server, or gateway-receiver? > > -Dan > > On Tue, Jun 4, 2019 at 12:02 PM Jinmei Liao wrote: > > > We have "jmx-manager-hostname-for-clients" in the geode properties, we > > think it would be a good idea to deprecate that and use > > "hostname-for-clients" for the entire server. Currently we already need > > this property when launching a locator and start a gateway-receiver, and > we > > have no way to pre-configure them. > > > > This is first step towards fixing our current problem of cluster > > configuration prescribing specific hostname-for-senders for gateway > > receivers. > > > > Thoughts/comments? > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Test coverage and PRs
+1 On Mon, Apr 15, 2019 at 11:22 PM Alexander Murmann wrote: > Hi everyone, > > TL;DR: > 1. Let's call each other out on our PRs when test coverage is missing or > insufficient > 2. Let's require test coverage for code that gets refactored as well > > We already have a really great wiki page > < > https://cwiki.apache.org/confluence/display/GEODE/Criteria+for+Code+Submissions > > > on how we evaluate code contributions. Among other things it calls out test > coverage for new features as well as for bug fixes. I don't think we've > always been diligent in ensuring healthy test coverage when we've been > reviewing PRs. This seems like something we should correct as a community > to ensure Geode can provide the high quality our users expect and at the > same time give us fast feedback loops in our daily work. We need to get > better code coverage, so rejecting PRs till they have appropriate test > coverage, seems like an obvious thing we need to do. > > It's also interesting to me that the wiki page calls out coverage for > features and bugs. What about refactoring code that is currently missing > coverage? To refactor with confidence we need test coverage. Therefore test > coverage should be a precondition for refactoring. I'd like to amend our > wiki to also require test coverage for refactored code. Ideally that > coverage would already be there, but we all know that's unfortunately > rarely the case. This is going to hurt in the short run, but gets us closer > where we want to be. > > As always, if it's unreasonable to provide certain types of test coverage, > then let's have an explicit conversation on the PR. > > Note: When I say "appropriate test coverage", I am referring to coverage of > the functionality in breadth, but also in depth via a healthy testing > pyramid <https://martinfowler.com/bliki/TestPyramid.html> that includes > unit tests, integration tests etc. > > What do you all think? > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [DISCUSS] - Create new repository for geode benchmarks
+1 On Thu, Nov 15, 2018 at 8:22 PM Udo Kohlmeyer wrote: > +1, never thought there was any other option... > > > On 11/15/18 10:47, Dan Smith wrote: > > Hi all, > > > > We (Naba, Sean, Brian and I) would like to add some benchmarks for geode, > > with a goal of eventually running them as part of our concourse build and > > detecting performance changes. > > > > We think it makes sense to put these benchmarks in a separate > > geode-benchmarks repository. That will make them less tightly coupled to > a > > specific revision of geode. What do you all think? If folks are okay with > > this, I will go ahead and create the repository. > > > > We have some prototype code in this repository with a simple > client/server > > put benchmark: https://github.com/upthewaterspout/geode-performance. > > > > -Dan > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Develop Compilation Failure
Hey Jens, No worries at all, it's fixed now. Cheers. On Tue, Nov 13, 2018 at 3:33 PM Jens Deppe wrote: > No worries, and thanks for the heads-up Juan. > > I'm really not sure anything can be done that wouldn't end up being > heavy-handed and mostly just be an impedance. > > I should have sent out a notice describing the commons-lang change (given > its scope) and that any in-flight PRs might be affected. > > --Jens > > On Tue, Nov 13, 2018 at 6:51 AM Ju@N wrote: > > > Hello devs, > > > > Compilation in *develop* is failing after my latest commit. > > Long story short: the *concourse-ci* tests were all green but the > > *commons-lang* version was upgraded in develop *AFTER* the build finished > > but *BEFORE* the merge was executed, so the import is currently outdated > > (should be *org.apache.commons.lang3.ArrayUtils* instead of > > *org.apache.commons.lang.ArrayUtils*). > > I've already submitted a PR with the fix, I'll merge it once the > > *concourse-ci *finishes. > > BTW: even when the chances of this happening again are quite low, is > there > > anything we can do to automatically prevent the problem?. > > Cheers. > > > > -- > > Ju@N > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [DISCUSS] Deprecating JUnit4CacheTestCase (and maybe JUnit4DistributedTestCase?)
; > > >> builder.setServerPort(0); > > > > > > >> builder.set(LOCATORS, "localHost[" + locatorPort + "]"); > > > > > > >> builder.set(DISABLE_AUTO_RECONNECT, "false"); > > > > > > >> builder.set(ENABLE_CLUSTER_CONFIGURATION, "false"); > > > > > > >> builder.set(MAX_WAIT_TIME_RECONNECT, "1000"); > > > > > > >> builder.set(MEMBER_TIMEOUT, "2000"); > > > > > > >> > > > > > > >> serverLauncher = builder.build(); > > > > > > >> serverLauncher.start(); > > > > > > >> assertThat(serverLauncher.isRunning()).isTrue(); > > > > > > >>} > > > > > > >> > > > > > > >> In particular, I think we should be using ServerLauncher and > > > > > > >> LocatorLauncher instead of Rules when we want a full-stack > > Locator > > > > or > > > > > > >> full-stack Server that looks like what a User is going to > > startup. > > > > > > >> > > > > > > >> Here are my reasons: > > > > > > >> > > > > > > >> 1) I want to learn and use the Geode User APIs directly, not > > > > someone's > > > > > > >> (even mine) Testing API that hides the Geode User APIs. If I > > see a > > > > > test > > > > > > >> fail, I want to see exactly what was configured and what User > > APIs > > > > > were > > > > > > >> used right there in the test without having to open other > > > classes. I > > > > > > don't > > > > > > >> want to have to spend even 15 minutes digging through some > JUnit > > > > Rule > > > > > to > > > > > > >> figure out how PDX was configured. > > > > > > >> > > > > > > >> 2) We need to make sure that the Geode User APIs are easy to > use > > > and > > > > > are > > > > > > >> complete. If we're writing tests against Testing APIs instead > > then > > > > we > > > > > > >> don't > > > > > > >> feel the Users' pain if our API is painful. If the reason to > > use a > > > > > Rule > > > > > > is > > > > > > >> because our User API is overly-verbose of difficult, then > that's > > > > even > > > > > > more > > > > > > >> reason to use the Geode User API, so we recognize that it > needs > > to > > > > > > change! > > > > > > >> > > > > > > >> GemFire had a long history of hiding its User APIs behind > > > elaborate > > > > > > >> Testing > > > > > > >> APIs and we all used these fancy, easier to use, more compact > > > > Testing > > > > > > >> APIs. > > > > > > >> This promotes complicated, inconsistent and potentially > > incomplete > > > > > User > > > > > > >> APIs for Users to actually use. The result: difficult to use > > > product > > > > > > with > > > > > > >> difficult to use APIs and User APIs that are missing important > > > > things > > > > > > that > > > > > > >> then Users have to resort to internal APIs to use. I'm > strongly > > > > > > convinced > > > > > > >> that using elaborate Testing APIs is at least largely > > responsible > > > > for > > > > > > >> making GemFire and now Geode difficult to use and that's why > I'm > > > > > pushing > > > > > > >> so > > > > > > >> hard to write tests with Geode User APIs instead of convenient > > > > custom > > > > > > >> Rules > > > > > > >> > > > > > > >> Since I started using ServerLauncher and LocatorLauncher APIs > > > > directly > > > > > > in > > > > > > >> my DistributedTests I made a very important discovery: the > User > > > has > > > > no > > > > > > way > > > > > > >> to get a reference to the Cache. This is why I recently > started > > a > > > > > > >> discussion thread about add getCache and getLocator to these > > > > Launcher > > > > > > >> APIs. > > > > > > >> If we keep using elaborate Testing APIs including custom Geode > > > JUnit > > > > > > Rules > > > > > > >> to hide these APIs, we'll never make these discoveries that I > > feel > > > > are > > > > > > >> vital for our Users. We need to make things a LOT easier for > the > > > > Users > > > > > > >> going forward. > > > > > > >> > > > > > > >> The above is why I think we should be using User APIs in the > > tests > > > > > even > > > > > > >> for > > > > > > >> setUp and tearDown. Save the custom JUnit Rules for NON-GEODE > > > things > > > > > > like > > > > > > >> configuring JSON or LOG4J or allowing use of ErrorCollector in > > all > > > > > DUnit > > > > > > >> VMs. > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Kirk > > > > > > >> > > > > > > >> On Fri, Nov 9, 2018 at 10:49 AM, Galen O'Sullivan < > > > > > > gosulli...@pivotal.io> > > > > > > >> wrote: > > > > > > >> > > > > > > >> I was looking at a test recently that extended > > JUnit4CacheTestCase > > > > and > > > > > > >>> only > > > > > > >>> used a single server, and changed it to use > ClusterStartupRule. > > > > > > >>> > > > > > > >>> JUnit4CacheTestCase adds additional complexity to > > > > > > >>> JUnit4DistributedTestCase > > > > > > >>> and with the move to ClusterStartupRule for distributed > tests, > > > > rather > > > > > > >>> than > > > > > > >>> class inheritance, I think we should deprecate > > > JUnit4CacheTestCase > > > > > and > > > > > > >>> change the comment to imply that classes should inherit from > it > > > > just > > > > > > >>> because they require a Cache. > > > > > > >>> > > > > > > >>> Is is worth deprecating JUnit4DistributedTestCase as well and > > > > > > encouraging > > > > > > >>> the use of ClusterStartupRule instead? > > > > > > >>> > > > > > > >>> Thanks, > > > > > > >>> Galen > > > > > > >>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Cheers > > > > > > > > > > Jinmei > > > > > > > > > > > > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Build Jobs not accessible?
Thanks Jake, Cheers. On Fri, Nov 9, 2018 at 3:14 PM Jacob Barrett wrote: > I bet that after we regenerated it yesterday it isn’t public anymore. I’ll > fix that shortly. > > -Jake > > > > On Nov 9, 2018, at 6:59 AM, Juan José Ramos wrote: > > > > Hello Jens, > > > > Yes, I've tried from different browsers after deleting all cookies, > history > > and data, same result. > > I can access https://concourse.apachegeode-ci.info without any problems, > > but I only see *apache-develop-main*, *apache-develop-metrics*, > > *apache-develop-examples* and *apache-develop-image* pipelines. There > used > > to be another one called *apache-develop-pr*, right?, I recall seeing it > > recently. > > Best regards. > > > > > >> On Fri, Nov 9, 2018 at 2:19 PM Jens Deppe wrote: > >> > >> Hi Juan. > >> > >> It could be a cookie problem - have you tried clearing them? > >> > >> Can you access the top-level https://concourse.apachegeode-ci.info? I > can > >> access your build just fine. > >> > >> --Jens > >> > >>> On Fri, Nov 9, 2018 at 5:52 AM Ju@N wrote: > >>> > >>> Hello team, > >>> > >>> I've submitted https://github.com/apache/geode/pull/2822 a couple of > >>> hours ago and the concourse-ci/IntegrationTest failed, even when > >> *./gradlew > >>> integrationTest* runs fine locally. > >>> I've tried to access the failed build ( > >>> https://concourse.apachegeode-ci.info/builds/14585) to see the output > >> and > >>> check what's happening but, after logging in, I only see the empty page > >>> with the *loading* message: > >>> > >>> [image: Screen Shot 2018-11-09 at 1.42.17 PM.png] > >>> > >>> I've also tried to access other failures from other *pull requests* > (not > >>> mine) and the result is exactly the same. Is everyone having the same > >>> issue?, is it just my user?. > >>> Best regards. > >>> > >>> -- > >>> Ju@N > >>> > >> > > > > > > -- > > Juan José Ramos Cassella > > Senior Technical Support Engineer > > Email: jra...@pivotal.io > > Office#: +353 21 4238611 > > Mobile#: +353 87 2074066 > > After Hours Contact#: +1 877 477 2269 > > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT > > How to upload artifacts: > > https://support.pivotal.io/hc/en-us/articles/204369073 > > How to escalate a ticket: > > https://support.pivotal.io/hc/en-us/articles/203809556 > > > > [image: support] <https://support.pivotal.io/> [image: twitter] > > <https://twitter.com/pivotal> [image: linkedin] > > <https://www.linkedin.com/company/3048967> [image: facebook] > > <https://www.facebook.com/pivotalsoftware> [image: google plus] > > <https://plus.google.com/+Pivotal> [image: youtube] > > < > https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl> > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Build Jobs not accessible?
Hello Jens, Yes, I've tried from different browsers after deleting all cookies, history and data, same result. I can access https://concourse.apachegeode-ci.info without any problems, but I only see *apache-develop-main*, *apache-develop-metrics*, *apache-develop-examples* and *apache-develop-image* pipelines. There used to be another one called *apache-develop-pr*, right?, I recall seeing it recently. Best regards. On Fri, Nov 9, 2018 at 2:19 PM Jens Deppe wrote: > Hi Juan. > > It could be a cookie problem - have you tried clearing them? > > Can you access the top-level https://concourse.apachegeode-ci.info? I can > access your build just fine. > > --Jens > > On Fri, Nov 9, 2018 at 5:52 AM Ju@N wrote: > > > Hello team, > > > > I've submitted https://github.com/apache/geode/pull/2822 a couple of > > hours ago and the concourse-ci/IntegrationTest failed, even when > *./gradlew > > integrationTest* runs fine locally. > > I've tried to access the failed build ( > > https://concourse.apachegeode-ci.info/builds/14585) to see the output > and > > check what's happening but, after logging in, I only see the empty page > > with the *loading* message: > > > > [image: Screen Shot 2018-11-09 at 1.42.17 PM.png] > > > > I've also tried to access other failures from other *pull requests* (not > > mine) and the result is exactly the same. Is everyone having the same > > issue?, is it just my user?. > > Best regards. > > > > -- > > Ju@N > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [VOTE] Time-based release schedule for minor releases
+1 for time-based releases. On Tue, Oct 9, 2018 at 12:02 AM Michael Stolz wrote: > +1 Quarterly releases > > -- > Mike Stolz > Principal Engineer - Gemfire Product Manager > Mobile: 631-835-4771 > > On Oct 8, 2018 6:52 PM, "Sai Boorlagadda" > wrote: > > > +1 for time-based minors (if patches are reserved for security fixes) > > > > On Mon, Oct 8, 2018 at 2:55 PM Anthony Baker wrote: > > > > > We reserve the patch version number for when we want to issue a fix > for a > > > security vulnerability or address a critical bug. We did that with > 1.1.1 > > > and 1.2.1. > > > > > > I think the release schedule and SemVer are separate topics. AFAICT > this > > > proposal is just about putting a more deterministic schedule around > what > > we > > > are already doing / have been doing. Given that releases have > > significant > > > overhead, I think it makes sense to plan ahead. If you want to know > more > > > about that ask Naba, the last release manager :-) > > > > > > > > > Anthony > > > > > > > > > > On Oct 8, 2018, at 2:42 PM, Udo Kohlmeyer wrote: > > > > > > > > -0 > > > > > > > > It seems we have completely disregarded the *patch* version number. > > Does > > > this mean Geode versions will be *major*,*minor*? Can we then remove > the > > > *patch* version on the release version? > > > > > > > > In addition to this, should the test coverage not be sufficient > enough > > > to allow "release when green"? I must agree with @Jacob, I would prefer > > > something a lot less formalized. If the community has contributed a > > > significant fix, should that not warrant an ad-hoc patch release? Or > what > > > if the community has added functionality, that could "fill" a single > > minor > > > release by itself, should that not warrant a pre-emptive release. > > > > > > > > All these questions are not enough to warrant this effort to be > > blocked, > > > but I prefer those use cases to be considered for a more comprehensive > > > documentation effort, than what is currently on the wiki. > > > > > > > > In addition to that, is a release with only bug fixes in it, really > > > still a worthy of minor release number, or does it not count as a patch > > > release? > > > > > > > > --Udo > > > > > > > > > > > > On 10/8/18 14:27, Jacob Barrett wrote: > > > >> +0 > > > >> > > > >> My preference is to release when there is something worth releasing > > > rather > > > >> than arbitrary points in time but I don't hold that preference > > strongly > > > >> enough to spike this effort. > > > >> > > > >> On Mon, Oct 8, 2018 at 2:24 PM Alexander Murmann < > amurm...@pivotal.io > > > > > > >> wrote: > > > >> > > > >>> Hi everyone, > > > >>> > > > >>> As discussed in "Predictable minor release cadence", I'd like us to > > > find > > > >>> agreement on releasing a new minor version every three months. > There > > > are > > > >>> more details in the other thread and I should have captured > > everything > > > >>> relevant on the wiki: > > > >>> https://cwiki.apache.org/confluence/display/GEODE/Release+Schedule > > > >>> > > > >>> There are also some discussions about patch releases. Let's please > > > focus > > > >>> this vote on the proposed minor release schedule and carry on other > > > >>> discussions in the [DISCUSS] thread. > > > >>> > > > >>> Thank you all! > > > >>> > > > > > > > > > > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Steps to follow after becoming a Geode committer
Thanks Kirk!. Cheers. On Tue, Sep 18, 2018 at 5:51 PM Kirk Lund wrote: > I reviewed your changes and added approval. Thanks Juan! > > On Tue, Sep 18, 2018 at 12:46 AM, Juan José Ramos > wrote: > > > Hello Jinmei and Dan, > > > > Thanks both for the reply!!. > > Regarding pull 2250, I think the changes requested by Galen are outdated > > now, but I'll ping him directly in the *pull request * anyway to be sure. > > Best regards. > > > > > > On Mon, Sep 17, 2018 at 5:37 PM Dan Smith wrote: > > > > > Hi Juan, > > > > > > Great questions! I agree with what Jinmei said - go ahead and merge if > > you > > > have: > > > > > > 1) at least one approval > > > 2) No changes requested > > > 3) Checks are passing > > > > > > The "at least one review" part is somewhat subjective, if you feel the > PR > > > is complicated and you want more reviews, feel free to ask the list for > > > more feedback. > > > > > > I see https://github.com/apache/geode/pull/2250 has one reviewer > > > requesting > > > changes. You should look into the suggestions, or maybe just poke Galen > > if > > > he needs to review your fixes and approve them? > > > > > > -Dan > > > > > > On Mon, Sep 17, 2018 at 6:45 AM, Jinmei Liao > wrote: > > > > > > > I believe you can go ahead. :-) > > > > > > > > The way I see it, you can merge a PR when all three are satisfied: > > > > 1) you have the privilege to merge it (the button is activated for > you > > to > > > > push). > > > > 2) The PR is approved by a least one reviewer. > > > > 3) all checks passed. > > > > > > > > Thanks! > > > > > > > > > > > > On Mon, Sep 17, 2018, 3:48 AM Ju@N wrote: > > > > > > > > > Hello all, > > > > > > > > > > My apache account is already created and everything seems to be > > working > > > > > just fine, I've already linked my account through > > > > > https://gitbox.apache.org/setup/ and I can see the option *Merge > > pull > > > > > request* in the *GitHub* interface (disabled in the past when I > > didn't > > > > have > > > > > commit privileges on the project). > > > > > I have two old pull requests (2376 > > > > > <https://github.com/apache/geode/pull/2376> for GEODE-5353 > > > > > <https://issues.apache.org/jira/browse/GEODE-5353> and 2250 > > > > > <https://github.com/apache/geode/pull/2250> for GEODE-5314 > > > > > <https://issues.apache.org/jira/browse/GEODE-5314>) already > approved > > > by > > > > > other committers, am I ready to go and merge them myself?, or > should > > I > > > > wait > > > > > for an announcement or something else?. I've already gone through > > > > Becoming > > > > > a Committer > > > > > < > > > https://cwiki.apache.org/confluence/display/GEODE/Becoming+a+committer > > > > > > > and Code Contributions > > > > > <https://cwiki.apache.org/confluence/display/GEODE/Code+ > > contributions> > > > > but > > > > > couldn't find a definitive answer about how/when to merge a *pull > > > > > request* opened > > > > > by yourself when you're a committer, how does this process work?, > > > should > > > > I > > > > > ask in this list for reviews before merging the changes?, should I > > > wait X > > > > > amount of days before asking for reviewers?, etc. > > > > > As a side note, I've received an email from *r...@apache.org > > > > > * with a link to the vote reference in the > > > > > *private.apache.geode.org > > > > > <http://private.apache.geode.org/>* list. I believe that list is > > only > > > > for > > > > > *PMC* members, so I don't (and won't) have access to that list, am > I > > > > > right?. > > > > > Sorry for the long email and the amount of questions, just trying > to > > > make > > > > > sure I get things right from the very beginning :-). > > > > > Best regards. > > > > > > > > > > -- > > > > > Ju@N > > > > > > >
Re: Steps to follow after becoming a Geode committer
Hello Jinmei and Dan, Thanks both for the reply!!. Regarding pull 2250, I think the changes requested by Galen are outdated now, but I'll ping him directly in the *pull request * anyway to be sure. Best regards. On Mon, Sep 17, 2018 at 5:37 PM Dan Smith wrote: > Hi Juan, > > Great questions! I agree with what Jinmei said - go ahead and merge if you > have: > > 1) at least one approval > 2) No changes requested > 3) Checks are passing > > The "at least one review" part is somewhat subjective, if you feel the PR > is complicated and you want more reviews, feel free to ask the list for > more feedback. > > I see https://github.com/apache/geode/pull/2250 has one reviewer > requesting > changes. You should look into the suggestions, or maybe just poke Galen if > he needs to review your fixes and approve them? > > -Dan > > On Mon, Sep 17, 2018 at 6:45 AM, Jinmei Liao wrote: > > > I believe you can go ahead. :-) > > > > The way I see it, you can merge a PR when all three are satisfied: > > 1) you have the privilege to merge it (the button is activated for you to > > push). > > 2) The PR is approved by a least one reviewer. > > 3) all checks passed. > > > > Thanks! > > > > > > On Mon, Sep 17, 2018, 3:48 AM Ju@N wrote: > > > > > Hello all, > > > > > > My apache account is already created and everything seems to be working > > > just fine, I've already linked my account through > > > https://gitbox.apache.org/setup/ and I can see the option *Merge pull > > > request* in the *GitHub* interface (disabled in the past when I didn't > > have > > > commit privileges on the project). > > > I have two old pull requests (2376 > > > <https://github.com/apache/geode/pull/2376> for GEODE-5353 > > > <https://issues.apache.org/jira/browse/GEODE-5353> and 2250 > > > <https://github.com/apache/geode/pull/2250> for GEODE-5314 > > > <https://issues.apache.org/jira/browse/GEODE-5314>) already approved > by > > > other committers, am I ready to go and merge them myself?, or should I > > wait > > > for an announcement or something else?. I've already gone through > > Becoming > > > a Committer > > > < > https://cwiki.apache.org/confluence/display/GEODE/Becoming+a+committer> > > > and Code Contributions > > > <https://cwiki.apache.org/confluence/display/GEODE/Code+contributions> > > but > > > couldn't find a definitive answer about how/when to merge a *pull > > > request* opened > > > by yourself when you're a committer, how does this process work?, > should > > I > > > ask in this list for reviews before merging the changes?, should I > wait X > > > amount of days before asking for reviewers?, etc. > > > As a side note, I've received an email from *r...@apache.org > > > * with a link to the vote reference in the > > > *private.apache.geode.org > > > <http://private.apache.geode.org/>* list. I believe that list is only > > for > > > *PMC* members, so I don't (and won't) have access to that list, am I > > > right?. > > > Sorry for the long email and the amount of questions, just trying to > make > > > sure I get things right from the very beginning :-). > > > Best regards. > > > > > > -- > > > Ju@N > > > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [DISCUSS] Apache Geode 1.7.0 release branch created
Thanks!!
On Tue, Aug 28, 2018 at 9:13 AM Nabarun Nag wrote:
> Hi Juan,
>
> GEODE-5618 as PR#2360 has been merged in to develop. The new branch has not
> yet been created hence this fix will be in 1.7.0
>
> Regards
> Nabarun Nag
>
> On Tue, Aug 28, 2018 at 12:33 AM Juan José Ramos
> wrote:
>
> > Hello team,
> >
> > Can we also include GEODE-5618 in the next release?. The pull request has
> > been approved already, it just needs to be merged.
> > Best regards.
> >
> >
> > On Mon, Aug 27, 2018 at 11:45 PM Bruce Schuchardt <
> bschucha...@pivotal.io>
> > wrote:
> >
> > > great! thanks
> > >
> > >
> > > On 8/27/18 1:42 PM, Nabarun Nag wrote:
> > > > I completely agree. Once the branch is created, it will undergo all
> > > > compatibility and upgrade tests.
> > > >
> > > > The commit that you have mentioned will be reverted in 1.7.0, as well
> > as
> > > > any related commits
> > > >
> > > > Regards
> > > > Nabarun Nag
> > > >
> > > > On Mon, Aug 27, 2018 at 1:34 PM Bruce Schuchardt <
> > bschucha...@pivotal.io
> > > >
> > > > wrote:
> > > >
> > > >> I don't think it's as easy as doing a rebase. Someone added the 1.8
> > > >> version to Version.java and we need to revert that. We also need to
> > see
> > > >> if it's being used anywhere for backward-compatibility. If it's in
> > use
> > > >> those changes need to be examined and probably undone on the branch
> if
> > > >> they're targeting 1.7 peers/clients.
> > > >>
> > > >> On 8/27/18 12:11 PM, Nabarun Nag wrote:
> > > >>> @Bruce those changes were done when 1.7.0 release process was
> > > >> in-progress,
> > > >>> and a release branch was already created. But we stopped that
> process
> > > mid
> > > >>> way. This happened in May 2018.
> > > >>> We are planning to rebase the 1.7.0 brach with the current develop
> > > pretty
> > > >>> soon.
> > > >>>
> > > >>> Regards
> > > >>> Nabarun
> > > >>> On Mon, Aug 27, 2018 at 12:02 PM Bruce Schuchardt <
> > > >> bschucha...@pivotal.io>
> > > >>> wrote:
> > > >>>
> > > >>>> It looks like we've cut a 1.7.0 release branch that says its
> 1.8.0.
> > > Is
> > > >>>> that intentional?
> > > >>>>
> > > >>>>
> > > >>>> private static final byte GEODE_180_ORDINAL =95;
> > > >>>>
> > > >>>> public static final VersionGEODE_180 =
> > > >>>>new Version("GEODE","1.8.0", (byte)1, (byte)8, (byte)0,
> > > >>>> (byte)0,GEODE_180_ORDINAL);
> > > >>>>
> > > >>>>
> > > >>>> On 8/27/18 9:50 AM, Sai Boorlagadda wrote:
> > > >>>>> After reading through the weekend, validating against CN as a
> > > >>>>> fallback should be acceptable and dont have any further concerns
> > > >>>>> with default JDK's implementation as expressed[1].
> > > >>>>>
> > > >>>>> Planning to merge GEODE-5594 today and following with GEODE-5338.
> > > >>>>>
> > > >>>>> Sai
> > > >>>>> [1]
> > > >>>>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/906540e18fa6f85fc77c88c28fc74a61402471d2eed4ee9dab4813c9@%3Cdev.geode.apache.org%3E
> > > >>>>> On Fri, Aug 24, 2018 at 5:07 PM Sai Boorlagadda <
> > > >>>> sai.boorlaga...@gmail.com>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> Regarding GEODE-5594, though the current implementation is good
> > and
> > > >>>> needed
> > > >>>>>> more coverage.
> > > >>>>>>
> > > >>>>>> While adding tests to cover negative cases, I found something
> > about
> > > >>>> JDK's
> > > >>>>>> default implementation of
> > > >>>>>> hostname validation which I am not happy about and so it needs a
> > > >&g
Re: [DISCUSS] Apache Geode 1.7.0 release branch created
t;>>>> mean? > >>>>>>> > >>>>>>> Is there anything else that needs to go into 1.7? > >>>>>>> > >>>>>>> It seems like the best we all can do is to review Sai's PRs. Is > that > >>>>>>> correct? > >>>>>>> > >>>>>>> On Wed, Aug 22, 2018 at 10:59 AM, Jens Deppe > >>>> wrote: > >>>>>>>> I'd also like to include GEODE-5619 > >>>>>>>> > >>>>>>>> On Tue, Aug 21, 2018 at 3:59 PM Xiaojian Zhou > >>>> wrote: > >>>>>>>>> +1 > >>>>>>>>> > >>>>>>>>> The release will be a great one with so many historical bugs > fixed. > >>>>>>>>> > >>>>>>>>> Today I tried to use IJ to build and run with latest build.gradle > >> and > >>>>>>>>> recent moved test packages, it worked. So this refactoring is > also > >>>>>>>> success. > >>>>>>>>> On Tue, Aug 21, 2018 at 3:52 PM, Anthony Baker < > aba...@pivotal.io> > >>>>>>>> wrote: > >>>>>>>>>> I most definitely agree! > >>>>>>>>>> > >>>>>>>>>> Anthony > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On Aug 21, 2018, at 2:26 PM, Dan Smith > >> wrote: > >>>>>>>>>>> I think we do want to wait for GEODE-5615 (DistributedTest > OOMEs) > >>>>>>> and > >>>>>>>>>>> GEODE-5601 (AcceptanceTest port conflicts) to be fixed before > >>>>>>> cutting > >>>>>>>>> the > >>>>>>>>>>> new 1.7 branch. It would be better if we don't create a release > >>>>>>>> branch > >>>>>>>>>> from > >>>>>>>>>>> a point where we have these systematic issues with our > pipeline. > >>>>>>>>>>> > >>>>>>>>>>> -Dan > >> > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: Finer Grained Security When Invoking Methods in OQL
Hello Dave, Thanks for the heads up, spelling errors fixed (at least that one :-/). Cheers!. On Mon, Jun 25, 2018 at 4:34 PM Dave Barnes wrote: > Juan, > Nice work - you've obviously given this plenty of thought. I'm not > qualified to comment on the technical aspects of your proposal, but as a > proofreader I noticed that there are a couple of occurrences of > "invokation" that should be spelled "invocation". > Dave > > > > On Mon, Jun 25, 2018 at 2:52 AM, Ju@N wrote: > > > Hello all, > > > > The current approach used to authorize methods during OQL execution seems > > to be way too restrictive, I've drafted a proposal to change the current > > behavior and allow further customization: > > https://cwiki.apache.org/confluence/display/GEODE/ > > Customizable+Security+When+Invoking+Methods+in+OQL > > > > > > Please have a look and provide your feedback on the proposal. > > Best regards. > > > > -- > > Ju@N > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Reviews and Conflicts in Pull Requests
Hello Bruce, Sounds good to me, thanks for the quick reply!. Best regards. On Tue, May 22, 2018 at 4:44 PM Bruce Schuchardt <bschucha...@apache.org> wrote: > I don't think the "single commit" notion is even a good one for an > original PR. I've discussed this with other people and we think it's > okay for the PR to have multiple commits if they serve different > purposes such as renaming variables or restructuring code before > attacking a problem. Likewise it's okay to address review comments in > an additional commit. The PR template just addresses the initial PR > being a single commit anyway. > > That being said, if the changes required by reviewers are so substantial > that the PR becomes difficult to review it should be pulled and a new PR > should be created. Otherwise who's going to bother with it? > > Regarding conflicts, that's between the contributor and the committer. > A PR that picks up conflicts due to other peoples' work isn't invalid. > Most conflicts are easy to resolve but if it's too gnarly would the > contributor want someone else to do the work and maybe mess up? > > > On 5/22/18 2:41 AM, Ju@N wrote: > > Hello geode devs, > > > > The Geode Wiki has a lot of useful information, not only about the > > usage/internal architecture of Geode, but also explanations and > guidelines > > about how Code Contributions [1] should be made. However, some common > cases > > are not explained in detail and, as such, contributors (and also > reviewers) > > might have a hard time trying to deal with these situations in an > > *standardize* manner. In particular, I'm referring to the following two > > scenarios, which can happen for every single pull request: > > > > - *Reviewers request a change: *do contributors have to make the > changes > > and just add another commit to the original pull request?. Do > > contributors have to make changes and squash everything in one single > > commit (executing a *push --force *afterwards)?. Do contributors > have to > > close the pull request and open a new one with everything squashed?. > > > > > > - *Pull Request looks good but, after a while without being merged, > > somebody else makes a change in develop, causing the original pull > request > > to become conflictive with the current develop branch: *should the > > committer merging the pull request solve these conflicts as well?. Do > > contributors have to solve the conflict locally?, if so, do they > have to > > add a new commit to the pull request or squash everything in one > single > > commit (executing a *push --force *afterwards)?. > > > > It would be good if we could add these details to the Wiki so everyone > > knows how to proceed whenever they hit this situations (git commands will > > also be helpful). Would anyone be able to add this to the Wiki?, or reply > > to this thread so the approved/recommended process is documented > somewhere?. > > Cheers. > > > > [1]: > https://cwiki.apache.org/confluence/display/GEODE/Code+contributions > > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [PROPOSAL]: configure pdx command mandatory flags
> > MXBeanIntrospector.java:117) > > at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2( > > MXBeanIntrospector.java:54) > > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM( > > MBeanIntrospector.java:237) > > at com.sun.jmx.mbeanserver.PerInterface.invoke( > > PerInterface.java:138) > > at com.sun.jmx.mbeanserver.MBeanSupport.invoke( > > MBeanSupport.java:252) > > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke( > > DefaultMBeanServerInterceptor.java:819) > > at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke( > > JmxMBeanServer.java:801) > > at javax.management.remote.rmi.RMIConnectionImpl.doOperation( > > RMIConnectionImpl.java:1468) > > at javax.management.remote.rmi.RMIConnectionImpl.access$300( > > RMIConnectionImpl.java:76) > > at javax.management.remote.rmi.RMIConnectionImpl$ > > PrivilegedOperation.run(RMIConnectionImpl.java:1309) > > at javax.management.remote.rmi.RMIConnectionImpl. > > doPrivilegedOperation(RMIConnectionImpl.java:1401) > > at javax.management.remote.rmi.RMIConnectionImpl.invoke( > > RMIConnectionImpl.java:829) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at sun.reflect.NativeMethodAccessorImpl.invoke( > > NativeMethodAccessorImpl.java:62) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at sun.rmi.server.UnicastServerRef.dispatch( > > UnicastServerRef.java:357) > > at sun.rmi.transport.Transport$1.run(Transport.java:200) > > at sun.rmi.transport.Transport$1.run(Transport.java:197) > > at java.security.AccessController.doPrivileged(Native Method) > > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) > > at sun.rmi.transport.tcp.TCPTransport.handleMessages( > > TCPTransport.java:568) > > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0( > > TCPTransport.java:826) > > at sun.rmi.transport.tcp.TCPTransport$ > > ConnectionHandler.lambda$run$0(TCPTransport.java:683) > > at java.security.AccessController.doPrivileged(Native Method) > > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run( > > TCPTransport.java:682) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > ThreadPoolExecutor.java:1149) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > > > Currently the configure pdx command only allows the user to configure the > > ReflectionBasedAutoSerializer class, which requires a decision on whether > > the check-portability flag is set as true or false. This, in turn, > enforces > > the usage of either the > > portable-auto-serializable-classes(check-portability=true) or > > auto-serializable-classes(check-portability=false) so that the command > > knows exactly which of ReflectionBasedAutoSerializer should create. > Within > > the command we could easily create an empty > ReflectionBasedAutoSerializer, > > but it won't match any classes so it doesn't make any sense at all... > > > > The proposal is to make the flags portable-auto-serializable-classes and > > auto-serializable-classes mandatory, mutually exclusive. > > > > Please let me know your thoughts. > > > > Best regards. > > > > -- > > Ju@N > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: sun.misc.ObjectInputFilter in rel/v1.4.0
Hello Daniel, I hit this a compilation error a couple of months ago, you just need to upgrade your JDK to get it working. I remember seeing within the dev list which was the minimum JDK version required but I can't seem to find it now... either way, I know that it works correctly with 1.8.0_152 as that's the one I'm currently using. Hope this helps. Best regards. On Mon, Feb 12, 2018 at 2:18 PM, Daniel Farcovich < daniel.farcov...@amdocs.com> wrote: > Hi > We are facing a compilation error on geode-core/src/main/java/org/ > apache/geode/internal/ObjectInputStreamFilterWrapper.java > sun.misc.ObjectInputFilter can't be found since we use java 8. > Has anyone encountered this in rel/v1.4.0? > > Thanks, > Daniel > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, > > you may review at https://www.amdocs.com/about/email-disclaimer < > https://www.amdocs.com/about/email-disclaimer> > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: [DISCUSS]: Handling of SDTOUT/STDERR and Signals
Thanks both, Using yet another GFSH command (*export stack-traces*) when the system is already a distributed deadlock doesn't seem right :-). On the other hand, *jstack* and *jcmd* both work as expected, but my goal was to make *KILL -QUIT / KILL -3* work, it's easier and widely known. I'll keep working on it. Best regards. On Wed, Dec 20, 2017 at 3:38 PM, Kirk Lund <kl...@apache.org> wrote: > Sorry, my info on redirecting of stdout and stderr was stale (circa GemFire > 7.0). Now that the code base is an Apache project, I believe redirecting of > stdout and stderr no longer occurs. However, it does shut off the Log4j2 > ConsoleAppender to stdout when the LogWriterAppender is activated and turns > it back on when the LogWriterAppender is closed. The old code base > redirected stdout and stderr using native code which is not in Apache > Geode. > > If you want to dump stack traces, you need to use the GFSH command "export > stack-traces". > > On Wed, Dec 20, 2017 at 1:33 AM, Juan José Ramos <jra...@pivotal.io> > wrote: > > > Hello Kirk, > > > > Thanks for your reply. Please find my answers below, in line for an > easier > > follow-up. > > > > *>> By default Geode redirects stdout and stderr to the value of the > > log-file property when the Cache starts. If log-file is set to "" then it > > won't redirect them.* > > > > This doesn't seem to be true at the moment, I've tested the behavior > > before sending the email and creating the JIRA. If you start a basic > server > > with the defaults + *log-file*, the output written from a > > function/listener to stdout/stderr is lost, nothing is printed on the > > member's logs. > > > > > > *>> So we reluctantly made the code redirect stdout and stderr when > > LogWriterAppender is registered (at the same point in Cache startup where > > the behavior originally existed). If you specify a custom log4j2.xml then > > this behavior does NOT happen.* > > > > Also tested this one and it doesn't seem to be true, the result is the > same > > as before: the output written to *stderr/stdout* is lost, nothing is > > printed on the member's logs. > > > > > > *>> So a user currently has two options:* > > *>> 1) keep the old behavior,* > > *>> 2) use a custom log4j2.xml which can be specified using the Log4J 2 > > system property "log4j.configurationFile".* > > > > The attached zip file contains a reproducible scenario (you just need to > > modify *setenv.txt* and execute *reproduce.sh*) showing that both > > approaches fail, the output from *System.out.println()* and > > *System.err.println()* is lost, along with the thread dump generated by > *kill > > -3 PID*. > > Hope this is clear now, hope you can answer the questions from my > previous > > email so I can proceed with the fix. > > Best regards > > > > > > On Tue, Dec 19, 2017 at 2:56 PM, Kirk Lund <kl...@apache.org> wrote: > > > >> By default Geode redirects stdout and stderr to the value of the > log-file > >> property when the Cache starts. If log-file is set to "" then it won't > >> redirect them. > >> > >> When we changed GemFire to use Log4J2 internally, we tried to leave > stdout > >> and stderr alone such that the LogWriterAppender would append log > >> statements to the log-file and it would leave the stdout appender > >> registered and appending to stdout. This was a change in behavior from > the > >> previous version of GemFire, and the management at that time decided > that > >> it needed to maintain the old behavior, so we reluctantly made the code > >> redirect stdout and stderr when LogWriterAppender is registered (at the > >> same point in Cache startup where the behavior originally existed). > >> > >> If you specify a custom log4j2.xml then this behavior does NOT happen. > So > >> a > >> User currently has two options: 1) keep the old behavior, 2) use a > custom > >> log4j2.xml which can be specified using the Log4J 2 system property > >> "log4j.configurationFile". > >> > >> If you copy geode-core/src/main/resources/log4j2.xml for use as the > >> starting basis for your log4j2.xml then you must NOT keep this line (or > >> set > >> it to false): > >> > >> true > >> > >> That is the line that Geode uses to determine that it is using its own > >> default log4j2.xml and this enables redirecting of stdout and stderr > when > >
Re: [DISCUSS]: Handling of SDTOUT/STDERR and Signals
Hello again, Regarding my last question from the original email (unformatted jetty logging when enabling --*redirect-output*), it looks like the offending library within the *geode-pulse.war* is *commons-logging-x.x.jar.* The library is configured as *providedCompile* in *build.gradle* and, by definition, this means it should be included in the *compile classpath* but *not included in the packaged war file*. This happens because *commons-logging* is a transient dependency of another direct pulse dependency, not configured as *providedCompile; *the fix is quite simple and it just implies adding *exclude module: 'commons-logging'* to this direct dependencies. That said, two questions remain unanswered: - Any comments about the suggested approach to fix GEODE-4101 (deprecate old system properties, add new flag and set an internal system properties to ignore the old ones)?. - Anyone knows if Geode is, somehow, catching the QUIT signal and preventing the thread dump from going to the member's log file?. Can you point me to the relevant section of code or class?. Best regards. On Wed, Dec 20, 2017 at 6:33 AM, Juan José Ramos <jra...@pivotal.io> wrote: > Hello Kirk, > > Thanks for your reply. Please find my answers below, in line for an easier > follow-up. > > *>> By default Geode redirects stdout and stderr to the value of the > log-file property when the Cache starts. If log-file is set to "" then it > won't redirect them.* > > This doesn't seem to be true at the moment, I've tested the behavior > before sending the email and creating the JIRA. If you start a basic server > with the defaults + *log-file*, the output written from a > function/listener to stdout/stderr is lost, nothing is printed on the > member's logs. > > > *>> So we reluctantly made the code redirect stdout and stderr when > LogWriterAppender is registered (at the same point in Cache startup where > the behavior originally existed). If you specify a custom log4j2.xml then > this behavior does NOT happen.* > > Also tested this one and it doesn't seem to be true, the result is the same > as before: the output written to *stderr/stdout* is lost, nothing is > printed on the member's logs. > > > *>> So a user currently has two options:* > *>> 1) keep the old behavior,* > *>> 2) use a custom log4j2.xml which can be specified using the Log4J 2 > system property "log4j.configurationFile".* > > The attached zip file contains a reproducible scenario (you just need to > modify *setenv.txt* and execute *reproduce.sh*) showing that both > approaches fail, the output from *System.out.println()* and > *System.err.println()* is lost, along with the thread dump generated by *kill > -3 PID*. > Hope this is clear now, hope you can answer the questions from my previous > email so I can proceed with the fix. > Best regards > > > On Tue, Dec 19, 2017 at 2:56 PM, Kirk Lund <kl...@apache.org> wrote: > >> By default Geode redirects stdout and stderr to the value of the log-file >> property when the Cache starts. If log-file is set to "" then it won't >> redirect them. >> >> When we changed GemFire to use Log4J2 internally, we tried to leave stdout >> and stderr alone such that the LogWriterAppender would append log >> statements to the log-file and it would leave the stdout appender >> registered and appending to stdout. This was a change in behavior from the >> previous version of GemFire, and the management at that time decided that >> it needed to maintain the old behavior, so we reluctantly made the code >> redirect stdout and stderr when LogWriterAppender is registered (at the >> same point in Cache startup where the behavior originally existed). >> >> If you specify a custom log4j2.xml then this behavior does NOT happen. So >> a >> User currently has two options: 1) keep the old behavior, 2) use a custom >> log4j2.xml which can be specified using the Log4J 2 system property >> "log4j.configurationFile". >> >> If you copy geode-core/src/main/resources/log4j2.xml for use as the >> starting basis for your log4j2.xml then you must NOT keep this line (or >> set >> it to false): >> >> true >> >> That is the line that Geode uses to determine that it is using its own >> default log4j2.xml and this enables redirecting of stdout and stderr when >> the LogWriterAppender starts up. >> >> On Tue, Dec 19, 2017 at 7:11 AM, Ju@N <jujora...@gmail.com> wrote: >> >> > Hello Geode devs, >> > >> > Currently GEODE is "swallowing" all output sent to stdout and stderr by >> > default and there's no way of changing this beh
Re: Permissions to Add Content in the Wiki
Thanks Dan!. Cheers. On Wed, Dec 13, 2017 at 2:30 PM, Dan Smith <dsm...@pivotal.io> wrote: > Hi Juan, > > You should have permissions now! > > Thanks, > -Dan > > On Wed, Dec 13, 2017 at 3:33 AM, Juan José Ramos <jra...@pivotal.io> > wrote: > > > Hello team, > > > > Can I get permissions to create content on the wiki?. My username is > > *jujoramos*. > > Best regards. > > > > -- > > Juan José Ramos Cassella > > Senior Technical Support Engineer > > Email: jra...@pivotal.io > > Office#: +353 21 4238611 > > Mobile#: +353 87 2074066 > > After Hours Contact#: +1 877 477 2269 > > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT > > How to upload artifacts: > > https://support.pivotal.io/hc/en-us/articles/204369073 > > How to escalate a ticket: > > https://support.pivotal.io/hc/en-us/articles/203809556 > > > > [image: support] <https://support.pivotal.io/> [image: twitter] > > <https://twitter.com/pivotal> [image: linkedin] > > <https://www.linkedin.com/company/3048967> [image: facebook] > > <https://www.facebook.com/pivotalsoftware> [image: google plus] > > <https://plus.google.com/+Pivotal> [image: youtube] > > <https://www.youtube.com/playlist?list=PLAdzTan_ > eSPScpj2J50ErtzR9ANSzv3kl> > > > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Permissions to Add Content in the Wiki
Hello team, Can I get permissions to create content on the wiki?. My username is *jujoramos*. Best regards. -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Consistent results from gfsh commands
+1, we should provide the user with the full details about what happened when there was at least one failure. Following this line of thought, and considering the cluster configuration service, what should the commands do when the operation fails on at least one member?. Persist the changes no matter what?, don't persist the changes?, it depends on the operation (create region vs create index)?, add a new flag to *gfsh* and let the user decide?. Cheers. On Tue, Oct 31, 2017 at 2:59 PM, Jens Deppe <jensde...@apache.org> wrote: > Hi, > > I've noticed that various commands, that execute on multiple members, > return either tabulated results (one line per member - for example > CreateRegionCommand) or a single pass/fail line. In the latter case it’s > possible that information would get lost. For example if a command fails on > multiple members (but not all) then the user would not have complete > insight into where the failure occurred. > > I'd like to propose that if a command executes on multiple members it > should always return and display all the results. Alternatively, if the > command succeeds then it should simply indicate success, however if it > fails (even partially) it should return a full list of each member and the > result of the command on each of the members. > > Thoughts; comments? > > --Jens > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
Re: Geode Native Client date format
Hello Robert, The *JSONFormatter* class, used internally in Geode to handle REST requests/responses, supports only "MM/DD/" dates, with no HH:MM:SS; while PDX supports all date fields, that's the reason why you see the difference in the output. There's already a GEODE ticket created for this: GEODE-226. You might want to add yourself as watcher to get updates about the progress. Hope this helps. Best regards. On Thu, Aug 31, 2017 at 9:33 AM, Rupert St John Webster < rup...@impress-solutions.com> wrote: > Sorry, I can now see in Pulse that a dateTime format is stored as for > example: Thu Aug 31 09:29:25 BST 2017 > > The problem is when the REST API does a GET for the value stored, it is > being returned as DD/MM/ only > > Any ideas? Thanks > > From: Rupert St John Webster > Sent: 31 August 2017 09:23 > To: dev@geode.apache.org > Subject: Geode Native Client date format > > Hi, > > The PDX writer has a writeDate method. This results in a PDX date in > DD/MM/ format. I’m looking for HH.MM.SS.fff as well but can’t find a > way to do that? > > Kind regards, > -- Juan José Ramos Cassella Senior Technical Support Engineer Email: jra...@pivotal.io Office#: +353 21 4238611 Mobile#: +353 87 2074066 After Hours Contact#: +1 877 477 2269 Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT How to upload artifacts: https://support.pivotal.io/hc/en-us/articles/204369073 How to escalate a ticket: https://support.pivotal.io/hc/en-us/articles/203809556 [image: support] <https://support.pivotal.io/> [image: twitter] <https://twitter.com/pivotal> [image: linkedin] <https://www.linkedin.com/company/3048967> [image: facebook] <https://www.facebook.com/pivotalsoftware> [image: google plus] <https://plus.google.com/+Pivotal> [image: youtube] <https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
