API (Recommanded way) to get heap and disk usage for cluster nodes
Hello Geode Dev and users We have a requirement to constantly monitor the resource utilization (Disk and Heap usage) for the cluster nodes from external processes. Seeking help to understand the recommended way (or APIs available ) to get this in a separate process...We need to trigger some actions/custom logic if it goes above some threshold.. Thanks Steve.
Re: geode docker question
Here is a command that actually runs.. You need to be in the Geode directory, the run the command below to start everything. docker run -it -v $(pwd):/apache-geode openjdk:8 sh -c "apache-geode/bin/gfsh -e 'start locator --name=Locator1' -e 'start server --name=Server1'" -c "" Thanks, Mark On 7/7/20, 10:42 AM, "Mark Hanson" wrote: Hi Barry, Are you looking for something like this? docker run -it -v $(pwd):/apache_geode openjdk:8 sh -c "apache-geode/bin/gfsh -e "start locator --name=Locator1" -e "start server --name=Server1" ^ shared location ^ not sure this path is right but you get the idea. Thanks, Mark On 7/7/20, 10:29 AM, "Barry Barrios" wrote: Do you have Apache Geode examples using docker? Is there a way to run a docker container that automatically creates locator and server by specifying some parameters without typing them in the gfsh shell prompt? Also, same for deploying jars, is there a way to automatically deploy jars when running docker? I was browsing through your apache geode examples github repo to see if there were examples but didn't find what I was looking for. Best, Barry
Re: Fate of master branch
Since the branch proposed for deletion is the default branch in geode-examples, you will need to file an ASF INFRA ticket to change that default. This is a great discussion thread, but ASF will require a [VOTE] thread to be cited. I am concerned about keeping it easy for someone who has just cloned geode to identify the most stable branch for their purpose. Before, they could always be assured `git checkout master` would give the flagship release. Now, new users will be immediately forced into some daunting detective work to sift through hundreds of haphazard tags and branches (a task even veteran committers frequently fail). I would strongly encourage an aggressive cleanup of unhelpful branches and tags, as Jacob proposed last month, before getting rid of the latest_release concept. On 7/7/20, 8:24 AM, "Blake Bender" wrote: Just to follow up on this: I've filed GEODE-8335 (https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8335data=02%7C01%7Conichols%40vmware.com%7Cb3a06863e89e4b9f06bb08d82289cebe%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637297322561614965sdata=bXkoPZBLk3CR29lfV6AP5OcnjPWUvjuhSk3AZAXC7Do%3Dreserved=0) to track, respectfully (cowardly __ ) deferring to individuals who regularly contribute to the various Geode repos to handle it as they see fit. I'll take care of the several Geode Native associated repos. Thanks, Blake On 6/26/20, 12:21 PM, "Dave Barnes" wrote: +1 if we can override git’s ‘master’ default and establish ‘develop’ in its place. Otherwise renaming to ‘main’ would solve the problem with the negative connotations. NB: Mark, I think the 3-vote convention is just for back-ports to the release-branch. I don’t think of it as applying to a general discussion like this one. > On Jun 26, 2020, at 9:42 AM, Anthony Baker wrote: > > By just do it, I assume you mean: > > - Contact delete master where not needed > - Rename master to main when needed > - Contact INFRA to change the default branch > - Update README and CI jobs as needed > > Across *all* geode repos. > > > Anthony > > >> On Jun 26, 2020, at 9:38 AM, Mark Hanson wrote: >> >> +1 to delete. No good reason to keep it that I know of. I think I am the third +1 with no -1s so just do it. >> >> Thanks, >> Mark >> >>> On Jun 26, 2020, at 9:13 AM, Alexander Murmann wrote: >>> >>> +1 to deleting. Given we tag everything properly on the other branches, I >>> don't see the need for it either. >>> >>> On Fri, Jun 26, 2020 at 9:03 AM Alberto Bustamante Reyes >>> wrote: >>> +1 for deleting master branch. An also for updating the wiki page about branching that Alberto pointed out. De: Bruce Schuchardt Enviado: viernes, 26 de junio de 2020 17:37 Para: dev@geode.apache.org Asunto: Re: Fate of master branch Let's just delete it. I need to do that in my own repos as well. On 6/26/20, 8:05 AM, "Blake Bender" wrote: Apologies if this has been addressed already and I missed it. In keeping with other OSS projects, I believe it’s time we did something about removing the insensitive term master from Geode repositories. One choice a lot of projects appear to be going with is a simple rename from master • main. In our own case, however, master isn’t really in use for anything vital. We track releases with a tag and a branch to backport fixes to, and the develop branch is the “source of truth” latest-and-greatest version of the code. We could thus simply delete master with no loss I’m aware of. Any opinions? Thanks, Blake >> >
Re: geode docker question
Great (and timely) question Barry! And great answers from Mark and John. The question is timely because we recently addressed some similar needs in the development of the new Dockerized acceptance tests for the new TLS SNI features. I've taken a stab at a tutorial that leverages some of those ideas to hopefully answer Barry's question here: https://cwiki.apache.org/confluence/display/GEODE/Using+the+Official+Geode+Docker+Image It's not perfect (at all) but hopefully it'll give you a few techniques you can start using! On Tue, Jul 7, 2020 at 3:21 PM John Blum wrote: > Hi Barry- > > Have a look at this... > > > https://docs.spring.io/spring-boot-data-geode-build/1.3.x/reference/html5/#geode-docker > > I recently put this together as part of the SBDG 1.3 GA release. It > contains references to other pertinent documentation as well. > > There aren't any pre-canned Docker Images with a Locator/Locators or > Server/Servers running, unfortunately. > > However, I intend to tackle that concern as part of the STDG project ( > https://github.com/spring-projects/spring-test-data-geode). See Issue > #19 (https://github.com/spring-projects/spring-test-data-geode/issues/19) > by myself and David Turanski. > > Some early experimentation has already taken place. > > Regards, > John > > > > > From: Barry Barrios > Sent: Tuesday, July 7, 2020 10:26 AM > To: dev@geode.apache.org > Subject: geode docker question > > Do you have Apache Geode examples using docker? Is there a way to run a > docker container that automatically creates locator and server by > specifying some parameters without typing them in the gfsh shell prompt? > Also, same for deploying jars, is there a way to automatically deploy jars > when running docker? I was browsing through your apache geode examples > github repo to see if there were examples but didn't find what I was > looking for. > > Best, > Barry >
Re: geode docker question
Hi Barry- Have a look at this... https://docs.spring.io/spring-boot-data-geode-build/1.3.x/reference/html5/#geode-docker I recently put this together as part of the SBDG 1.3 GA release. It contains references to other pertinent documentation as well. There aren't any pre-canned Docker Images with a Locator/Locators or Server/Servers running, unfortunately. However, I intend to tackle that concern as part of the STDG project (https://github.com/spring-projects/spring-test-data-geode). See Issue #19 (https://github.com/spring-projects/spring-test-data-geode/issues/19) by myself and David Turanski. Some early experimentation has already taken place. Regards, John From: Barry Barrios Sent: Tuesday, July 7, 2020 10:26 AM To: dev@geode.apache.org Subject: geode docker question Do you have Apache Geode examples using docker? Is there a way to run a docker container that automatically creates locator and server by specifying some parameters without typing them in the gfsh shell prompt? Also, same for deploying jars, is there a way to automatically deploy jars when running docker? I was browsing through your apache geode examples github repo to see if there were examples but didn't find what I was looking for. Best, Barry
Re: geode docker question
Hi Barry, Are you looking for something like this? docker run -it -v $(pwd):/apache_geode openjdk:8 sh -c "apache-geode/bin/gfsh -e "start locator --name=Locator1" -e "start server --name=Server1" ^ shared location ^ not sure this path is right but you get the idea. Thanks, Mark On 7/7/20, 10:29 AM, "Barry Barrios" wrote: Do you have Apache Geode examples using docker? Is there a way to run a docker container that automatically creates locator and server by specifying some parameters without typing them in the gfsh shell prompt? Also, same for deploying jars, is there a way to automatically deploy jars when running docker? I was browsing through your apache geode examples github repo to see if there were examples but didn't find what I was looking for. Best, Barry
geode docker question
Do you have Apache Geode examples using docker? Is there a way to run a docker container that automatically creates locator and server by specifying some parameters without typing them in the gfsh shell prompt? Also, same for deploying jars, is there a way to automatically deploy jars when running docker? I was browsing through your apache geode examples github repo to see if there were examples but didn't find what I was looking for. Best, Barry
Re: about Liberica JDK
Liberica is an OpenJDK distribution like AdoptOpenJDK, Oracle, RedHat, Amazon, Azul, etc. I have yet to find a behavioral difference between the distributions—it’s mostly about ease of acquiring binaries and LTS support. Just for simplicity I would prefer to use a single JDK distribution across versions in our CI pipelines. Anthony > On Jul 7, 2020, at 2:21 AM, Alberto Bustamante Reyes > wrote: > > Hi devs, > > I have seen in develop branch this commit that changes openjdk by Liberica > JDK ( > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fpull%2F5312data=02%7C01%7Cbakera%40vmware.com%7C92aa27c47a164abdd6b808d822573a33%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637297105319626477sdata=X3V4jcQ7DWEVtJuUaXOQy%2F0l3xri9g4a0%2BfflLmui1g%3Dreserved=0 > ), although it was reverted later so I suppose there are still issues to be > solved. > > I didn't know Liberica and I'm curious about the change. Why is this change > being implemented? > > BR/ > > Alberto B. >
Re: Fate of master branch
Just to follow up on this: I've filed GEODE-8335 (https://issues.apache.org/jira/browse/GEODE-8335) to track, respectfully (cowardly __ ) deferring to individuals who regularly contribute to the various Geode repos to handle it as they see fit. I'll take care of the several Geode Native associated repos. Thanks, Blake On 6/26/20, 12:21 PM, "Dave Barnes" wrote: +1 if we can override git’s ‘master’ default and establish ‘develop’ in its place. Otherwise renaming to ‘main’ would solve the problem with the negative connotations. NB: Mark, I think the 3-vote convention is just for back-ports to the release-branch. I don’t think of it as applying to a general discussion like this one. > On Jun 26, 2020, at 9:42 AM, Anthony Baker wrote: > > By just do it, I assume you mean: > > - Contact delete master where not needed > - Rename master to main when needed > - Contact INFRA to change the default branch > - Update README and CI jobs as needed > > Across *all* geode repos. > > > Anthony > > >> On Jun 26, 2020, at 9:38 AM, Mark Hanson wrote: >> >> +1 to delete. No good reason to keep it that I know of. I think I am the third +1 with no -1s so just do it. >> >> Thanks, >> Mark >> >>> On Jun 26, 2020, at 9:13 AM, Alexander Murmann wrote: >>> >>> +1 to deleting. Given we tag everything properly on the other branches, I >>> don't see the need for it either. >>> >>> On Fri, Jun 26, 2020 at 9:03 AM Alberto Bustamante Reyes >>> wrote: >>> +1 for deleting master branch. An also for updating the wiki page about branching that Alberto pointed out. De: Bruce Schuchardt Enviado: viernes, 26 de junio de 2020 17:37 Para: dev@geode.apache.org Asunto: Re: Fate of master branch Let's just delete it. I need to do that in my own repos as well. On 6/26/20, 8:05 AM, "Blake Bender" wrote: Apologies if this has been addressed already and I missed it. In keeping with other OSS projects, I believe it’s time we did something about removing the insensitive term master from Geode repositories. One choice a lot of projects appear to be going with is a simple rename from master • main. In our own case, however, master isn’t really in use for anything vital. We track releases with a tag and a branch to backport fixes to, and the develop branch is the “source of truth” latest-and-greatest version of the code. We could thus simply delete master with no loss I’m aware of. Any opinions? Thanks, Blake >> >
Odg: Odg: negative ActiveCQCount
Hi,
Thank you all for the response!
What I got for now is that when I register CQ on the one server it
processMessage to the other server through FilterProfile and in the message
opType is REGISTER_CQ.
In fromData() method in FilterProfile.java states following:
if (isCqOp(this.opType)) {
this.serverCqName = in.readUTF();
if (this.opType == operationType.REGISTER_CQ || this.opType ==
operationType.SET_CQ_STATE) {
this.cq = CqServiceProvider.readCq(in);
}
And there it register cq on the other server and not increment cqActiveCount,
which is ok as redundancy is 0. But it now has on both server different
instances of ServerCqImpl for the same cq. The ones created with constructor
with arguments at the execute cq and another with empty constructor while
deserializing the message with opType=REGISTER_CQ. For me this is ok as we need
to follow up all changes on both servers as maybe some fullfil CQ condition on
the other server. Correct me if I'm wrong.
But when it is going to close cq it executes it on both server, for me it is ok
that what is started should be closed. But in the close method we have
decrement if stateBeforeClosing is RUNNING. So it will be good if we can
somehow process cq_state of this ServerCqImpl instance which is created by
constructor with parameters before closing this created by deserialization.
Does anyone has an idea how to get this? Or some other idea to solve this issue?
BR,
Mario
Šalje: Kirk Lund
Poslano: 1. srpnja 2020. 19:52
Prima: dev@geode.apache.org
Predmet: Re: Odg: negative ActiveCQCount
Yeah, https://issues.apache.org/jira/browse/GEODE-8293 sounds like a
statistic decrement bug for activeCqCount. Somewhere, each Server is
decrementing it once too many times.
You could find the statistics class containing activeCqCount and try adding
some debugging log statements or even add some breakpoints for debugger if
it's easily reproduced.
On Wed, Jul 1, 2020 at 5:52 AM Mario Kevo wrote:
> Hi Kirk, thanks for the response!
>
> I just realized that I wrongly describe the problem as I tried so many
> case. Sorry!
>
> We have system with two servers. If the redundancy is 0 then we have
> properly that on the first server is activeCqCount=1 and on the second is
> activeCqCount=0.
> After close CQ we got on first server activeCqCount=0 and on the second is
> activeCqCount=-1.
> gfsh>show metrics --categories=query
> Cluster-wide Metrics
>
> Category | Metric | Value
> | | -
> query| activeCQCount| -1
> | queryRequestRate | 0.0
>
>
> In case we set redundancy to 1 it increments properly as expected, on both
> servers by one. But when cq is closed we got on both servers
> activeCqCount=-1. And show metrics command has the following output
> gfsh>show metrics --categories=query
> Cluster-wide Metrics
>
> Category | Metric | Value
> | | -
> query| activeCQCount| -1
> | queryRequestRate | 0.0
>
> What I found is that when server register cq on one server it send message
> to other servers in the system with opType=REGISTER_CQ and in that case it
> creates new instance of ServerCqImpl on second server(with empty
> constructor of ServerCqImpl). When we close CQ there is two different
> instances on servers and it closed both of them, but as they are in RUNNING
> state before closing, it decrements activeCqCount on both of them.
>
> BR,
> Mario
>
>
> Šalje: Kirk Lund
> Poslano: 30. lipnja 2020. 19:54
> Prima: dev@geode.apache.org
> Predmet: Re: negative ActiveCQCount
>
> I think *show metrics --categories=query* is showing you the query stats
> from DistributedSystemMXBean (see
> ShowMetricsCommand#writeSystemWideMetricValues). DistributedSystemMXBean
> aggregates values across all members in the cluster, so I would have
> expected activeCQCount to initially show a value of 2 after you create a
> ServerCQImpl in 2 servers. Then after closing the CQ, it should drop to a
> value of 0.
>
> When you create a CQ on a Server, it should be reflected asynchronously on
> the CacheServerMXBean in that Server. Each Server has its own
> CacheServerMXBean. Over on the Locator (JMX Manager), the
> DistributedSystemMXBean aggregates the count of active CQs in
> ServerClusterStatsMonitor by invoking
> DistributedSystemBridge#updateCacheServer when the CacheServerMXBean state
> is federated to the Locator (JMX Manager).
>
> Based on what I see in code and in the description on GEODE-8293, I think
> you might want to see if increment has a problem instead of decrement.
>
> I don't see anything that would limit the activeCQCount to only count the
> CQs on primaries. So, I would expect redundancy=1 to result in a value of
> 2. Does anyone else have different info about this?
>
> On Tue, Jun 30, 2020 at 5:31 AM Mario Kevo wrote:
>
> > Hi geode-dev,
> >
> > I have a question
about Liberica JDK
Hi devs, I have seen in develop branch this commit that changes openjdk by Liberica JDK ( https://github.com/apache/geode/pull/5312 ), although it was reverted later so I suppose there are still issues to be solved. I didn't know Liberica and I'm curious about the change. Why is this change being implemented? BR/ Alberto B.
