Also, it matters if your region is partitioned or not. Only partitioned regions support parallel queues.
I’m not sure about the gfsh behavior you’re seeing when you set parallel=true. Anthony > On Nov 2, 2018, at 11:12 AM, John Blum <[email protected]> wrote: > > Hi Rob- > > Sorry, just noticed your follow up where you figured out the hot CPU issue > caused by your batch-time-interval of 0. > > As for setting parallel to true, it may be you have other severs currently > running that do not have parallel currently configured to true, which I think > (don't remember for sure) might be considered incompatible Region > configuration by the cluster. All Regions with the same name hosted by > members in the cluster must have compatible configuration. > > Check whether you may already have existing members with the same Region (by > name) that may not currently have the associated AEQ set with parallel=true. > > Cheers, > -j > > > On Fri, Nov 2, 2018 at 11:07 AM, John Blum <[email protected] > <mailto:[email protected]>> wrote: > Hi Rob- > > Please see here [1]. Default is 5 Threads. Also see [2]. > > Alternatively, look at [3] (<async-event-queue dispatcher-threads=".." ..>) > or [4] (create async-event-queue --dispatcher-threads). > > The threads should not be having an impact, but you can experiment with the > number to find out. > > -j > > > [1] > http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueue.html#getDispatcherThreads-- > > <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueue.html#getDispatcherThreads--> > [2] > http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueueFactory.html#setDispatcherThreads-int- > > <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueueFactory.html#setDispatcherThreads-int->[3] > > http://geode.apache.org/docs/guide/17/reference/topics/cache_xml.html#async-event-queue > > <http://geode.apache.org/docs/guide/17/reference/topics/cache_xml.html#async-event-queue> > [4] > http://geode.apache.org/docs/guide/17/tools_modules/gfsh/command-pages/create.html#topic_ryz_pb1_dk > > <http://geode.apache.org/docs/guide/17/tools_modules/gfsh/command-pages/create.html#topic_ryz_pb1_dk> > > > On Fri, Nov 2, 2018 at 9:56 AM, Rob Shepherd <[email protected] > <mailto:[email protected]>> wrote: > Thank you Nabarun, > > Having started to pull out my config to send to you, I noticed the following > in my cache.xml: > > <async-event-queue > id="expiry-event-queue" > parallel="false" > enable-batch-conflation="false" > disk-synchronous="true" > forward-expiration-destroy="true" > —-> batch-time-interval=“0" > batch-size="1" > >... > > …Which is a copy from the example here: > https://github.com/apache/geode-examples/blob/master/async/scripts/start.gfsh > <https://github.com/apache/geode-examples/blob/master/async/scripts/start.gfsh> > create async-event-queue --id=example-async-event-queue \ > --parallel=false \ > --enable-batch-conflation=false \ > --batch-size=1 \ > --batch-time-interval=0 \ > --listener=org.apache.geode_examples.async.ExampleAsyncEventListener > > > I pondered on “batch-time-interval” and set it to 1000 and it has fixed the > issue. > > I think I understand what this parameter is for and so a delay here would be > tolerable. > > > > I have another question, if I set parallel=“true” the gfsh start server > command hangs and I have to kill the new server process and the gfsh > launcher. > > it is not important to me now, but i would like to evaluate this at some > point and so i’ll happily try and debug the cause. > > Thanks > > Rob > > > > >> On 2 Nov 2018, at 16:22, Nabarun Nag <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Rob, >> >> We will look into this, meanwhile could you please elaborate on what >> configuration is Apache Geode running, like how many servers, how many AEQs >> regions etc, what workload is it running. >> >> Thank you >> Nabarun Nag >> >>> On Nov 2, 2018, at 8:37 AM, Rob Shepherd <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi, >>> >>> I’m using Geode (1.7.0) locally (OSX) and on a server. (Linux Arm64) >>> >>> On both I’m seeing maxed out CPUs. >>> >>> I’ve profiled it locally on a dormant server instance (no application >>> activity) and the Async Queue routines are the highest contributor to CPU >>> activity by a long stretch. >>> >>> <PastedGraphic-1.png> >>> >>> Back Traces - Method Total Time [%] Total Time [µs] Total Time >>> (CPU) Samples >>> org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueue.getHeadKey() >>> 100.00% 2829377292 2829377292 8639 >>> .org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueue.getCurrentKey() >>> 0.00% 0 0 8639 >>> ..org.apache.geode.internal.ca >>> <http://org.apache.geode.internal.ca/>che.wan.serial.SerialGatewaySenderQueue.peekAhead() >>> 0.00% 0 0 8263 >>> ...org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueue.peek() >>> 1.21% 9906300 9906300 8180 >>> ....org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue() >>> 100.00% 1144915695 1144915695 5 >>> .....org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderEventProcessor.run() >>> 0.00% 0 0 5 >>> >>> >>> How can I determine if this is a problem with my setup or if it is a bug? >>> >>> A supposition: I notice that there are multiple instances of a thread >>> named after my Async Event queue ID >>> >>> Event Processor for GatewaySender_AsyncEventQueue_expiry-event-queue.1 >>> Event Processor for GatewaySender_AsyncEventQueue_expiry-event-queue.2 >>> Event Processor for GatewaySender_AsyncEventQueue_expiry-event-queue.3 >>> Event Processor for GatewaySender_AsyncEventQueue_expiry-event-queue.4 >>> >>> Are there supposed to be 4? are they interfering with each other >>> (wait/notify) on an empty queue? >>> >>> Thanks for any insight >>> >>> Rob >> > > > > > -- > -John > john.blum10101 (skype) > > > > -- > -John > john.blum10101 (skype)
