Hello Tim,
Sorry to come back to you.
I think I have understood the purpose of the 3 stores (persistent, memory and
temp).
But, I remind you that I used TomEE 7.0.4 plus flavor with ActiveMQ 5.14.5 with
the following settings:
- no activemq.xml file
- BrokerXmlConfig =
broker:(tcp://0.0.0.0:61616)?useJmx=false&persistent=false
In the TomEE server statup log, I was able to see:
01-Nov-2017 14:16:38.142 INFO [ActiveMQFactory start and checkpoint]
org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter Using
Persistence Adapter: MemoryPersistenceAdapter
So, my understanding is that by having set the persistence property to false,
ActiveMQ runtime set the PersistenceAdapter to a MemoryPersistenceAdapter: is
my understanding right or wrong (eg persistence=false => set the
PersistenceAdapter to a MemoryPersistenceAdapter) ?
In such case, the persistence storage is not used, right ?
As I don't have any activemq.xml: no value has been set for memory and temp
store limits.
Nevertheless, I have 2 WARNING messages:
01-Nov-2017 14:16:38.140 WARNING [ActiveMQFactory start and checkpoint]
org.apache.activemq.broker.BrokerService.checkMemorySystemUsageLimit s Memory
Usage for the Broker (1024mb) is more than the maximum available for the JVM:
726 mb - resetting to 70% of maximum available: 508 mb
....
01-Nov-2017 14:16:38.360 WARNING [ActiveMQFactory start and checkpoint]
org.apache.activemq.broker.BrokerService.checkUsageLimit Temporary Store limit
is 51200 mb (current store usage is 0 mb). The data directory: /opt/gemalto/bin
only has 16571 mb of usable space. - resetting to maximum available disk space:
16571 mb
Our goal is to remove these WARNING messages.
If we use an activemq.xml file with the following content:
<systemUsage>
<systemUsage sendFailIfNoSpace="true">
<memoryUsage>
<memoryUsage limit="128 mb"/>
</memoryUsage>
</systemUsage>
</systemUsage>
The 2 WARNING messages disappears but the drawback of this solution is that we
have to customize TomEE for that (add some spring jars) and we want to avoid
that.
So, according to this test (usage of actvemq.xml and setting only the
memoryUsage limit), we are able to remove these WARNING messages. There is a
TomEE jira already submitted for setting a memoryUsage limit in the
BrokerXmlConfig.
This is why my understanding is the following:
- setting the persistence property to false, activemq runtime set the
PersistenceAdapter to a MemoryPersistenceAdapter, right ?
- having the persistence set to false:
* if no store limit is used (eg no activemq.xml), the
activemq runtime will compute these limits thank to some checking for Memory
usage and Temporary usage (as we can see in the WARNING logs), right ?
* if we set only a Memory usage limit as we did by using
the above activemq.xml, then the tempUsage limit is set to unlimited, right ?
* in a symmetric way, if we set only a tempUsage limit,
then the memoryUsage limit is set to unlimited, right ?
Best Regards.
-----Original Message-----
From: tbai...@gmail.com [mailto:tbai...@gmail.com] On Behalf Of Tim Bain
Sent: jeudi 16 novembre 2017 06:39
To: ActiveMQ Users <users@activemq.apache.org>
Subject: Re: ]: RE: : Re: WARNING log message to be removed
Francois,
I think you're misunderstanding a few things, so let me try to explain more
fully and then respond to your questions.
In ActiveMQ, there are three places messages are stored.
1. The persistence store. All persistent messages are stored here. May
be backed by a SQL database, a NoSQL database (KahaDB or the deprecated
LevelDB), or memory.
2. The memory store. All non-persistent messages are stored here.
3. The temp store. Used for overflow storage of the memory store.
If you're setting the broker's persistent property to false, then all messages
received by the broker will have their persistent flag overridden to false, and
so all messages will be stored in the memory store, and you're correct that the
storeUsage limit is irrelevant in that case.
However, that has absolutely nothing to do with using a
MemoryPersistenceAdapter, which is how you choose to store persistent messages
in memory. (Not in the memory store, in memory, in a series of HashMaps.
MemoryPersistenceAdapter != memory store.) I'm not aware of any reason to both
set the broker's persistent property to false and also use a
MemoryPersistenceAdapter, since the former disables any possibility of use of
the latter.
If you're using a MemoryPersistenceAdapter, then all three limits are used.
The storeUsage controls how many persistent messages are stored in the
memory-backed persistence store, the memoryUsage controls how many messages are
stored in the memory store, and the tempUsage controls how many messages are
stored in the temp store. All three of them ALSO use the "physical limit" in
that if you try to put more messages into any of the stores than your physical
RAM or disk can hold, the broker will do awful things and may crash. The point
of the limits is to prevent this from happening by making the broker refuse to
store messages at a software level rather than allowing it to result in a
hardware failure that might kill the broker process, but you're in charge of
setting those limits appropriately and if you screw it up, the broker can't
prevent you from shooting your own foot. Caveat emptor.
No, the broker doesn't compute any limits at all for you; setting no limit for
any of the three effectively means that the software limit is infinite (no
limit), so you'll hit the "physical limit" (with all the negatives that come
with that) if anything. It's your responsibility to protect your broker process
from doing that, and if you choose not to, that's on you.
You've probably realized this already from what I wrote above, but to your
final question, no, if you use a MemoryPersistenceAdapter (without setting the
broker's persistent property to false, presumably, otherwise what's the
point?), you control everything by setting all three limits, not just
memoryUsage and tempUsage.
Tim
On Mon, Nov 13, 2017 at 2:37 PM, COURTAULT Francois <
francois.courta...@gemalto.com> wrote:
> Hello Tim,
>
> Try to summarize:
> - If we use a MemoryPersistenceAdapter that is to say that the
> Broker persistent property is set to false, only the memoryUsage and
> tempUsage limit could be set.
> The storeUsage limit is useless in this case, right ?
> - If we use a MemoryPersistenceAdapter and if one of the
> limits, either the memoryUsage or the tempUsage, is used, the other
> will use the physical limit.
> That is to say:
> * if we only set the memoryUsage limit then the
> tempUsage limit will be the physical available disk space.
> * if we only set the tempUsage limit then the
> memoryUsage limit will be the physical available memory (maybe limited
> by the Xmx JVM settings).
> - If we use a MemoryPersistenceAdapter and none of the limits
> are set (neither memoryUsage nor the tempUsage limits) then the
> Broker will compute these limits
> using some formula, right ?
> - If we use a MemoryPersistenceAdapter, the way to control
> everything is to set the memoryUsage and tempUsage limits, right ?
>
> Best Regards.
>
> -----Original Message-----
> From: tbai...@gmail.com [mailto:tbai...@gmail.com] On Behalf Of Tim
> Bain
> Sent: jeudi 9 novembre 2017 06:17
> To: ActiveMQ Users <users@activemq.apache.org>
> Subject: Re: [++SPAM++]: RE: : Re: WARNING log message to be removed
>
> On Wed, Nov 8, 2017 at 7:29 AM, COURTAULT Francois <
> francois.courta...@gemalto.com> wrote:
>
> > Hello Tim,
> >
> > Understood for the temp store: it looks like a swap mechanism in
> > case there is no more room in memory, right ?
> >
>
> Just to clarify: it's a swap mechanism in case there is no more room
> in *the memory store*, not in memory overall. This is not for paging
> generic heap content out to disk if the JVM runs out of heap, etc.,
> and I believe it's also not for paging out content if you overfill a
> persistence store that happens to be backed by memory (though I've not
> investigated that specific question). But yes, you understand the overall
> concept.
>
>
> > Let's say that, in our configuration, we only set the memoryUsage
> > limit without specifying any storeUsage and tempUsage limit .
> > In such case, does it exist a formula to calculate the tempUsage
> > limit ( eg tempUsage limit=f(memoryUsage limit) )?
> >
>
> In that case, no limit is applied, so you're allowed to write content
> to disk right up to the point where the write fails (maybe
> spectacularly) because there is no more available space on the volume.
>
>
> > I ask you the question, because if we only set the memoryUsage limit
> > to
> > 128 mb (no storeUsage and tempUsage limits are set), the WARNING
> > message disappears with the same available space on the disk.
> >
>
> Yes, the warning message disappears because you've taken the safety
> off and told the broker that you're on your own, so it allows you to
> shoot yourself in the foot if you so choose without trying to help you
> avoid doing so. The net effect for you is the same (the broker won't
> prevent you from filling the disk in either configuration), but in one
> case it tells you that there's a potential for doing that, and in the
> other it doesn't attempt to warn you.
>
> Tim
> ________________________________
> This message and any attachments are intended solely for the
> addressees and may contain confidential information. Any unauthorized
> use or disclosure, either whole or partial, is prohibited.
> E-mails are susceptible to alteration. Our company shall not be liable
> for the message if altered, changed or falsified. If you are not the
> intended recipient of this message, please delete it and notify the sender.
> Although all reasonable efforts have been made to keep this
> transmission free from viruses, the sender will not be liable for
> damages caused by a transmitted virus.
>
________________________________
This message and any attachments are intended solely for the addressees and
may contain confidential information. Any unauthorized use or disclosure,
either whole or partial, is prohibited.
E-mails are susceptible to alteration. Our company shall not be liable for the
message if altered, changed or falsified. If you are not the intended recipient
of this message, please delete it and notify the sender.
Although all reasonable efforts have been made to keep this transmission free
from viruses, the sender will not be liable for damages caused by a transmitted
virus.
