[CONF] Apache ActiveMQ > Why do KahaDB log files remain after cleanup
Title: Message Title
Paul Gale edited a page
Why do KahaDB log files remain after cleanup
Clean-up of unreferenced un-referenced KahaDB journal log files files data-.log will occur every 30 seconds by default. If a data file is in-use it will not be cleaned up. ...
It contains a pending message for a destination or durable topic subscription
It contains an ack ACK for a message which is in an in-use data file - the ack ACK cannot be removed as a recovery would then mark the message for redelivery
The journal references a pending transaction
It is a journal file, and there may be a pending write to it
The trace level logging of the the org.apache.activemq.store.kahadb.MessageDatabase class provides insight into the cleanup process and will allow you to determine why a given data file is considered in-use and as a result, not a candidate for cleanup. To debug, add the following (or similar) to your your log4j.properties file (if needed):
Code Block
log4j.appender.kahadb=org.apache.log4j.RollingFileAppender
log4j.appender.kahadb.file=${activemq.base}/data/kahadb.log
log4j.appender.kahadb.maxFileSize=1024KB
log4j.appender.kahadb.maxBackupIndex=5
log4j.appender.kahadb.append=true
log4j.appender.kahadb.layout=org.apache.log4j.PatternLayout
log4j.appender.kahadb.layout.ConversionPattern=%d [%-15.15t] %-5p %-30.30c{1} - %m%n
log4j.logger.org.apache.activemq.store.kahadb.MessageDatabase=TRACE, kahadb
Either restart AMQ ActiveMQ and let the cleanup process run (give it a minute or two for example) or alternatively apply this logging configuration to a running broker via JMX. The "The Broker" MBean exposes an operation called "called reloadLog4jProperties" in JMX that can be used to tell the broker to reload its log4j.properties. Often its enough to apply this logging configuration for 2-5 minutes and then analyze the broker's log file. Examine the log file and look for cleanup of the data files. The process starts with the complete set of known data files and queries the index on a per destination basis to prune this list. Anything that remains is a candidate for cleanup. The trace logging gives the destination and the log file numbers that remain candidates for removal as it iterates through the index. At some point you'll hit a destination and the number of data file ids will suddenly drop because that destination references them. It could be a DLQ or an offline durable subscriber. In any event, the logging will help you pinpoint the destinations that are hogging disk space. Here is a quick sample:
Code Block
TRACE | Last update: 164:41712, full gc candidates set: [86, 87, 163, 164] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after first tx:164:41712, [86, 87, 163] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:A, [86, 87, 163] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:1:B, [86, 87, 163] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:D, [86, 87, 163] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:E, [86, 87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:H, [86, 87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:I, [86, 87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates after dest:0:J, [87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
TRACE | gc candidates: [87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
DEBUG | Cleanup removing the data files: [87] | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal Checkpoint Worker
We get one candidate, data-87.log from the existing set of journal data files [86, 87, 163, 164]. There is a current transaction using 164, destination (Queue named named E) '0\:E' has some messages in 163, destination '0:I' has messages in in 86 and and 87 is unreferencedun-referenced. In this case, there must be some long standing unacked un-acknowledged messages or a very slow consumer on destination '0:I'.
Info
The '0:' prefix is shorthand for a queue, '1:' for a topic
...
.
...
Example: dest:1:B
...
refers to a topic named B.
... Non-persistent messages Similar for non-persistent messages that are not stored in your configured KahaDB persistence adapter but get swapped to temp
[CONF] Apache ActiveMQ > Configuring Wire Formats
Title: Message Title Paul Gale edited a page Change comment: Fix formatting for table contents. Configuring Wire Formats ... OpenWire is the default wire format used by ActiveMQ. It provides a highly efficient binary format for high speed messaging. OpenWire options can be configured on a JMS client's connection URI or on a broker's transport bind URI. Option Default Value Description cacheEnabled true Should commonly repeated values be cached so that less marshaling occurs? cacheSize 1024 When cacheEnabled=true then this parameter is used to specify the number of values to be cached. maxInactivityDuration 3 The maximum inactivity duration (before which the socket is considered dead) in milliseconds. On some platforms it can take a long time for a socket to die. Therefore allow the broker to kill connections when they have been inactive for the configured period of time. Used by some transports to enable a keep alive heart beat feature. Inactivity monitoring is disabled when set to a value <= 0. maxInactivityDurationInitalDelay 1 The initial delay before starting inactivity checks. Yes, the word 'Inital' is supposed to be misspelled like that. maxFrameSize MAX_LONG Maximum allowed frame size. Can help help prevent OOM DOS attacks. sizePrefixDisabled false Should the size of the packet be prefixed before each packet is marshaled? stackTraceEnabled true Should the stack trace of exception that occur on the broker be sent to the client? tcpNoDelayEnabled true Does not affect the wire format, but provides a hint to the peer that TCP_NODELAY should be enabled on the communications Socket. tightEncodingEnabled true Should wire size be optimized over CPU usage? Warning title Use the Correct Prefix! Wire format options must have the prefix wireFormat. to take effect, e.g., wireFormat.maxInactivityDuration=1. Options missing this prefix will be ignored. ... View page • Like Stop watching space • Manage notifications This message was sent by Atlassian Confluence 5.8.17
[CONF] Apache ActiveMQ > Configuring Wire Formats
Title: Message Title Paul Gale edited a page Change comment: Change table column label. Configuring Wire Formats ... OpenWire is the default wire format used by ActiveMQ. It provides a highly efficient binary format for high speed messaging. OpenWire options can be configured on a JMS client's connection URI or on a broker's transport bind URI. Parameter NameOption Default Value Description cacheEnabled true Should commonly repeated values be cached so that less marshaling occurs? cacheSize 1024 When cacheEnabled=true then this parameter is used to specify the number of values to be cached. maxInactivityDuration 3 The maximum inactivity duration (before which the socket is considered dead) in milliseconds. On some platforms it can take a long time for a socket to die. Therefore allow the broker to kill connections when they have been inactive for the configured period of time. Used by some transports to enable a keep alive heart beat feature. Inactivity monitoring is disabled when set to a value <= 0. maxInactivityDurationInitalDelay 1 The initial delay before starting inactivity checks. Yes, the word 'Inital' is supposed to be misspelled like that. maxFrameSize MAX_LONG Maximum allowed frame size. Can help help prevent OOM DOS attacks. sizePrefixDisabled false Should the size of the packet be prefixed before each packet is marshaled? stackTraceEnabled true Should the stack trace of exception that occur on the broker be sent to the client? tcpNoDelayEnabled true Does not affect the wire format, but provides a hint to the peer that TCP_NODELAY should be enabled on the communications Socket. tightEncodingEnabled true Should wire size be optimized over CPU usage? ... View page • Like Stop watching space • Manage notifications This message was sent by Atlassian Confluence 5.8.17
[CONF] Apache ActiveMQ > Configuring Wire Formats
Title: Message Title
Paul Gale edited a page
Change comment: Reformatting changes.
Configuring Wire Formats
...
Parameter Name
Default Value
Description
cacheEnabled
true
Should commonly repeated values be cached so that less marshaling occurs?
cacheSize
1024
When cacheEnabled=true then this parameter is used to specify the number of values to be cached.
maxInactivityDuration
3
The maximum inactivity duration (before which the socket is considered dead) in milliseconds. On some platforms it can take a long time for a socket to die. Therefore allow the broker to kill connections when they have been inactive for the configured period of time.Used by some transports to enable a keep alive heart beat feature. Inactivity monitoring is disabled when set to a value <= 0.
maxInactivityDurationInitalDelay
1
The initial delay before starting inactivity checks. Yes, the word 'Inital' is supposed to be misspelled like that.
maxFrameSize
MAX_LONG
Maximum allowed frame size. Can help help prevent OOM DOS attacks.
sizePrefixDisabled
false
Should the size of the packet be prefixed before each packet is marshaled?
stackTraceEnabled
true
Should the stack trace of exception that occur on the broker be sent to the client?
tcpNoDelayEnabled
true
Does not affect the wire format, but provides a hint to the peer that that TCP nodelay _NODELAY should be enabled on the communications Socket.
tightEncodingEnabled
true
Should wire size be optimized over CPU usage?
... Example Configurations Java:
Code Block
language
java
ActiveMQConnectionFactory cf =
...
new ActiveMQConnectionFactory("tcp://localhost:61616?wireFormat.cacheEnabled=false=false");
Spring:
Code Block
language
xml
View page
•
Like
Stop watching space
•
Manage notifications
This message was sent by Atlassian Confluence 5.8.17
[CONF] Apache ActiveMQ > TCP Transport Reference
Title: Message Title Paul Gale edited a page Change comment: Formatting and typos. TCP Transport Reference ... Option Name Default Value Description backlog 5000 Specifies the maximum number of connections waiting to be accepted by the transport server socket. closeAsync true If true the socket close call happens asynchronously. This parameter should be set to false for protocols like STOMP, that are commonly used in situations where a new connection is created for each read or write. Doing so ensures the socket close call happens synchronously. A synchronous close prevents the broker from running out of available sockets owing to the rapid cycling of connections. connectionTimeout 3 If >=1 the value sets the connection timeout in milliseconds. A value of 0 denotes no timeout. Negative values are ignored. daemon false If true the transport thread will run in daemon mode. Set this parameter to true when embedding the broker in a Spring container or a web container to allow the container to shut down correctly. diffServ 0 (Client only) The preferred Differentiated Services traffic class to be set on outgoing packets, as described in RFC 2475. Valid integer values: [0,64]. Valid string values: EF, AF[1-3][1-4] or CS[0-7]. With JDK 6, only works when the JVM uses the IPv4 stack. To use the IPv4 stack set the system property java.net.preferIPv4Stack=true. Note: it's invalid to specify both diffServ and typeOfService since at the same time as they share the same position in the TCP/IP packet headers. dynamicManagement false If true the TransportLogger can be managed by JMX. ioBufferSize 8 * 1024 Specifies the size of the buffer to be used between the TCP layer and the OpenWire layer where wireFormat based marshaling occurs. jmxPort 1099 (Client Only) Specifies the port that will be used by the JMX server to manage the TransportLoggers. This should only be set in an , via URI, by either a client ( producer or consumer ) since a broker will already create a as the broker creates its own JMX server. Specifying an alternate JMX port is useful for developers that test a broker and client on the same machine and need to control both via JMX. In such cases a different JMX port is needed. keepAlive false If true, enables TCP KeepAlive on the broker connection to prevent connections from timing out at the TCP level. This should not be confused with KeepAliveInfo messages as used by the InactivityMonitor. logWriterName default Sets the name of the org.apache.activemq.transport.LogWriter implementation to use. Names are mapped to classes in the resources/META-INF/services/org/apache/activemq/transport/logwriters directory. maximumConnections Integer.MAX_VALUE The maximum number of sockets allowed for this broker. minmumWireFormatVersion 0 The minimum remote wireFormat version that will be accepted (note the misspelling). A value of 0 denotes no checking of the remote wireFormat version.Note: if Info Note: when the remote wireFormat version is lower than the configured minimum acceptable version an exception is will be thrown and the connection attempt is will be refused. A value of 0 denotes no checking of the remote wireFormat version. socketBufferSize 64 * 1024 Sets the size, in bytes, for the accepted socket's read and write buffers. soLinger Integer.MIN_VALUE Sets the socket's option soLinger when the value is > -1. When the value is set to -1 the soLinger socket option is disabled. Since ActiveMQ 5.6.0. soTimeout 0 Sets the socket's read timeout in milliseconds. A value of 0 denotes no timeout. soWriteTimeout 0 Sets the socket's write timeout in milliseconds. If the socket write operation does not complete before the specified timeout, the socket will be closed. A value of 0 denotes no timeout. stackSize 0 Set the stack size of the transport's background reading thread. Must be specified in multiples of 128K. A value of 0 indicates that this parameter is ignored. startLogging true If true the TransportLogger object of the Transport stack will initially write messages to the log. This parameter is only used when trace is ignored unless trace=true. tcpNoDelay false If true the socket's option TCP_NODELAY is set. This disables Nagle's algorithm for small packet transmission. threadName N/A When this parameter is specified the name of the thread is modified during the invocation of a transport. The remote address is appended so that a call stuck in a transport method will have the
[CONF] Apache ActiveMQ > Total Ordering
Title: Message Title Paul Gale edited a page Change comment: Fix formatting. Total Ordering Sometimes it can be useful to ensure that every topic consumer sees messages arriving on the topic in exactly the same order. Normally we guarrentee the broker will guarantee the order of all messages sent by the same producer; however due to multi-threading . However, owing to the broker's use of multiple threads and asynchronous processing, the messages from different producers could arrive in different consumers in different orders. e.g. For example, if we have producers producers P and and Q sending messages such that at about the same time time P sends P1, P2, P3 and and Q sends Q1, Q2. Then 2 different consumeres Therefore, two different consumers could see messages arrive in this order...the following order: Code Block consumer1: P1 P2 Q1 P3 Q2 consumer2: P1 Q1 Q2 P2 P3 i.e. each producers messages are in order; but In this example each producer's message's are in self-relative order. However, the streams of messages across producers can get intermixed. Total orderingOrdering of a destination in ActiveMQ ensures that each consumer will see the same total order on that topic. This has a peformance performance cost, since more greater synchronization is required, but . This can be useful - e.g. particularly in times when you want to implement , particularly when very fast optimistic transactions are required. With total ordering the messages would arrive like this...: Code Block consumer1: P1 P2 Q1 P3 Q2 consumer2: P1 P2 Q1 P3 Q2 Configuring Total Ordering Enable the the on the Per Destination Policies. Here's an example Code BlockpolicyEntries>policyMap> View page • Like Stop watching space • Manage notifications This message was sent by Atlassian Confluence 5.8.4
[CONF] Apache ActiveMQ > KahaDB
Title: Message Title Paul Gale edited a page Change comment: Font formatting KahaDB ... Code Block language xml KahaDB Properties property nameProperty default valueDefault Comments archiveCorruptedIndex false If true, corrupted indexes found at startup will be archived (not deleted). archiveDataLogs false If true, will move a message data log to the archive directory instead of deleting it. checkForCorruptJournalFiles false If true, will check for corrupt journal files on startup and try and recover them. checkpointInterval 5000 Time (ms) before check-pointing the journal. checksumJournalFiles true Create a checksum for a journal file. The presence of a checksum is required in order for the persistence adapter to be able to detect corrupt journal files. Before ActiveMQ 5.9.0: the default is false. cleanupInterval 3 The interval (in ms) between consecutive checks that determine which journal files, if any, are eligible for removal from the message store. An eligible journal file is one that has no outstanding references. compactAcksAfterNoGC 10 From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls how many store GC cycles must be completed with no other files being cleaned up before the compaction logic is triggered to possibly compact older acknowledgements spread across journal files into a new log file. The lower the value set the faster the compaction may occur which can impact performance if it runs to often. compactAcksIgnoresStoreGrowth false From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls whether compaction is run when the store is still growing or if it should only occur when the store has stopped growing (either due to idle or store limits reached). If enabled the compaction runs regardless of the store still having room or being active which can decrease overall performance but reclaim space faster. concurrentStoreAndDispatchQueues true Enable the dispatching of Queue messages to interested clients to happen concurrently with message storage. concurrentStoreAndDispatchTopics false Enable the dispatching of Topic messages to interested clients to happen concurrently with message storage Warning Enabling this property is not recommended. directory activemq-data The path to the directory to use to store the message store data and log files. directoryArchive null Define the directory to move data logs to when they all the messages they contain have been consumed. enableAckCompaction true From ActiveMQ 5.14.0: this setting controls whether the store will perform periodic compaction of older journal log files that contain only Message acknowledgements. By compacting these older acknowledgements into new journal log files the older files can be removed freeing space and allowing the message store to continue to operate without hitting store size limits. enableIndexWriteAsync false If true, the index is updated asynchronously. enableJournalDiskSyncs true Ensure every journal write is followed by a disk sync (JMS durability requirement). Warning This property is deprecated as of ActiveMQ 5.14.0. From ActiveMQ 5.14.0: see journalDiskSyncStrategy. journalDiskSyncStrategy always From ActiveMQ 5.14.0: this setting configures the disk sync policy. The list of available sync strategies are (in order of decreasing safety, and increasing performance): always Ensure every journal write is followed by a disk sync (JMS durability requirement). This is the safest option but is also the slowest because it requires a sync after every message write. This is equivalent to the deprecated property enableJournalDiskSyncs=true. periodic The disk will be synced at set intervals (if a write has occurred) instead of after every journal write which will reduce the load on the disk and should improve throughput. The disk will also be synced when rolling over to a new journal file. The default interval is 1 second. The default interval offers very good performance, whilst being safer than never disk syncing, as data loss is limited to a maximum of 1 second's worth. See journalDiskSyncInterval to change the frequency of disk syncs. never A sync will never be explicitly called and it will be up to the operating system to flush to disk. This is equivalent to setting the deprecated property enableJournalDiskSyncs=false. This is the fastest option but is the least safe as there's no guarantee as to
[CONF] Apache ActiveMQ > KahaDB
Title: Message Title Paul Gale edited a page Change comment: Minor typos etc. KahaDB KahaDB is a file based persistence database that is local to the message broker that is using it. It has been optimised optimized for fast persistence and . It is the the default storage mechanism from since ActiveMQ 5.4 onwards. KahaDB uses less file descriptors and provides faster recovery than its predecessor, the AMQ Message Store. Configuration You can configure ActiveMQ to To use KahaDB for its persistence adapter - like belowas the broker's persistence adapter configure ActiveMQ as follows (example): Code Block language xml ... > ... KahaDB Properties property name default value Comments archiveCorruptedIndex false If enabled true, corrupted indexes found at startup will be archived (not deleted). archiveDataLogs false If enabled true, will move a message data log to the archive directory instead of deleting it. checkForCorruptJournalFiles false If enabled true, will check for corrupted Journal corrupt journal files on startup and try and recover them. checkpointInterval 5000 Time (ms) before check-pointing the journal. checksumJournalFiles true Create a checksum for a journal file - to enable checking for corrupted journals. The presence of a checksum is required in order for the persistence adapter to be able to detect corrupt journal files. Before ActiveMQ 5.9.0: the default is false. cleanupInterval 3 Time The interval (in ms) before checking for a discarding/moving message data logs that are no longer usedbetween consecutive checks that determine which journal files, if any, are eligible for removal from the message store. An eligible journal file is one that has no outstanding references. compactAcksAfterNoGC 10 From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls how many store GC cycles must be completed with no other files being cleaned up before the compaction logic is triggered to possibly compact older acknowledgements spread across journal files into a new log file. The lower the value set the faster the compaction may occur which can impact performance if it runs to often. compactAcksIgnoresStoreGrowth false From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls whether compaction is run when the store is still growing or if it should only occur when the store has stopped growing (either due to idle or store limits reached). If enabled the compaction runs regardless of the store still having room or being active which can decrease overall performance but reclaim space faster. concurrentStoreAndDispatchQueues true Enable the dispatching of Queue messages to interested clients to happen concurrently with message storage. concurrentStoreAndDispatchTopics false Enable the dispatching of Topic messages to interested clients to happen concurrently with message storage Warning Enabling this property is not recommended. directory activemq-data The path to the directory to use to store the message store data and log files. directoryArchive null Define the directory to move data logs to when they all the messages they contain have been consumed. enableAckCompaction true From ActiveMQ 5.14.0: this setting controls whether the store will perform periodic compaction of older journal log files that contain only Message acknowledgements. By compacting these older acknowledgements into new journal log files the older files can be removed freeing space and allowing the message store to continue to operate without hitting store size limits. enableIndexWriteAsync false If set, will asynchronously write indexes true, the index is updated asynchronously. enableJournalDiskSyncs true Ensure every journal write is followed by a disk sync (JMS durability requirement). Warning This property is deprecated as of ActiveMQ 5.14.0 , see journalDiskSyncStrategy for version . From ActiveMQ 5.14.0 and newer : see journalDiskSyncStrategy. journalDiskSyncStrategy always From ActiveMQ 5.14.0: this setting configures the disk sync policy. The default strategy is set to always.list of available sync strategies are (in order of decreasing safety, and increasing performance): always Ensure every journal write is followed by a disk sync (JMS durability requirement). This is the safest option but is also the slowest because it requires a sync after every message write. This is equivalent to the deprecated property
[CONF] Apache ActiveMQ > KahaDB
Title: Message Title Paul Gale edited a page KahaDB ... You can configure ActiveMQ to use KahaDB for its persistence adapter - like below: Code Block language xml ... ... (as of 5 12.0) This setting configures how the broker will try to preallocate the journal files when a new journal file is needed. The default allocation strategy sets the file length, but does not populate it with any data. The 'os_kernel_copy' strategy delegates the preallocation to the Operating System. The 'zeros' strategy configures ActiveMQ to do the preallocation by writing 0x00 to all of the positions in the journal file. (as of ) This the broker will preallocate the in ' ' ' ' using ' ' note , stick with property name default value Comments directory activemq-data the path to the directory to use to store the message store data and log files indexDirectory If set, configures where the KahaDB index files (db.data and db.redo) will be stored. If not set, the index files are stored in the directory specified by the 'directory' attribute. Note Available as of ActiveMQ 5.10 storeOpenWireVersion 11 Determines the version of OpenWire commands that are marshalled to the KahaDB journal. In versions prior to 5.12.0 this value defaulted to v6. Some features of the broker depend on information stored in the OpenWire commands from newer protocol revisions and these may not work correctly if the store version is set to a lower value. KahaDB stores from broker versions greater than 5.9.0 will in many cases still be readable by the broker but will cause the broker to continue using the older store version meaning newer features may not work as intended. For KahaDB stores that were created in versions prior to v5.9.0 it will be necessary to manually set the store version to v6 in order to start a broker without error. indexWriteBatchSize 1000 number of indexes written in a batch indexCacheSize 1 number of index pages cached in memory enableIndexWriteAsync false if set, will asynchronously write indexes journalMaxFileLength 32mb a hint to set the maximum size of the message data logs enableJournalDiskSyncs true ensure archiveCorruptedIndex false If enabled, corrupted indexes found at startup will be archived (not deleted). archiveDataLogs false If enabled, will move a message data log to the archive directory instead of deleting it. checkForCorruptJournalFiles false If enabled, will check for corrupted Journal files on startup and try and recover them. checkpointInterval 5000 Time (ms) before check-pointing the journal. checksumJournalFiles true Create a checksum for a journal file - to enable checking for corrupted journals. Before ActiveMQ 5.9.0: the default is false. cleanupInterval 3 Time (ms) before checking for a discarding/moving message data logs that are no longer used. compactAcksAfterNoGC 10 From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls how many store GC cycles must be completed with no other files being cleaned up before the compaction logic is triggered to possibly compact older acknowledgements spread across journal files into a new log file. The lower the value set the faster the compaction may occur which can impact performance if it runs to often. compactAcksIgnoresStoreGrowth false From ActiveMQ 5.14.0: when the acknowledgement compaction feature is enabled this value controls whether compaction is run when the store is still growing or if it should only occur when the store has stopped growing (either due to idle or store limits reached). If enabled the compaction runs regardless of the store still having room or being active which can decrease overall performance but reclaim space faster. concurrentStoreAndDispatchQueues true Enable the dispatching of Queue messages to interested clients to happen concurrently with message storage. concurrentStoreAndDispatchTopics false Enable the dispatching of Topic messages to interested clients to happen concurrently with message storage Warning Enabling this property is not recommended. directory activemq-data The path to the directory to use to store the message store data and log files. directoryArchive null Define the directory to move data logs to when they all the messages they contain have been consumed. enableAckCompaction true From ActiveMQ 5.14.0: this setting
[CONF] Apache ActiveMQ > Per Destination Policies
Title: Message Title Paul Gale edited a page Change comment: Sorted options alphabetically Per Destination Policies ... Common Property Default Value Description producerFlowControladvisoryForConsumed true If true the broker will throttle (flow-control) the producer. Throttling is achieved either by withholding the producer's ACK or by raising a javax.jms.ResourceAllocationException (that's propagated back to the client) when local resources e.g., memory and/or storage, have been exhausted. If false excess messages will be written to the message store to prevent memory exhaustion. However, when the message store reaches capacity the producer will be throttled until resources are freedfalse Send an advisory message when a message is consumed by a client. advisoryForDelivery false Send an advisory message when a message is sent to a client. advisoryForFastProducers false Send an advisory message if a producer is deemed fast. advisoryForSlowConsumers false Send an advisory message if a consumer is deemed slow. advisoryWhenFull false Send an advisory message when a limit (memory, store, temp disk) is full. cursorMemoryHighWaterMark 70 The percentage (%) threshold applied either to the / or the destination's memoryLimit (when defined) which when exceeded will cause the destination's cursor to either block or write to disk. enableAudit true When true the broker will track duplicate messages. Duplicates can happen for non-persistent messages during failover. useCachegcInactiveDestinations true If true persistent messages are cached for fast retrieval from storefalse Garbage collect inactive destinations. inactiveTimoutBeforeGC 5000 The timeout (in ms) after which a destination is considered inactive. includeBodyForAdvisory false Includes the body of the original message that triggered the advisory as part of the dataStructure field in the advisory message (where applicable). Normally the message body is cleared. maxBrowsePageSize 400 The maximum number of messages to page in from the store at one time for a browser. maxPageSize 200 The maximum number of messages to page in from the store at one time. Increase this value to improve performance for queue destination's that contain grouped messages that are consumed by multiple concurrent consumers. maxBrowsePageSizemaxDestinations 400 The -1 (v5.12) If 0 or greater, sets the maximum number of messages to page in from the store at one time for a browserdestinations that can be created. This parameter is intended to limit the number of hierarchical destinations that can be created under a wildcard destination. memoryLimit n/a The memory limit (in bytes) of the destination's cursor. This memory limit is subordinate to the system level memory limit, as specified by the / attribute. There is no default for this value; it simply acts as a child to the overall broker memory until the broker memory is exhausted. Note: when this limit is specified the destination's cursorMemoryHighWaterMark will be applied against it and not the /> memory limit. minimumMessageSize 1024 For non-serialized messages (embedded broker) - the assumed size of the message used for memory usage calculation. Serialized messages use the serialized size as the basis for the memory calculation. cursorMemoryHighWaterMarkprioritizedMessages 70 The percentage (%) threshold applied either to the / or the destination's memoryLimit (when defined) which when exceeded will cause the destination's cursor to either block or write to disk. false Persist message priority information. producerFlowControl true If true the broker will throttle (flow-control) the producer. Throttling is achieved either by withholding the producer's ACK or by raising a javax.jms.ResourceAllocationException (that's propagated back to the client) when local resources e.g., memory and/or storage, have been exhausted. If false excess messages will be written to the message store to prevent memory exhaustion. However, when the message store reaches capacity the producer will be throttled until resources are freed. slowConsumerStrategy null Sets the strategy for handling slow consumers. See abortSlowConsumerStrategy. storeUsageHighWaterMark 100 The percentage (%) threshold of the / store limit which when exceeded causes a send to block. prioritizedMessagesuseCache false Persist message priority information. advisoryForConsumed false Send an advisory message when a message is consumed by a client.
[CONF] Apache ActiveMQ > Pluggable storage lockers
Title: Message Title Paul Gale edited a page Change comment: Updated shared file locker definition Pluggable storage lockers With As of the 5.7.0 release of ActiveMQ the choice of storage locking mechanism, as used by a persistence adapter is , has been made pluggable. This feature is only applies meaningful to brokers configured in a shared storage master/slave topology. Prior to release 5.7.0 the storage locking mechanism (and thus master election) was dictated by the choice of persistence adapter. With the KahaDB persistence adapter, for example, the storage locking mechanism was based on a shared file lock. Similarly, the JDBC persistence adapter used a database backed storage lock. Now that the choice of storage lock locker is divorced from that of the persistence adapter one can mix and match combinations of the two. Storage locker pluggability is made possible because by the Locker interface that all pluggable lockers must implement the Locker interface. This interface makes it easy to implement your own a custom storage locker when you have special that meets local requirements. Of course, everyEvery persistence adapter still , however, has its own default locker which works as before. ... Every locker must implement the Locker interface. The locker interface has the following properties: Property Name Default Value Description lockAcquireSleepInterval 1 Delay The polling interval (in milliseconds) between lock acquire attempts. failIfLocked false Should the broker start fail immediately if lock cannot be obtainedthe lock is not immediately available. When true slave brokers will not start. Persistence Adapters Every persistence adapter (or any other broker service that wants wishes to use locks) needs to must implement the Lockable interface. This adds interface has the following properties: Property Name Default Value Description useLock true can be used to turn off locking if necessary Should the persistence adapter use the configured locker. Intended primarily for use during development to temporarily disable the use of the locker without having to remove its configuration. lockKeepAlivePeriod 0 If bigger than 0, time period The duration (in milliseconds) to keep the lock alive, when greater than 0. Existing Lockers Shared File Locker The Shared File Locker is the default locker for the KahaDB persistence adapter. It locks a file to ensure that only the broker holding the lock (the master) is granted access to the message store. Example: Code Block language xml = "target/activemq-data" lockKeepAlivePeriod="1"> This locker implements the keepAlive method from The lockKeepAlivePeriod attribute is not applicable to versions of ActiveMQ older than 5.9.0 onwards so there's no point in using lockKeepAlivePeriod settings on older versions than ActiveMQ 5.9.0. . Warning title Consequences of lockKeepAlivePeriod = 0 For this locker lockKeepAlivePeriod should be greater than 0.This period is the frequency with which the master broker makes lock keep alive calls. When lockKeepAlivePeriod = 0 slave brokers are still unable to obtain the file lock. However, if some third party modifies the lock file (either modification or deletion) the master broker will not detect the change. Therefore a slave broker's next attempt (per its configured lockAcquireSleepInterval) to obtain the file lock will succeed. When this happens there will be two master brokers in the cluster. This situation will result in message store corruption! When lockKeepAlivePeriod is greater than 0, the master broker will make a lock keep alive call every lockKeepAlivePeriod milliseconds. Therefore the master broker will detect any lock file changes when it makes its next keep alive call. Upon detecting said change the master broker will demote itself to a slave broker. Info Note that as of ActiveMQ 5.9.0 the KahaDB persistence adapter can also use the Lease Database Locker (see below). Database Locker The Database Locker is the default locker for the JDBC persistence adapter. It locks a database table in a transaction to ensure that only single resource is used. Example: Code Block language xml The Database Locker uses its keepAlive method to ensure the broker still holds the lock. You can set the keep alive period using the lockKeepAlivePeriod property. The default period is 3 ms. If a broker fails to acquire the lock on the database, it will retry every lockAcquireSleepInterval milliseconds. This locker opens a JDBC transaction against a
[CONF] Apache ActiveMQ > TCP Transport Reference
Title: Message Title Paul Gale edited a page Change comment: Added missing entries, sorted options alphabetically, introduced consistent formatting and reworded some definitions. TCP Transport Reference ... The TCP transport allows clients to connect to a remote ActiveMQ broker using a TCP socket. These configuration options are can be used to tune the underlying TCP transport , and can be configured on a on either the client-side using the JMS client's connection URI string or on a Brokers broker's transport bind connector URI. The options below should be prefixed with transport.* when used on the server-side; for example, on a broker's transport connector definition. On the client-side, however, the transport.* prefix must be omitted. Configuration Syntax tcp://hostname:port?key=value Transport Options minmumWireFormatVersion Tells the transport thread to run as a daemon or not. Useful to enable when embedding properly useLocalHost When . Ensures connections don't time out on TCP level. Not used by InactivityMonitor! client only) The preferred Differentiated Services traffic class to be set on outgoing packets, as described in RFC 2475. Valid integer values: [0,64). Valid string values: EF, AF[1-3][1-4] or CS[0-7]. With JDK 6, only works when the Java Runtime uses the IPv4 stack, which can be done by setting the java.net.preferIPv4Stack system property to be true. Cannot be used at the same time as the typeOfService option client Java Runtime uses , which can be done by setting the system property to be true. Cannot be used at the same time as the diffServ option When true, the TCP_NODELAY setting is enabled on the socket. Option Name Default Value Description backlog 0 The minimum version wireformat that is allowed trace false Causes all commands that are sent over the transport to be logged (to view use config of the form: log4j.logger.org.apache.activemq.transport.TransportLogger=DEBUG) daemon false 5000 Specifies the maximum number of connections waiting to be accepted by the transport server socket. closeAsync true If true the socket close call happens asynchronously. This parameter should be set to false for protocols like STOMP, that are commonly used in situations where a new connection is created for each read or write. Doing so ensures the socket close call happens synchronously. A synchronous close prevents the broker from running out of available sockets owing to the rapid cycling of connections. connectionTimeout 3 If >=1 the value sets the connection timeout in milliseconds. A value of zero denotes no timeout. Negative values are ignored. daemon false If true the transport thread will run in daemon mode. Set this parameter to true when embedding the broker in a Spring container or a web container to allow the container to shut down correctly. diffServ false When true, it causes the local machines name to resolve to "localhost". socketBufferSize 64 * 1024 Sets the socket buffer size in bytes keepAlive false 0 (Client only) The preferred Differentiated Services traffic class to be set on outgoing packets, as described in RFC 2475. Valid integer values: [0,64). Valid string values: EF, AF[1-3][1-4] or CS[0-7]. With JDK 6, only works when the JVM uses the IPv4 stack. To use the IPv4 stack set the system property java.net.preferIPv4Stack=true. It is invalid to specify both diffServ and typeOfService since they share the same position in the TCP/IP packet headers. dynamicManagement false If true the TransportLogger can be managed by JMX. ioBufferSize 8 * 1024 Specifies the size of the buffer to be used between the TCP layer and the OpenWire layer where wireFormat based marshaling occurs. jmxPort 1099 Specifies the port that will be used by the JMX server to manage the TransportLoggers. This should only be set in an URI by a client (producer or consumer) since a broker will already create a JMX server. Specifying an alternate JMX port useful for developers that test a broker and client on the same machine and need to control both via JMX. In such cases a different JMX port is needed. keepAlive false If true, enables TCP KeepAlive on the broker connection useKeepAlive true Whether or not the InactivityMonitor should send KeepAliveInfo messages on idle connections. Enabled by default. to prevent connections from timing out at the TCP level. This should not be confused with KeepAliveInfo messages as used by the InactivityMonitor. logWriterName default Sets the name of the LogWriter implementation to use. Names are mapped to classes in the
[CONF] Apache ActiveMQ Pluggable storage lockers
Paul Gale edited the page: Pluggable storage lockers With the 5.7.0 release , we introduced the change on locking strategies for of ActiveMQ the storage locking mechanism used by a persistence adapter is pluggable. This feature only applies to brokers configured in a shared storage master/slave topologies. Previously topology. Prior to release 5.7.0 the storage locking mechanism (and thus master election) was hard-coded directly in the particular store. So KahaDB had only the option to use shared file lock, while JDBC was using database dictated by the choice of persistence adapter. With the KahaDB persistence adapter, for example, the storage locking mechanism was based on a shared file lock. Similarly, the JDBC persistence adapter used a database backed storage lock. Now , that the choice of storage locking lock is separated divorced from the store, so you can implement your own locking strategies if necessary (or tune existing ones)that of the persistence adapter one can mix and match. Storage locker pluggability is made possible because all lockers must implement the Locker interface. This interface makes it easy to implement your own storage locker when you have special requirements. Of course, every store has it's persistence adapter still has its own default locker which works as before. Lockers Every locker must implement the Locker interface. So there are a couple of settings you can tune for every lockerThe locker has the following configurable parameters: Property Name Default Value Description lockAcquireSleepInterval 1 Delay interval in milliseconds between lock acquire attempts failIfLocked false Should the start fail immediately if lock cannot be obtained ... Every persistence adapter (or other broker service that wants to use locks) needs to implement the Lockable interface. You can use a few settings for every persistence adapter (that supports locking)This adds the following configurable parameters: Property Name
[CONF] Apache ActiveMQ Configurable IOException Handling
Paul Gale edited the page: Configurable IOException Handling Starting with 5.3.1, ActiveMQ provides configurable IOException handling for its file-based message stores. From version 5.5 the handler is also invoked when the JDBC persistence adapter gets a failure on getConnection(). Default IOException handler ... There are a couple of properties you can use to tune the behavior of default IOException handler. But first you need to the DefaultIOExceptionHandler. First, instantiate the exception handler as a bean. Then configure the broker to use customized handler. You can do that by instantiating a handler and setting it using the exception handler by setting the broker's ioExceptionHandler property. Example: Code Block language xml bean id=ioExceptionHandler class=org.apache.activemq.util.DefaultIOExceptionHandler property name=ignoreAllErrorsvaluetrue/value/property /bean broker xmlns=http://activemq.apache.org/schema/core ioExceptionHandler=#ioExceptionHandler ... /broker Here are the Handler configuration properties: Property Default Value Description ignoreAllErrors false ignore When true all errors are ignored and don't stop the broker remains running.
[CONF] Apache ActiveMQ Pluggable storage lockers
Paul Gale edited the page: Pluggable storage lockers Comment: Typos, basic formatting. Extended lease locker section to show how to use it with KahaDB persistence adapter. ... Property Name Default Value Description lockAcquireSleepInterval 1 Delay interval in milliseconds between lock acquire attempts failIfLocked false Should the start fail immediately if lock cannot be obtained ... Property Name Default Value Description useLock true can be used to turn off locking if necessary lockKeepAlivePeriod 0 If bigger than 0, time period (in milliseconds) to keep lock alive ... Shared File Locker is a default locker used by KahaDB. It locks a file to ensure that only single resource is used. You can configure it like this: Code
[CONF] Apache ActiveMQ Pluggable storage lockers
Paul Gale edited the page:
Pluggable storage lockers
Comment: Forgot to add the 'dataSource' attribute on the locker element. Wordsmithing.
...
Existing Lockers
Shared File Locker
The Shared File Locker is a default locker used by KahaDB. It locks a file to ensure that only single resource is used. You can configure it like this:
...
This locker doesn't implement keep alive function so there's no point in using lockKeepAlivePeriod settings. Note that as of ActiveMQ 5.9.0 the KahaDB persistence adapter can also utilize use the lease database locker (see below).
Database Locker
The Database Locker is used by default by the default locker for the JDBC persistence adapter. It locks a database table in a transaction to ensure that only single resource is used. You can configure it like this:
Code Block
language
xml
persistenceAdapter
jdbcPersistenceAdapter dataDirectory=${activemq.data} dataSource=#mysql-ds lockKeepAlivePeriod=1
locker
database-locker lockAcquireSleepInterval=5000/
/locker
/jdbcPersistenceAdapter
/persistenceAdapter
The Database locker Locker uses keep alive to make sure broker still holds a lock. You can set the keep alive period with using the lockKeepAlivePeriod property. Default value is 3ms (meaning 30 seconds)3 ms.
The A problem with this locker can occur when the master broker lose crashes or loses its connection to the database or crash unexpectedly. The information about the lock remains in the database, until the database responds to the half closed socket connection via a tcp timeout. The database lock expiry requirement can prevent the slave from starting for a period. In addition, where the database supports failover, and the connection is dropped in the event of a replica
[CONF] Apache ActiveMQ Pluggable storage lockers
Paul Gale edited the page: Pluggable storage lockers Comment: Typo. ... The Lease Database Locker solves the master/slave problem of the default Database Locker. The master broker acquires a lock that's only valid for a short period. To retain the lock the master broker must periodically extend the lock's lease. The slave broker also checks periodically to see if the lease has expired. The lease can survive a db replica failover. You can configure it like this: ... The lease based lock is acquired by blocking at startup. It is then retained for a period whose duration (in ms) is given by the lockKeepAlivePeriod attribute. To retain the lock the master broker continually periodically extends its lease by lockAcquireSleepInterval milliseconds each time. In theory, therefore, the master broker is always (lockAcquireSleepInterval- lockKeepAlivePeriod) ahead of the slave broker w.r.t the lease. It is imperative that lockAcquireSleepInterval lockKeepAlivePeriod, to ensure the lease is always current. As of ActiveMQ 5.9.0 a warning message is logged if this condition is not met. ... View Online Like View Changes Stop watching space Manage Notifications This message was sent by Atlassian Confluence 5.0.3, Team Collaboration Software
