Author: buildbot Date: Thu Jul 7 16:22:01 2016 New Revision: 992263 Log: Production update by buildbot for activemq
Modified: websites/production/activemq/content/cache/main.pageCache websites/production/activemq/content/pluggable-storage-lockers.html Modified: websites/production/activemq/content/cache/main.pageCache ============================================================================== Binary files - no diff available. Modified: websites/production/activemq/content/pluggable-storage-lockers.html ============================================================================== --- websites/production/activemq/content/pluggable-storage-lockers.html (original) +++ websites/production/activemq/content/pluggable-storage-lockers.html Thu Jul 7 16:22:01 2016 @@ -81,15 +81,15 @@ <tbody> <tr> <td valign="top" width="100%"> -<div class="wiki-content maincontent"><p>With the 5.7.0 release of ActiveMQ the storage locking mechanism used by a persistence adapter is pluggable. This feature only applies to brokers configured in a shared storage master/slave topology. Prior to release 5.7.0 the storage locking mechanism (and thus master election) was dictated by the choice of persistence adapter. With the KahaDB persistence adapter, for example, the storage locking mechanism was based on a shared file lock. Similarly, the JDBC persistence adapter used a database backed storage lock.</p><p>Now that the choice of storage lock is divorced from that of the persistence adapter one can mix and match. Storage locker pluggability is made possible because all lockers must implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true" rel="nofollow">Locker</a> interface. This interface makes it e asy to implement your own storage locker when you have special requirements. Of course, every persistence adapter still has its own default locker which works as before.</p><h2 id="Pluggablestoragelockers-Lockers">Lockers</h2><p>Every locker must implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true" rel="nofollow">Locker</a> interface. The locker has the following properties:</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1" rowspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><code>lockAcquireSleepInterval</code></p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>10000</p></td><td colspan="1" rowsp an="1" class="confluenceTd"><p>Delay interval in milliseconds between lock acquire attempts</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><code>failIfLocked</code></p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>Should the start fail immediately if lock cannot be obtained</p></td></tr></tbody></table></div><h2 id="Pluggablestoragelockers-PersistenceAdapters">Persistence Adapters</h2><p>Every persistence adapter (or other broker service that wants to use locks) needs to implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Lockable.java?r=1383400" rel="nofollow">Lockable</a> interface. This adds the following properties:</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1" rowspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan ="1" rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><code>useLock</code></p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>can be used to turn off locking if necessary</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><code>lockKeepAlivePeriod</code></p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>0</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>If bigger than 0, time period (in milliseconds) to keep lock alive</p></td></tr></tbody></table></div><h2 id="Pluggablestoragelockers-ExistingLockers">Existing Lockers</h2><h3 id="Pluggablestoragelockers-SharedFileLocker">Shared File Locker</h3><p>The Shared File Locker is the default locker for the KahaDB persistence adapter. It locks a file to ensure that only the broker holding th e lock (the master) is granted access to the message store.</p><p>Example:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> +<div class="wiki-content maincontent"><p>As of the 5.7.0 release of ActiveMQ the choice of storage locking mechanism, as used by a persistence adapter, has been made pluggable. This feature is only meaningful to brokers configured in a shared storage master/slave topology. Prior to release 5.7.0 the storage locking mechanism (and thus master election) was dictated by the choice of persistence adapter. With the KahaDB persistence adapter, for example, the storage locking mechanism was based on a shared file lock. Similarly, the JDBC persistence adapter used a database backed storage lock.</p><p>Now that the choice of storage locker is divorced from that of the persistence adapter one can mix and match combinations of the two. Storage locker pluggability is made possible by the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true" rel="nofollow">Locker</a> interface that all pluggable lockers must implement. This interface makes it easy to implement a custom storage locker that meets local requirements.</p><p>Every persistence adapter, however, has its own default locker which works as before.</p><h2 id="Pluggablestoragelockers-Lockers">Lockers</h2><p>Every locker must implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true" rel="nofollow">Locker</a> interface. The locker interface has the following properties:</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1" rowspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><strong><code>lockAcquireSleepInterval</code></strong></p></ td><td colspan="1" rowspan="1" class="confluenceTd"><pre>10000</pre></td><td colspan="1" rowspan="1" class="confluenceTd"><p>The polling interval (in milliseconds) between lock acquire attempts.</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><strong><code>failIfLocked</code></strong></p></td><td colspan="1" rowspan="1" class="confluenceTd"><pre>false</pre></td><td colspan="1" rowspan="1" class="confluenceTd"><p>Should the broker start fail if the lock is not immediately available. When <strong><code>true</code></strong> slave brokers will not start.</p></td></tr></tbody></table></div><h2 id="Pluggablestoragelockers-PersistenceAdapters">Persistence Adapters</h2><p>Every persistence adapter (or any other broker service that wishes to use locks) must implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Lockable.java?r=1383400" rel="nofollow">Lock able</a> interface. This interface has the following properties:</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1" rowspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><strong><code>useLock</code></strong></p></td><td colspan="1" rowspan="1" class="confluenceTd"><pre>true </pre></td><td colspan="1" rowspan="1" class="confluenceTd"><p>Should the persistence adapter use the configured locker. Intended primarily for use during development to temporarily disable the use of the locker without having to remove its configuration.</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p><strong><code>lockKeepAlivePeriod</code></strong></p></td><td colspan="1" rowspan="1" class="confluenceTd"><pre>0</pre></td><td colspan="1" rowspan="1" class="confluenceTd"><p>The duration (in milliseconds) to keep the lock alive, when greater than <code>0</code>.</p></td></tr></tbody></table></div><h2 id="Pluggablestoragelockers-ExistingLockers">Existing Lockers</h2><h3 id="Pluggablestoragelockers-SharedFileLocker">Shared File Locker</h3><p>The Shared File Locker is the default locker for the KahaDB persistence adapter. It locks a file to ensure that only the broker holding the lock (the master) is granted access to the message store.</p><h4 id="Pluggablestoragelockers-Example:">Example:</h4><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> <pre class="brush: xml; gutter: false; theme: Default" style="font-size:12px;"><persistenceAdapter> - <kahaDB directory = "target/activemq-data"> + <kahaDB directory="target/activemq-data" lockKeepAlivePeriod="10000"> <locker> <shared-file-locker lockAcquireSleepInterval="5000"/> </locker> </kahaDB> </persistenceAdapter></pre> -</div></div><p>This locker implements the <code>keepAlive</code> method from 5.9.0 onwards so there's no point in using <code>lockKeepAlivePeriod</code> settings on older versions than ActiveMQ 5.9.0. Note that as of ActiveMQ 5.9.0 the KahaDB persistence adapter can also use the Lease Database Locker (see below).</p><h3 id="Pluggablestoragelockers-DatabaseLocker">Database Locker</h3><p>The Database Locker is the default locker for the JDBC persistence adapter. It locks a database table in a transaction to ensure that only single resource is used.</p><p>Example:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> +</div></div><p>The <strong><code>lockKeepAlivePeriod</code></strong> attribute is not applicable to versions of ActiveMQ older than 5.9.0.</p><div class="confluence-information-macro confluence-information-macro-warning"><p class="title">Consequences of lockKeepAlivePeriod = 0</p><span class="aui-icon aui-icon-small aui-iconfont-error confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>For this locker <strong><code>lockKeepAlivePeriod</code></strong> should be greater than <strong><code>0</code></strong>.This period is the frequency with which the master broker makes lock keep alive calls.</p><p>When <strong><code>lockKeepAlivePeriod = 0</code></strong> slave brokers are still unable to obtain the file lock. However, if some third party modifies the lock file (either modification or deletion) the master broker will not detect the change. Therefore a slave broker's next attempt (per its configured <strong><code>lockAcquireSleepInterva l</code></strong>) to obtain the file lock will succeed. When this happens there will be two master brokers in the cluster. <em><u>This situation will result in message store corruption</u></em>!</p><p>When <strong><code>lockKeepAlivePeriod</code></strong> is greater than <strong><code>0</code></strong>, the master broker will make a lock keep alive call every <strong><code>lockKeepAlivePeriod</code></strong> milliseconds. Therefore the master broker will detect any lock file changes when it makes its next keep alive call. Upon detecting said change the master broker will demote itself to a slave broker.</p></div></div><div class="confluence-information-macro confluence-information-macro-information"><span class="aui-icon aui-icon-small aui-iconfont-info confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>Note that as of ActiveMQ 5.9.0 the KahaDB persistence adapter can also use the Lease Database Locker (see below).</p></div></div>< h3 id="Pluggablestoragelockers-DatabaseLocker">Database Locker</h3><p>The Database Locker is the default locker for the JDBC persistence adapter. It locks a database table in a transaction to ensure that only single resource is used.</p><h4 id="Pluggablestoragelockers-Example:.1">Example:</h4><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> <pre class="brush: xml; gutter: false; theme: Default" style="font-size:12px;"><persistenceAdapter> <jdbcPersistenceAdapter dataDirectory="${activemq.data}" dataSource="#mysql-ds" lockKeepAlivePeriod="10000"> <locker> @@ -97,7 +97,7 @@ </locker> </jdbcPersistenceAdapter> </persistenceAdapter></pre> -</div></div><p>The Database Locker uses its <code>keepAlive</code> method to ensure the broker still holds the lock. You can set the keep alive period using the <code>lockKeepAlivePeriod</code> property. The default period is 30000 ms. If a broker fails to acquire the lock on the database, it will retry every <code>lockAcquireSleepInterval</code> milliseconds.</p><p>This locker opens a JDBC transaction against a database table (<code>activemq_lock</code>) that lasts as long as the broker remains alive. This locks the entire table and prevents another broker from accessing the store. In most cases this will be a fairly long running JDBC transaction which occupies resources on the database over time.</p><p>A problem with this locker can arise when the master broker crashes or loses its connection to the database causing the lock to remain in the database until the database responds to the half closed socket connection via a TCP timeout. The database lock expiry requirement can pr event the slave from starting some time. In addition, if the database supports failover, and the connection is dropped in the event of a replica failover, that JDBC transaction will be rolled back. The broker sees this as a failure. Both master and slave brokes will again compete for a lock.</p><h3 id="Pluggablestoragelockers-LeaseDatabaseLocker">Lease Database Locker</h3><p>The Lease Database Locker was created to solve the shortcomings of the Database Locker. The Lease Database Locker does not open a long running JDBC transaction. Instead it lets the master broker acquire a lock that's valid for a fixed (usually short) duration after which it expires. To retain the lock the master broker must periodically extend the lock's lease before it expires. Simultaneously the slave broker also checks periodically to see if the lease has expired. If, for whatever reason, the master broker fails to update its lease on the lock the slave will take ownership of the lock becoming the new master in the process. The leased lock can survive a DB replica failover.</p><p>Example:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> +</div></div><p>The Database Locker uses its <strong><code>keepAlive</code></strong> method to ensure the broker still holds the lock. You can set the keep alive period using the <strong><code>lockKeepAlivePeriod</code></strong> property. The default period is 30000 ms. If a broker fails to acquire the lock on the database, it will retry every <strong><code>lockAcquireSleepInterval</code></strong> milliseconds.</p><p>This locker opens a JDBC transaction against a database table (<strong><code>activemq_lock</code></strong>) that lasts as long as the broker remains alive. This locks the entire table and prevents another broker from accessing the store. In most cases this will be a fairly long running JDBC transaction which occupies resources on the database over time.</p><p>A problem with this locker can arise when the master broker crashes or loses its connection to the database causing the lock to remain in the database until the database responds to the half closed socket conne ction via a TCP timeout. The database lock expiry requirement can prevent the slave from starting some time. In addition, if the database supports failover, and the connection is dropped in the event of a replica failover, that JDBC transaction will be rolled back. The broker sees this as a failure. Both master and slave brokes will again compete for a lock.</p><h3 id="Pluggablestoragelockers-LeaseDatabaseLocker">Lease Database Locker</h3><p>The Lease Database Locker was created to solve the shortcomings of the Database Locker. The Lease Database Locker does not open a long running JDBC transaction. Instead it lets the master broker acquire a lock that's valid for a fixed (usually short) duration after which it expires. To retain the lock the master broker must periodically extend the lock's lease before it expires. Simultaneously the slave broker checks periodically to see if the lease has expired. If, for whatever reason, the master broker fails to update its lease on the lock the slave will take ownership of the lock becoming the new master in the process. The leased lock can survive a DB replica failover.</p><h4 id="Pluggablestoragelockers-Example:.2">Example:</h4><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> <pre class="brush: xml; gutter: false; theme: Default" style="font-size:12px;"><persistenceAdapter> <jdbcPersistenceAdapter dataDirectory="${activemq.data}" dataSource="#mysql-ds" lockKeepAlivePeriod="5000"> <locker> @@ -105,7 +105,7 @@ </locker> </jdbcPersistenceAdapter> </persistenceAdapter></pre> -</div></div><p>In order for this mechanism to work correctly, each broker in the master/slave pair must have a different <code>brokerName</code> attribute defined on the <code>broker</code> tag or use the <span><code>leaseHolderId</code> attribute</span> on the <code><span style="line-height: 1.4285715;">lease-database-locker</span></code><span>, as it is this value that is used to reserve a lease.</span></p><p>The lease based lock is acquired by blocking at startup. It is then retained for a period whose duration (in ms) is given by the <code>lockKeepAlivePeriod</code> attribute. To retain the lock the master broker periodically extends its lease by <code>lockAcquireSleepInterval</code> milliseconds each time. In theory, therefore, the master broker is always (<code>lockAcquireSleepInterval</code> <code>-</code> <code>lockKeepAlivePeriod</code>) ahead of the slave broker with regard to the lease. It is imperative that <code>lockAcquireSleepInterval > lockKeepAlivePerio d</code>, to ensure the lease is always current. As of ActiveMQ 5.9.0 a warning message is logged if this condition is not met.</p><p>In the simplest case, the clocks between master and slave must be in sync for this solution to work properly. If the clocks cannot be in sync, the locker can use the system time from the database CURRENT TIME and adjust the timeouts in accordance with their local variance from the DB system time. If <code>maxAllowableDiffFromDBTime</code> is greater than zero the local periods will be adjusted by any delta that exceeds <code>maxAllowableDiffFromDBTime</code>.</p><div class="confluence-information-macro confluence-information-macro-information"><span class="aui-icon aui-icon-small aui-iconfont-info confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>It is important to know if the default rules your JDBC driver uses for converting <code>TIME</code> values are JDBC compliant. If you're using MySQL, for example , the driver's JDBC URL should contain <code>useJDBCCompliantTimezoneShift=true</code> to ensure that <code>TIME</code> value conversion is JDBC compliant. If not the locker could report a large time difference when it compares the retrieved lease expiration time against the current system time. Consult your JDBC driver's documentation for more details.</p></div></div><p>As of ActiveMQ 5.9.0 the lease database locker can be used in conjunction with the KahaDB persistence adapter. However, this particular combination requires that the lease database locker element contains a <code><statements/></code> child element. In the example below the <code>lockTableName</code> is also configured, although doing so is not mandatory.</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> +</div></div><p>In order for this mechanism to work correctly, each broker in a master/slave(s) cluster must have a unique value for the <strong><code>brokerName</code></strong> attribute as defined on the <strong><code><broker/></code></strong> tag. Alternatively, use unique values for the <span><strong><code>leaseHolderId</code></strong> attribute</span> on the <strong><code><span style="line-height: 1.42857;"><lease-database-locker/></span></code></strong><span> tag as this value is used to create a lease lock definition.</span></p><p>The lease based lock is acquired by blocking at startup. It is then retained for a period whose duration (in ms) is given by the <strong><code>lockKeepAlivePeriod</code></strong> attribute. To retain the lock the master broker periodically extends its lease by <strong><code>lockAcquireSleepInterval</code></strong> milliseconds each time. In theory, therefore, the master broker is always (<strong><code>lockAcquireSleepInterval - lockKeepAl ivePeriod</code></strong>) ahead of the slave broker with regard to the lease. It is imperative that <strong><code>lockAcquireSleepInterval > lockKeepAlivePeriod</code></strong>, to ensure the lease is always current. As of ActiveMQ 5.9.0 a warning message is logged if this condition is not met.</p><p>In the simplest case, the clocks between master and slave must be in sync for this solution to work properly. If the clocks cannot be in sync, the locker can use the system time from the database CURRENT TIME and adjust the timeouts in accordance with their local variance from the DB system time. If <strong><code>maxAllowableDiffFromDBTime</code></strong> is greater than zero the local periods will be adjusted by any delta that exceeds <strong><code>maxAllowableDiffFromDBTime</code></strong>.</p><div class="confluence-information-macro confluence-information-macro-information"><span class="aui-icon aui-icon-small aui-iconfont-info confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>It is important to know if the default rules your JDBC driver uses for converting <code>TIME</code> values are JDBC compliant. If you're using MySQL, for example, the driver's JDBC URL should contain <strong><code>useJDBCCompliantTimezoneShift=true</code></strong> to ensure that <code>TIME</code> value conversion is JDBC compliant. If not the locker could report a large time difference when it compares the retrieved lease expiration time against the current system time. Consult your JDBC driver's documentation for more details.</p></div></div><p>As of ActiveMQ 5.9.0 the lease database locker can be used in conjunction with the KahaDB persistence adapter. However, this particular combination requires that the lease database locker element contains a <strong><code><statements/></code></strong> child element. In the example below the <strong><code>lockTableName</code></strong> is also configured, although doing so is not mand atory.</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl"> <pre class="brush: xml; gutter: false; theme: Default" style="font-size:12px;"> <persistenceAdapter> <kahaDB directory="target/activemq-data" lockKeepAlivePeriod="5000"> <locker> @@ -120,7 +120,7 @@ </locker> </kahaDB> </persistenceAdapter></pre> -</div></div><p>To see the complete list of attributes and SQL statements that can be overridden see the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-jdbc-store/src/main/java/org/apache/activemq/store/jdbc/Statements.java?hb=true" rel="nofollow">Statements</a> class. When the KahaDB persistence adapter is configured to use the <code>lease-database-locker</code> you must configure the broker to use your own IO exception handler as neither the <code>DefaultIOExceptionHandler</code> nor the <code>JDBCIOExceptionHandler</code> will work correctly with this combination. See <a shape="rect" href="configurable-ioexception-handling.html">Configurable IOException Handlers</a> for details on how to write a handler.</p><p>As of ActiveMQ 5.11, however, the <code>JDBCIOExceptionHandler</code> has been deprecated. It has been replaced by the <code>org.apache.activemq.util.LeaseLockerIOExceptionHandler</code> that will work with any persi stence adapter that supports pluggable storage lockers (whether or not one is configured).</p><p> </p><p> </p></div> +</div></div><div class="confluence-information-macro confluence-information-macro-information"><span class="aui-icon aui-icon-small aui-iconfont-info confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>To see the complete list of attributes and SQL statements that can be overridden see the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-jdbc-store/src/main/java/org/apache/activemq/store/jdbc/Statements.java?hb=true" rel="nofollow">Statements</a> class.</p></div></div><p>When the KahaDB persistence adapter is configured to use the <strong><code>lease-database-locker</code></strong> you must configure the broker to use your own IO exception handler as neither the <strong><code>DefaultIOExceptionHandler</code></strong> nor the <strong><code>JDBCIOExceptionHandler</code></strong> will work correctly with this combination. See <a shape="rect" href="configurable-ioexception-handling.html">Con figurable IOException Handlers</a> for details on how to write a handler.</p><div class="confluence-information-macro confluence-information-macro-information"><span class="aui-icon aui-icon-small aui-iconfont-info confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>As of ActiveMQ 5.11, however, the <strong><code>JDBCIOExceptionHandler</code></strong> has been deprecated. It has been replaced by the <strong><code>org.apache.activemq.util.LeaseLockerIOExceptionHandler</code></strong> that will work with any persistence adapter that supports pluggable storage lockers, regardless if one is configured.</p></div></div><p> </p><p> </p><p> </p><p> </p><p> </p></div> </td> <td valign="top"> <div class="navigation">