On Mon, Feb 29, 2016 at 7:08 PM, artnaseef <a...@artnaseef.com> wrote:
> Something sounds very wrong there. The NFS lock file should prevent more
> than one broker writing to the store at a time.
>
> Is all of /var/log/activemq/activemq-data/ shared across all of the
> brokers?
>
Hi,
Everything under
/var/log/activemq/activemq-data/amq-dev-2
is shared between
3 nodes:
$ ls -l /var/log/activemq/activemq-data/amq-dev-2
total 396
-rw-r--r--. 1 activemq activemq 33554432 Feb 29 19:27 db-1.log
-rw-r--r--. 1 activemq activemq 176128 Feb 29 19:25 db.data
-rw-r--r--. 1 activemq activemq 176464 Feb 29 19:25 db.redo
-rw-r--r--. 1 activemq activemq 8 Feb 29 19:17 lock
drwxrwxr-x. 2 activemq activemq 4096 Feb 24 00:54 tmp_storage
L
ooking at the logs, sorting and merging the logs based on timestamp, I see
that
amq-dev-2b correctly identified itself as a slave but
that both amq-dev-1a and amq-dev-2c appear to think they were OK to
proceed as the master.
[amq-dev-2a]
2016-02-29 19:08:25,083 [erSimpleAppMain] INFO XBeanBrokerFactory$1
- Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@616d071a:
startup date [Mon Feb 29 19:08:25 PST 2016]; root of context hierarchy
[amq-dev-2b]
2016-02-29 19:08:27,162 [erSimpleAppMain] INFO XBeanBrokerFactory$1
- Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@616d071a:
startup date [Mon Feb 29 19:08:27 PST 2016]; root of context hierarchy
[amq-dev-2a]
2016-02-29 19:08:28,830 [erSimpleAppMain] INFO PListStoreImpl
- PListStore:[/var/log/activemq/activemq-data/amq-dev-2/tmp_storage]
started
[amq-dev-2a]
2016-02-29 19:08:28,937 [erSimpleAppMain] INFO BrokerService
- Using Persistence Adapter:
KahaDBPersistenceAdapter[/var/log/activemq/activemq-data/amq-dev-2]
[amq-dev-2a]
2016-02-29 19:08:29,176 [JMX connector ] INFO ManagementContext
- JMX consoles can connect to
service:jmx:rmi://localhost:2020/jndi/rmi://localhost:2020/jmxrmi
[amq-dev-2b]
2016-02-29 19:08:29,692 [erSimpleAppMain] INFO PListStoreImpl
- PListStore:[/var/log/activemq/activemq-data/amq-dev-2/tmp_storage]
started
[amq-dev-2b]
2016-02-29 19:08:29,760 [erSimpleAppMain] INFO BrokerService
- Using Persistence Adapter:
KahaDBPersistenceAdapter[/var/log/activemq/activemq-data/amq-dev-2]
[amq-dev-2b]
2016-02-29 19:08:29,857 [JMX connector ] INFO ManagementContext
- JMX consoles can connect to
service:jmx:rmi://localhost:2020/jndi/rmi://localhost:2020/jmxrmi
[amq-dev-2c]
2016-02-29 19:08:30,285 [erSimpleAppMain] INFO XBeanBrokerFactory$1
- Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@616d071a:
startup date [Mon Feb 29 19:08:30 PST 2016]; root of context hierarchy
[amq-dev-2c]
2016-02-29 19:08:32,345 [erSimpleAppMain] INFO PListStoreImpl
- PListStore:[/var/log/activemq/activemq-data/amq-dev-2/tmp_storage]
started
[amq-dev-2c]
2016-02-29 19:08:32,427 [erSimpleAppMain] INFO BrokerService
- Using Persistence Adapter:
KahaDBPersistenceAdapter[/var/log/activemq/activemq-data/amq-dev-2]
[amq-dev-2c]
2016-02-29 19:08:32,462 [JMX connector ] INFO ManagementContext
- JMX consoles can connect to
service:jmx:rmi://localhost:2020/jndi/rmi://localhost:2020/jmxrmi
[amq-dev-2b]
2016-02-29 19:08:46,086 [erSimpleAppMain] INFO SharedFileLocker
- Database /var/log/activemq/activemq-data/amq-dev-2/lock is locked by
another server. This broker is now in slave mode waiting a lock to be
acquired
[amq-dev-2c]
2016-02-29 19:13:32,105 [erSimpleAppMain] INFO Journal
- ignoring zero length, partially initialised journal data file:
db-1.log number = 1 , length = 0
[amq-dev-2c]
2016-02-29 19:13:32,471 [erSimpleAppMain] WARN MessageDatabase
- Index corrupted. Recovering the index through journal replay.
Cause:java.lang.IllegalStateException: File size is pre allocated
