Hey. I have a similar configuration and I’m getting a ton of "No reader available for position” messages as well as significant data loss on AMQ restart.
It literally loses about 95% of the messages I enqueued.. On Wed, Dec 17, 2014 at 9:06 AM, Christian Grassi <[email protected] > wrote: > > I have a 3 node 5.10 cluster with replicated levelDB as persistent store. > > > <persistenceAdapter> > <replicatedLevelDB > directory="activemq-data" > replicas="3" > bind="tcp://0.0.0.0:0" > zkAddress="queue1:2181,queue2:2181,queue3:2181" > zkPath="/activemq/leveldb-stores" > hostname="queue3" > sync="quorum_disk" > /> > </persistenceAdapter> > > > I'm Stresstesting the queue with around 5000 persistent msg/s. > 1 Producer > 2 Consumers > > I get some warning messages on the log: > On Master: > 2014-12-16 16:25:28,375 | INFO | Slave has disconnected: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-3 > 2014-12-16 16:25:28,761 | WARN | Unexpected session error: > java.io.FileNotFoundException: > /queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or > directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-1 > 2014-12-16 16:25:28,761 | INFO | Slave has disconnected: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > 2014-12-16 16:25:29,603 | INFO | Slave has connected: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > 2014-12-16 16:25:30,034 | INFO | Slave has connected: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-1 > 2014-12-16 16:25:31,360 | INFO | Slave has now caught up: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-4 > 2014-12-16 16:25:32,546 | INFO | Slave has now caught up: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > > > On the slaves > > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > I have a 3 node 5.10 cluster with replicated levelDB as persistent store. > > > <persistenceAdapter> > <replicatedLevelDB > directory="activemq-data" > replicas="3" > bind="tcp://0.0.0.0:0" > zkAddress="queue1:2181,queue2:2181,queue3:2181" > zkPath="/activemq/leveldb-stores" > hostname="queue3" > sync="quorum_disk" > /> > </persistenceAdapter> > > > I'm Stresstesting the queue with around 5000 persistent msg/s. > 1 Producer > 2 Consumers > > I get some warning messages on the log: > On Master: > 2014-12-16 16:25:28,375 | INFO | Slave has disconnected: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-3 > 2014-12-16 16:25:28,761 | WARN | Unexpected session error: > java.io.FileNotFoundException: > /queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or > directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-1 > 2014-12-16 16:25:28,761 | INFO | Slave has disconnected: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > 2014-12-16 16:25:29,603 | INFO | Slave has connected: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > 2014-12-16 16:25:30,034 | INFO | Slave has connected: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-1 > 2014-12-16 16:25:31,360 | INFO | Slave has now caught up: > 6a84579c-77fe-41eb-a728-1be472c12894 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-4 > 2014-12-16 16:25:32,546 | INFO | Slave has now caught up: > db6c9a23-7025-4384-b02f-dcda763113c3 | > org.apache.activemq.leveldb.replicated.MasterLevelDBStore | > hawtdispatch-DEFAULT-2 > > > On the slaves > > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > 2014-12-16 16:30:00,287 | WARN | No reader available for position: > 27fd52c562, log_infos: > > {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} > | org.apache.activemq.leveldb.RecordLog | Thread-16882 > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
