i validated that the fix did in fact fix our upgrade path. i made a new 2.1.0 broker. i added some address settings via JMX. i attempted an upgrade to 2.6.2 and watched it fail. i pulled the code that you linked above. i packaged a release and then i upgraded to the released code and the server started just fine.
thank you so much!! what is the timeline on a 2.6.3 or 2.7.0 release? On Fri, Aug 3, 2018 at 4:22 PM Clebert Suconic <clebert.suco...@gmail.com> wrote: > I'm considering doing a 2.6.3 very soon.. it will include the fix.. > you should be able to move from 2.1.0 . let me know if you have any > other issues. > > On Fri, Aug 3, 2018 at 6:22 PM, Clebert Suconic > <clebert.suco...@gmail.com> wrote: > > https://github.com/apache/activemq-artemis/pull/2214 > > > > > > I will make it into 2.6.x as soon as the PR tests are finished. > > > > > > If you could try a snapshot of master or 2.6.x (after I merge it of > > course). Let me know if you need help buildling it. > > > > On Wed, Jul 25, 2018 at 5:17 PM, Dan Langford <danlangf...@gmail.com> > wrote: > >> Michael for the win! > >> @Clebert im not opposed to coding up a test case it is just that i am > >> swamped at the moment so im glad to hear that Michael already has a > bead on > >> this > >> > >> i did a *./artemis data exp* out of my 2.1.0 and then an *./artemis data > >> imp* on a clean 2.6.2. the new system didnt have Address Settings and > >> Security Settings that had previously been added via the jolokia api. i > >> should look into the export options more to see if i just failed to > provide > >> a command line argument. fortunately in my scenario it was pretty > trivial > >> to recreate those settings quickly. so i did that. the import created > all > >> the addresses, routes, queues, and messages that i needed. it went > pretty > >> well. so my cluster is now on 2.6.2 > >> > >> i plan to rework our usage of AddressSetting to not create a new entry > for > >> every Address we create but instead only create them for the few > exceptions > >> to our default configuration > >> > >> then i can start working on running the broker in CF with the cluster in > >> more of a Master-Master mode. > >> thanks again for all your help. feel free to respond if there is any > more > >> data you need from me. > >> > >> *side note: my broker is now spamming " AMQ224088: Timeout (10 seconds) > >> while handshaking has occurred. " but thats another issue i will figure > >> out. * > >> > >> my 30 minute startup times are now down to 30 seconds max. more like > 10-15 > >> seconds regularly. THANK YOU ALL SO MUCH FOR YOUR HELP. > >> > >> On Wed, Jul 25, 2018 at 2:44 PM michael.andre.pearce > >> <michael.andre.pea...@me.com.invalid> wrote: > >> > >>> Clebert im able to recreate this and also worked out in code why this > >>> happens. Ill try catch you on irc tomorrow or friday. > >>> > >>> Sent from my Samsung Galaxy smartphone. > >>> -------- Original message --------From: Clebert Suconic < > >>> clebert.suco...@gmail.com> Date: 25/07/2018 20:20 (GMT+00:00) To: > >>> users@activemq.apache.org Subject: Re: [artemis 2.1.0] taking 30+ > minutes > >>> to boot & failover > >>> Yes.. it should include it. > >>> > >>> > >>> Do you have an example of the API you used with 2.1? > >>> > >>> > >>> I can't write a compatibility test this week.. but if you provide me > >>> the example I will do it early next week. > >>> > >>> > >>> In a perfect world, if you did it .. it would be great :) > >>> > >>> > >>> On Wed, Jul 25, 2018 at 1:40 PM, Dan Langford <danlangf...@gmail.com> > >>> wrote: > >>> > i tried 2.6.2 this morning to see if that was an improvement from > 2.5.0 i > >>> > tried a few months ago. unfortunately there was not. it may have > failed > >>> > much faster than 2.5.0 i dont recall the timing specifics but the > error > >>> is > >>> > the same > >>> > > >>> > 11:15:50,853 ERROR [org.apache.activemq.artemis.core.server] > AMQ224000: > >>> > Failure in initialisation: java.lang.NegativeArraySizeException > >>> > at > >>> > > >>> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(SimpleString.java:182) > >>> > [artemis-commons-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(SimpleString.java:171) > >>> > [artemis-commons-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.api.core.SimpleString.readNullableSimpleString(SimpleString.java:158) > >>> > [artemis-commons-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readNullableSimpleString(ChannelBufferWrapper.java:69) > >>> > [artemis-commons-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.settings.impl.AddressSettings.decode(AddressSettings.java:736) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.persistence.config.PersistedAddressSetting.decode(PersistedAddressSetting.java:95) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.newAddressEncoding(AbstractJournalStorageManager.java:1925) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.loadBindingJournal(AbstractJournalStorageManager.java:1466) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:2601) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:2376) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:116) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:535) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:474) > >>> > [artemis-server-2.6.2.jar:2.6.2] > >>> > at > >>> > > >>> > org.apache.activemq.artemis.integration.FileBroker.start(FileBroker.java:111) > >>> > [artemis-cli-2.6.2.jar:2.6.2] > >>> > at org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:82) > >>> > [artemis-cli-2.6.2.jar:2.6.2] > >>> > at > >>> > > org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:149) > >>> > [artemis-cli-2.6.2.jar:2.6.2] > >>> > at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:97) > >>> > [artemis-cli-2.6.2.jar:2.6.2] > >>> > at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:124) > >>> > [artemis-cli-2.6.2.jar:2.6.2] > >>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>> > [rt.jar:1.8.0_171] > >>> > at > >>> > > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > [rt.jar:1.8.0_171] > >>> > at > >>> > > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > [rt.jar:1.8.0_171] > >>> > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_171] > >>> > at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129) > >>> > [artemis-boot.jar:2.6.2] > >>> > at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49) > >>> > [artemis-boot.jar:2.6.2] > >>> > > >>> > > >>> > right there in the AddressSettings.decode > >>> > > >>> > > >>> > I am going to start looking into a journal export and then a journal > >>> import > >>> > and see what happens there. > >>> > are programmatically created addresses / queues stored in the data > export > >>> > along with the address and security settings? > >>> > > >>> > > >>> > > >>> > > >>> > On Wed, Jul 25, 2018 at 10:49 AM Clebert Suconic < > >>> clebert.suco...@gmail.com> > >>> > wrote: > >>> > > >>> >> @dan take a look at the compatibility tests in master.. We're help > to > >>> >> help if you don't understand anything.. it's using ClassLoaders and > >>> >> Groovy so you can produce stuff with a combination of clients and > >>> >> servers on 2.1.0 and consume on the current version. > >>> >> > >>> >> On Wed, Jul 25, 2018 at 12:04 PM, Clebert Suconic > >>> >> <clebert.suco...@gmail.com> wrote: > >>> >> > @Dan: what about this: > >>> >> > > >>> >> > you provide us a code path that could generate the > incompatibility in > >>> >> 2.1.0... > >>> >> > > >>> >> > an ultimate deal would be if you produced a compatibility test > with > >>> >> > 2.1.0.... you add some code that will generate the journal with > 2.1.0, > >>> >> > and then consume the messages on master (or 2.6.x)... > >>> >> > > >>> >> > > >>> >> > if you generate the compatibility test showing the issue, we can > try > >>> >> fixing it. > >>> >> > > >>> >> > > >>> >> > What about this idea? that would help you migrate into 2.6.x or > >>> master. > >>> >> > > >>> >> > On Tue, Jul 24, 2018 at 7:25 PM, Clebert Suconic > >>> >> > <clebert.suco...@gmail.com> wrote: > >>> >> >> On Fri, Jul 20, 2018 at 4:47 PM, Dan Langford < > danlangf...@gmail.com > >>> > > >>> >> wrote: > >>> >> >>> Thank you that was very helpful. we actually do have an address > >>> >> settings > >>> >> >>> entry for each queue. there could be a better pattern for us. > but > >>> >> currently > >>> >> >>> our automated system for creating queues creates an address > setting > >>> at > >>> >> the > >>> >> >>> same time. i will look into improved patterns. > >>> >> >>> > >>> >> >>> as far as upgrading goes. i agree we really want to upgrade. > until i > >>> >> can > >>> >> >>> find a work around for the config-delete-queues deserialization > bug > >>> >> >>> introduced in 2.2.0 i brought up back in April we will not be > able > >>> to > >>> >> >>> easily move. > >>> >> >> > >>> >> >> You should be able to move to 2.6.2. if you're not able to I > would > >>> >> >> like to know where it failed. > >>> >> >> > >>> >> >> > >>> >> >>> > >>> >> >>> thanks again for all the help > >>> >> >>> > >>> >> >>> On Fri, Jul 20, 2018 at 8:56 AM Clebert Suconic < > >>> >> clebert.suco...@gmail.com> > >>> >> >>> wrote: > >>> >> >>> > >>> >> >>>> If you do not want to upgrade for any reason export the > journal. > >>> >> Cleani > >>> >> >>>> uo. Edit the text and remove the garbage (you will see) > manually. > >>> >> Delete > >>> >> >>>> all data and te import. > >>> >> >>>> > >>> >> >>>> (Make a backup to be safe of course) > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> But I still recommend the upgrade. > >>> >> >>>> > >>> >> >>>> On Fri, Jul 20, 2018 at 10:54 AM Clebert Suconic < > >>> >> >>>> clebert.suco...@gmail.com> > >>> >> >>>> wrote: > >>> >> >>>> > >>> >> >>>> > The address setting is the garbage I was talking about. > Upgrade > >>> to > >>> >> the > >>> >> >>>> > latest broker and there will be a cleanup done at the load > >>> before it > >>> >> >>>> > starts. > >>> >> >>>> > > >>> >> >>>> > > >>> >> >>>> > I highly recommend upgrade. > >>> >> >>>> > > >>> >> >>>> > On Fri, Jul 20, 2018 at 10:05 AM Justin Bertram < > >>> >> jbert...@apache.org> > >>> >> >>>> > wrote: > >>> >> >>>> > > >>> >> >>>> >> Analyzing thread dumps like this is pretty simple. I > generally > >>> >> just > >>> >> >>>> >> scroll > >>> >> >>>> >> through and look for long stack-traces with lots of calls > from > >>> >> >>>> >> org.apache.activemq.artemis. In your case every single > thread > >>> >> dump has > >>> >> >>>> a > >>> >> >>>> >> thread doing something like this: > >>> >> >>>> >> > >>> >> >>>> >> "main" #1 prio=5 os_prio=0 tid=0x00007f902800eb20 nid=0x74fe > >>> >> runnable > >>> >> >>>> >> [0x00007f9031675000] > >>> >> >>>> >> java.lang.Thread.State: RUNNABLE > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.getPossibleMatches(HierarchicalObjectRepository.java:373) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.getMatch(HierarchicalObjectRepository.java:192) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl.reapplySettings(PagingManagerImpl.java:113) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl.onChange(PagingManagerImpl.java:108) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.onChange(HierarchicalObjectRepository.java:348) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:168) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:147) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:120) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.recoverStoredConfigs(ActiveMQServerImpl.java:2424) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:2374) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:2219) > >>> >> >>>> >> - locked <0x0000000080a8fce8> (a > >>> >> >>>> >> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:109) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:518) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:466) > >>> >> >>>> >> - locked <0x0000000080a8fce8> (a > >>> >> >>>> >> > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.integration.FileBroker.start(FileBroker.java:111) > >>> >> >>>> >> - locked <0x000000008098be80> (a > >>> >> >>>> >> org.apache.activemq.artemis.integration.FileBroker) > >>> >> >>>> >> at > >>> >> org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:73) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:148) > >>> >> >>>> >> at > >>> >> org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:95) > >>> >> >>>> >> at > >>> >> org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:122) > >>> >> >>>> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> >> >>>> >> at > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> > >>> >> > >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> >> >>>> >> at java.lang.reflect.Method.invoke(Method.java:498) > >>> >> >>>> >> at > >>> >> >>>> > org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129) > >>> >> >>>> >> at > >>> >> org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49) > >>> >> >>>> >> > >>> >> >>>> >> In every case it's the "main" thread which isn't surprising > as > >>> >> that is > >>> >> >>>> the > >>> >> >>>> >> thread responsible for starting the broker. Also, you can > >>> pretty > >>> >> >>>> clearly > >>> >> >>>> >> see in the trace that this is the thread starting the > broker, > >>> it's > >>> >> >>>> loading > >>> >> >>>> >> the journals, & restoring stored configuration (either > address > >>> >> settings > >>> >> >>>> or > >>> >> >>>> >> security settings). I've seen high broker start times when > >>> there > >>> >> are > >>> >> >>>> lots > >>> >> >>>> >> and lots of addresses and lots of and lots of settings. Do > >>> either > >>> >> (or > >>> >> >>>> >> both) of these situations apply to you? > >>> >> >>>> >> > >>> >> >>>> >> > >>> >> >>>> >> Justin > >>> >> >>>> >> > >>> >> >>>> >> On Fri, Jul 20, 2018 at 1:16 AM, Dan Langford < > >>> >> danlangf...@gmail.com> > >>> >> >>>> >> wrote: > >>> >> >>>> >> > >>> >> >>>> >> > Dang I can’t easily upgrade past 2.1.0 because of the > >>> >> >>>> >> config-delete-queues > >>> >> >>>> >> > deserialization bug introduced in 2.2.0. Unless that bug > was > >>> >> squashed > >>> >> >>>> in > >>> >> >>>> >> > 2.6+. I don’t think I made a jira for it (vacation and > work > >>> >> load) but > >>> >> >>>> we > >>> >> >>>> >> > discussed it back in April. I should go confirm that bug > on > >>> 2.6 > >>> >> and > >>> >> >>>> >> make a > >>> >> >>>> >> > jira for that > >>> >> >>>> >> > > >>> >> >>>> >> > Thanks > >>> >> >>>> >> > On Thu, Jul 19, 2018 at 5:46 PM Clebert Suconic < > >>> >> >>>> >> clebert.suco...@gmail.com > >>> >> >>>> >> > > > >>> >> >>>> >> > wrote: > >>> >> >>>> >> > > >>> >> >>>> >> > > There is an issue I remember where the journal would > have > >>> some > >>> >> dirt > >>> >> >>>> >> that > >>> >> >>>> >> > > was fixed on 2.3/0. > >>> >> >>>> >> > > > >>> >> >>>> >> > > I would ipgrade to 2.6.2. > >>> >> >>>> >> > > > >>> >> >>>> >> > > On Thu, Jul 19, 2018 at 6:34 PM Dan Langford < > >>> >> danlangf...@gmail.com > >>> >> >>>> > > >>> >> >>>> >> > > wrote: > >>> >> >>>> >> > > > >>> >> >>>> >> > > > would you be willing to help me translate these thread > >>> dumps? > >>> >> >>>> >> > > > > >>> >> >>>> >> > > > i attached a Zip file with some thread dumps in them. > i > >>> will > >>> >> also > >>> >> >>>> >> share > >>> >> >>>> >> > > > the fasthread.io links for each file. (i was > struggling > >>> >> getting > >>> >> >>>> >> > > > fastthread to do a combo report with the threads in > the > >>> >> correct > >>> >> >>>> >> order) > >>> >> >>>> >> > > > > >>> >> >>>> >> > > > artemis04-20180719-1525 https://goo.gl/d88azU > >>> >> >>>> >> > > > artemis04-20180719-1530 https://goo.gl/G78qn3 > >>> >> >>>> >> > > > artemis04-20180719-1535 https://goo.gl/aMBSBw > >>> >> >>>> >> > > > artemis04-20180719-1540 https://goo.gl/brKxxk > >>> >> >>>> >> > > > artemis04-20180719-1545 https://goo.gl/RaXXCs > >>> >> >>>> >> > > > artemis04-20180719-1550 https://goo.gl/r5dndK > >>> >> >>>> >> > > > artemis04-20180719-1555 https://goo.gl/YJRLxe > >>> >> >>>> >> > > > > >>> >> >>>> >> > > > at :35, :45, and :55 the young+old gen space gets > bigger > >>> >> than at > >>> >> >>>> the > >>> >> >>>> >> > > other > >>> >> >>>> >> > > > sample times. but i dont know what to look for in > here to > >>> >> >>>> determine > >>> >> >>>> >> > what > >>> >> >>>> >> > > > the broker is actually during during this time. > >>> >> >>>> >> > > > > >>> >> >>>> >> > > > thanks > >>> >> >>>> >> > > > > >>> >> >>>> >> > > > On Fri, Jul 6, 2018 at 10:42 AM Justin Bertram < > >>> >> >>>> jbert...@apache.org > >>> >> >>>> >> > > >>> >> >>>> >> > > > wrote: > >>> >> >>>> >> > > > > >>> >> >>>> >> > > >> The first place I would start is grabbing thread > dumps > >>> >> every so > >>> >> >>>> >> often > >>> >> >>>> >> > to > >>> >> >>>> >> > > >> see what the broker is actually doing during the 30+ > >>> >> minutes. > >>> >> >>>> >> > > >> > >>> >> >>>> >> > > >> > >>> >> >>>> >> > > >> Justin > >>> >> >>>> >> > > >> > >>> >> >>>> >> > > >> On Fri, Jul 6, 2018 at 11:34 AM, Dan Langford < > >>> >> >>>> >> danlangf...@gmail.com> > >>> >> >>>> >> > > >> wrote: > >>> >> >>>> >> > > >> > >>> >> >>>> >> > > >> > so my server startup times and failover times are > >>> growing > >>> >> >>>> pretty > >>> >> >>>> >> > big. > >>> >> >>>> >> > > >> but i > >>> >> >>>> >> > > >> > dont really know where to start looking. > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > here is a snippet of some logs to show you the time > >>> >> stamps: > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:31,801 INFO [org.apache.activemq.artemis. > >>> >> >>>> >> > integration.bootstrap] > >>> >> >>>> >> > > >> > AMQ101000: Starting ActiveMQ Artemis Server > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:31,818 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221000: > >>> >> >>>> >> > > >> > live Message Broker is starting with configuration > >>> Broker > >>> >> >>>> >> > > Configuration > >>> >> >>>> >> > > >> > (clustered=true,journalDirectory=./data/ > >>> >> >>>> >> > journal,bindingsDirectory=./ > >>> >> >>>> >> > > >> > > >>> >> data/bindings,largeMessagesDirectory=./data/large-messages, > >>> >> >>>> >> > > >> > pagingDirectory=./data/paging) > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,462 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221012: > >>> >> >>>> >> > > >> > Using AIO Journal > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,493 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221057: > >>> >> >>>> >> > > >> > Global Max Size is being adjusted to 1/2 of the > JVM max > >>> >> size > >>> >> >>>> >> (-Xmx). > >>> >> >>>> >> > > >> being > >>> >> >>>> >> > > >> > defined as 1,073,741,824 > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,555 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-server]. Adding > >>> protocol > >>> >> >>>> support > >>> >> >>>> >> > for: > >>> >> >>>> >> > > >> CORE > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,555 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-amqp-protocol]. > Adding > >>> >> protocol > >>> >> >>>> >> > > support > >>> >> >>>> >> > > >> > for: AMQP > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,556 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-hornetq-protocol]. > >>> Adding > >>> >> >>>> >> protocol > >>> >> >>>> >> > > >> support > >>> >> >>>> >> > > >> > for: HORNETQ > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,556 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-mqtt-protocol]. > Adding > >>> >> protocol > >>> >> >>>> >> > > support > >>> >> >>>> >> > > >> > for: MQTT > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,556 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-openwire-protocol]. > >>> Adding > >>> >> >>>> >> protocol > >>> >> >>>> >> > > >> support > >>> >> >>>> >> > > >> > for: OPENWIRE > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:11:34,556 INFO > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ221043: > >>> >> >>>> >> > > >> > Protocol module found: [artemis-stomp-protocol]. > Adding > >>> >> >>>> protocol > >>> >> >>>> >> > > support > >>> >> >>>> >> > > >> > for: STOMP > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,963 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222165: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Dead Letter Address configured for queue DLQ in > >>> >> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,963 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222166: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Expiry Address configured for queue DLQ in > >>> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,983 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222165: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Dead Letter Address configured for queue > ExpiryQueue in > >>> >> >>>> >> > > AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,983 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222166: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Expiry Address configured for queue ExpiryQueue in > >>> >> >>>> >> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,984 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222165: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Dead Letter Address configured for queue example in > >>> >> >>>> >> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,985 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222166: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Expiry Address configured for queue example in > >>> >> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,985 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222165: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Dead Letter Address configured for queue > exampleQueue > >>> in > >>> >> >>>> >> > > AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > 08:41:38,986 WARN > >>> >> [org.apache.activemq.artemis.core.server] > >>> >> >>>> >> > > AMQ222166: > >>> >> >>>> >> > > >> No > >>> >> >>>> >> > > >> > Expiry Address configured for queue exampleQueue in > >>> >> >>>> >> AddressSettings > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > and it continues. i have 138 queues. i wonder if i > need > >>> >> to be > >>> >> >>>> >> > looking > >>> >> >>>> >> > > at > >>> >> >>>> >> > > >> > PAGE configuration or some cache sizes. do i need > to be > >>> >> looking > >>> >> >>>> >> at > >>> >> >>>> >> > the > >>> >> >>>> >> > > >> > number of messages persisted on these queues? where > >>> would > >>> >> you > >>> >> >>>> >> look > >>> >> >>>> >> > to > >>> >> >>>> >> > > >> > determine why the startup times are so long? > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > also, i know i need to upgrade but i cannot upgrade > >>> off of > >>> >> >>>> 2.1.0 > >>> >> >>>> >> due > >>> >> >>>> >> > > to > >>> >> >>>> >> > > >> > some deseralization changes introduced in 2.2.0. i > >>> think > >>> >> i have > >>> >> >>>> >> > > another > >>> >> >>>> >> > > >> > thread on here (that i need to update) regarding > that > >>> >> upgrade > >>> >> >>>> >> issue. > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > thanks for any ideas or insight you have for me > >>> >> >>>> >> > > >> > > >>> >> >>>> >> > > >> > >>> >> >>>> >> > > > -- > >>> >> >>>> >> > > Clebert Suconic > >>> >> >>>> >> > > > >>> >> >>>> >> > > >>> >> >>>> >> > >>> >> >>>> > -- > >>> >> >>>> > Clebert Suconic > >>> >> >>>> > > >>> >> >>>> -- > >>> >> >>>> Clebert Suconic > >>> >> >>>> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> -- > >>> >> >> Clebert Suconic > >>> >> > > >>> >> > > >>> >> > > >>> >> > -- > >>> >> > Clebert Suconic > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Clebert Suconic > >>> >> > >>> > >>> > >>> > >>> -- > >>> Clebert Suconic > >>> > > > > > > > > -- > > Clebert Suconic > > > > -- > Clebert Suconic >
