You can also try setting this compiler flag: -XX:-OmitStackTraceInFastThrow and seeing if that will show the inner stack trace as it might be an optimization to hide it because log4j should usually log the inner stack trace.
>From Oracle: "The compiler in the server VM now provides correct stack backtraces for all "cold" built-in exceptions. For performance purposes, when such an exception is thrown a few times, the method may be recompiled. After recompilation, the compiler may choose a faster tactic using preallocated exceptions that do not provide a stack trace. To disable completely the use of preallocated exceptions, use this new flag: -XX:-OmitStackTraceInFastThrow." On Tue, Jun 14, 2016 at 8:42 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > This has been seen before (e.g. > > http://activemq.2283324.n4.nabble.com/ClassCastException-while-subscribing-to-a-topic-tp4710870.html > , > among others), but because we're not able to see the stack trace for the > inner ClassCastException, we don't know what's causing it. > > The post I linked to is in the case of a durable topic subscription, > whereas your example is with a queue, so it's something general and > cross-cutting, but we don't know what. > > Are you able to reproduce this behavior on a broker whose operation you can > afford to interrupt briefly? If so, please do the following: > * Download the source for the version of ActiveMQ you're using (5.13.3). > * In your IDE of choice, open the ActiveMQ source as new projects. > * Edit the ActiveMQ start script to cause the JVM to open a remote debug > port ( > > http://stackoverflow.com/questions/975271/remote-debugging-a-java-application > ). > * Attach the debugger from your IDE of choice to the broker as a remote > process. The exact steps will depend on your IDE, but are well-documented > for the main IDEs, so Google based on your IDE. > * Set a breakpoint on line 409 of > org.apache.activemq.broker.region.cursors.AbstractStoreCursor. > * When you hit that breakpoint, use your debugger's tools to get the inner > exception (the one for the ClassCastException) and print its stack trace > (e.g. by calling printStackTrace() on it and then looking for that output > either in the IDE's console or in the ActiveMQ log files). In Eclipse, > you'll use the Watch view; other IDEs may have different methods to do > this. > * Show us that inner stack trace, which might let us figure out what's > going on. > > Tim > > On Jun 13, 2016 5:49 AM, "darhon" <darhon...@gmail.com> wrote: > > > I get a lot of errors in activemq.log: > > > > 2016-06-11 22:23:50,616 | ERROR | > > org.apache.activemq.broker.region.cursors.QueueStorePrefetch@queue_name > > > ,batchResetNeeded=false,size=2437,cacheEnabled=false,maxBatchSize:200,hasSpace:true,pendingCachedIds.size:0,lastSyncCachedId:null,lastSyncCachedId-seq:null,lastAsyncCachedId:null,lastAsyncCachedId-seq:null,store=permits:10000,sd=nextSeq:257467,lastRet:MessageOrderCursor:[def:255030, > > low:0, high:0],pending:0 - Failed to fill batch | > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ > > BrokerService[erased] Task-7598 > > java.lang.ClassCastException > > 2016-06-11 22:23:50,618 | ERROR | > > org.apache.activemq.broker.region.cursors.QueueStorePrefetch@queue_name > > > ,batchResetNeeded=false,size=2437,cacheEnabled=false,maxBatchSize:200,hasSpace:true,pendingCachedIds.size:0,lastSyncCachedId:null,lastSyncCachedId-seq:null,lastAsyncCachedId:null,lastAsyncCachedId-seq:null,store=permits:10000,sd=nextSeq:257467,lastRet:MessageOrderCursor:[def:255030, > > low:0, high:0],pending:0 - Failed to fill batch | > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ > > BrokerService[erased] Task-7598 > > java.lang.RuntimeException: java.lang.ClassCastException > > at > > > > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:409)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:149)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:169)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1947)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2172)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.iterate(Queue.java:1619)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)[activemq-client-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)[activemq-client-5.13.3.jar:5.13.3] > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_71] > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_71] > > at java.lang.Thread.run(Thread.java:745)[:1.8.0_71] > > 2016-06-11 22:23:50,619 | ERROR | Failed to page in more queue messages > | > > org.apache.activemq.broker.region.Queue | ActiveMQ BrokerService[erased] > > Task-7598 > > java.lang.RuntimeException: java.lang.RuntimeException: > > java.lang.ClassCastException > > at > > > > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:152)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:169)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1947)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2172)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.broker.region.Queue.iterate(Queue.java:1619)[activemq-broker-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)[activemq-client-5.13.3.jar:5.13.3] > > at > > > > > org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)[activemq-client-5.13.3.jar:5.13.3] > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_71] > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_71] > > at java.lang.Thread.run(Thread.java:745)[:1.8.0_71] > > > > > > but I can not understand what exactly causes this errors. > > > > ActiveMQ version: 5.13.3 > > Configuration master-slave with shared KahaDB on GlusterFS > > > > My current activemq.xml: > > > > <beans > > xmlns="http://www.springframework.org/schema/beans"; > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > > xsi:schemaLocation="http://www.springframework.org/schema/beans > > http://www.springframework.org/schema/beans/spring-beans.xsd > > http://activemq.apache.org/schema/core > > http://activemq.apache.org/schema/core/activemq-core.xsd";> > > > > <bean > > > > > class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> > > <property name="locations"> > > <value>file:${activemq.conf}/credentials.properties</value> > > </property> > > </bean> > > > > <bean id="logQuery" > class="io.fabric8.insight.log.log4j.Log4jLogQuery" > > lazy-init="false" scope="singleton" > > init-method="start" destroy-method="stop"> > > </bean> > > > > <broker xmlns="http://activemq.apache.org/schema/core"; > > brokerName="erased" dataDirectory="/opt/activemq/data" > > schedulerSupport="true" persistent="true"> > > > > <destinationPolicy> > > <policyMap> > > <policyEntries> > > <policyEntry topic=">" > > > <pendingMessageLimitStrategy> > > <constantPendingMessageLimitStrategy limit="2000"/> > > </pendingMessageLimitStrategy> > > </policyEntry> > > <policyEntry queue=">" prioritizedMessages="true" > > useCache="false" expireMessagesPeriod="0" queuePrefetch="1" /> > > </policyEntries> > > </policyMap> > > </destinationPolicy> > > > > <managementContext> > > <managementContext createConnector="true"/> > > </managementContext> > > > > <persistenceAdapter> > > <kahaDB directory="/opt/activemq/data/kahadb"/> > > </persistenceAdapter> > > > > <systemUsage> > > <systemUsage> > > <memoryUsage> > > <memoryUsage percentOfJvmHeap="80" /> > > </memoryUsage> > > <storeUsage> > > <storeUsage limit="40 gb"/> > > </storeUsage> > > <tempUsage> > > <tempUsage limit="10 gb"/> > > </tempUsage> > > </systemUsage> > > </systemUsage> > > > > <transportConnectors> > > <transportConnector name="openwire" > > uri="tcp:// > > > 0.0.0.0:61616?maximumConnections=8000&wireFormat.maxFrameSize=104857600 > > "/> > > <transportConnector name="stomp+nio" > > uri="stomp+nio:// > > > 0.0.0.0:61613?maximumConnections=8000&wireFormat.maxFrameSize=104857600 > > "/> > > </transportConnectors> > > > > <shutdownHooks> > > <bean xmlns="http://www.springframework.org/schema/beans"; > > class="org.apache.activemq.hooks.SpringContextHook" /> > > </shutdownHooks> > > > > </broker> > > > > <import resource="jetty.xml"/> > > > > </beans> > > > > > > Can anyone help to find the cause of the problem? > > > > > > > > -- > > View this message in context: > > > http://activemq.2283324.n4.nabble.com/Failed-to-fill-batch-tp4712924.html > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > >
