Hello,
when I try to shutdown activemq instance it throws exception that it cannot
connect to jmxUrl for process. and then kills the process after 30 secs.
Here is stack strace:
14:34:39 # */opt/gwx/apache-activemq-5.8.0/activemq_8300/bin/activemq_8300
stop "-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://127.0.0.1:8300/jmxrmi
"*
dirname_prg: /opt/gwx/apache-activemq-5.8.0/activemq_8300/bin
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/bin/java'
INFO: Waiting at least 30 seconds for regular process termination of pid
'3234' :
Java Runtime: Oracle Corporation 1.7.0_25 /usr/java/jdk1.7.0_25/jre
Heap sizes: current=1004928k free=999683k max=1004928k
JVM args: -Xms1G -Xmx1G
-Djava.util.logging.config.file=logging.properties
-Dactivemq.classpath=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf;
-Dactivemq.home=/opt/gwx/apache-activemq-5.8.0
-Dactivemq.base=/opt/gwx/apache-activemq-5.8.0/activemq_8300
-Dactivemq.conf=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
-Dactivemq.data=/opt/gwx/apache-activemq-5.8.0/activemq_8300/data
Extensions classpath:
[/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib,/opt/gwx/apache-activemq-5.8.0/lib,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/camel,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/optional,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/web,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/extra,/opt/gwx/apache-activemq-5.8.0/lib/camel,/opt/gwx/apache-activemq-5.8.0/lib/optional,/opt/gwx/apache-activemq-5.8.0/lib/web,/opt/gwx/apache-activemq-5.8.0/lib/extra]
ACTIVEMQ_HOME: /opt/gwx/apache-activemq-5.8.0
ACTIVEMQ_BASE: /opt/gwx/apache-activemq-5.8.0/activemq_8300
ACTIVEMQ_CONF: /opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
ACTIVEMQ_DATA: /opt/gwx/apache-activemq-5.8.0/activemq_8300/data
..Connecting to pid: 3234
......INFO: failed to resolve jmxUrl for pid:3234, using default JMX url
.......Connecting to JMX URL:
service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
..ERROR: java.lang.RuntimeException: Failed to execute stop task. Reason:
java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
java.lang.RuntimeException: Failed to execute stop task. Reason:
java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:116)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
at
org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
ERROR: java.lang.Exception: java.io.IOException: Failed to retrieve
RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
java.lang.Exception: java.io.IOException: Failed to retrieve RMIServer
stub: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:117)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
at
org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
Caused by: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:358)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
at
org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnector(AbstractJmxCommand.java:279)
at
org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnection(AbstractJmxCommand.java:302)
at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:81)
... 11 more
Caused by: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
java.net.ConnectException: Connection refused]
at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
at
com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at
javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1925)
at
javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1892)
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:275)
... 16 more
Caused by: java.rmi.ConnectException: Connection refused to host:
localhost; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
... 21 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at
sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at
sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 26 more
............
INFO: Regular shutdown not successful, sending SIGKILL to process with pid
'3234'
-----------------
Here is what my activemq_8100 script looks like:
## Figure out the ACTIVEMQ_BASE from the directory this script was run from
PRG="$0"
progname=`basename "$0"`
saveddir=`pwd`
# need this for relative symlinks
dirname_prg=`dirname "$PRG"`
cd "$dirname_prg"
echo "dirname_prg: $dirname_prg"
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '.*/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
ACTIVEMQ_BASE=`dirname "$PRG"`/..
cd "$saveddir"
ACTIVEMQ_BASE=`cd "$ACTIVEMQ_BASE" && pwd`
## Add system properties for this instance here (if needed), e.g
#export ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx1G"
#export ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS_MEMORY
-Dorg.apache.activemq.UseDedicatedTaskRunner=true
-Djava.util.logging.config.file=logging.properties"
export
ACTIVEMQ_SUNJMX_CONTROL="-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://
127.0.0.1:8100/jmxrmi"
#
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=8100 "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_BASE}/conf/jmx.password"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_BASE}/conf/jmx.access"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.ssl=false"
#ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote"
export ACTIVEMQ_SUNJMX_START=$ACTIVEMQ_SUNJMX_START
export ACTIVEMQ_HOME=/opt/gwx/apache-activemq-5.8.0
export ACTIVEMQ_BASE=$ACTIVEMQ_BASE
${ACTIVEMQ_HOME}/bin/activemq "$@"
-----------------------------
here is my activemq_8100/conf/activemq.xml file:
<?xml version="1.0" encoding="UTF-8"?><beans xmlns="
http://www.springframework.org/schema/beans"; xmlns:amq="
http://activemq.apache.org/schema/core"; xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd";>
<!-- Allows us to use system properties as variables in this
configuration file -->
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core";
brokerName="activemq_8100" dataDirectory="${activemq.data}">
<!--
For better performances use VM cursor and small memory limit.
For more information, see:
http://activemq.apache.org/message-cursors.html
Also, if your producer is "hanging", it's probably due to
producer flow control.
For more information, see:
http://activemq.apache.org/producer-flow-control.html
-->
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry producerFlowControl="true" topic=">">
<!-- The constantPendingMessageLimitStrategy is used to
prevent
slow topic consumers to block producers and affect
other consumers
by limiting the number of messages that are
retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
<policyEntry memoryLimit="1mb" producerFlowControl="true"
queue=">">
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!--
The transport connectors expose ActiveMQ over a given protocol
to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and
frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:
${openwirePort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:
${amqpPort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans";
class="org.apache.activemq.hooks.SpringContextHook"/>
</shutdownHooks>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<!-- <import resource="jetty.xml"/> -->
</beans>
I have tried to stop without jmxUrl in in stop command and it had same
result.
thanks,
Chirag
