[jira] [Commented] (ARTEMIS-2941) Improve JDBC HA connection resiliency
[ https://issues.apache.org/jira/browse/ARTEMIS-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223760#comment-17223760 ] ASF subversion and git services commented on ARTEMIS-2941: -- Commit e4a2a20c228c5c46838494b5325274ff79ff0145 in activemq-artemis's branch refs/heads/master from franz1981 [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=e4a2a20 ] ARTEMIS-2941 Fixing query timeout value > Improve JDBC HA connection resiliency > - > > Key: ARTEMIS-2941 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2941 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker >Affects Versions: 2.15.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > This is aiming to replace the restart enhancement feature of > https://issues.apache.org/jira/browse/ARTEMIS-2918 because this last one is > too dangerous due to the numerous potential leaks that a server in production > could hit by allowing it to restart while keeping the Java process around. > Currently, JDBC HA uses an expiration time on locks that mark the time by > which a server instance is allowed to keep a specific role, dependent by the > owned lock (live or backup). > Right now, the first failed attempt to renew such expiration time force a > broker to shutdown immediately, while it could be more "relaxed" and just > keep retry until the very end ie when the expiration time is approaching to > end. > > The only concern of this feature is related to the relation between the > broker wall-clock time and the DBMS one, that's used to set the expiration > time and that should be within certain margins. > For this last part I'm aware that classic ActiveMQ lease locks use some > configuration parameter to set the magnitude of the allowed difference (and > to compute some base offset too). > > Right now this feature seems more risk-free and appealing then > https://issues.apache.org/jira/browse/ARTEMIS-2918, given it narrows the > scope of it to what's the very core issue ie a more resilient behaviour on > JDBC lost connectivity. > > To understand the implications of such change, consider a shared store HA > pair with configured 60 seconds of expiration time: > # DBMS goes down > # an in-flight persistent operation on the live data store cause the live > broker to kill itself immediately, because no reliable storage is connected > # backup is unable to renew its backup lease lock > # DBMS goes up in time, before the backup lock local expiration time is ended > # backup is able to renew its backup lease lock and retrieve the very last > state of live (that was live) and, if no script is configured to restart the > live, to failover and take its role > # backup is now live and able to serve clients > > > There are 2 legit questions re potential improvements on this: > # why the live cannot keep re-trying I/O (on the journal, paging or large > messages) until its local expiration time end? > # why the live isn't just returning back an I/O error to the clients? > > The former is complex: the main problem I see is from the resource > utilization point of view; keeping an accumulating backlog of pending > requests, blocked awaiting the last one for an arbitrary long time will > probably cause the broker memory to blown up, to not mention that clients > will timed out too. > The latter seems more appealing, because will allow clients to fail fast, but > it would affect the current semantic we use on the broker storage operations > and I need more investigation to understand how to implement it. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2823) Improve JDBC connection management
[ https://issues.apache.org/jira/browse/ARTEMIS-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223759#comment-17223759 ] ASF subversion and git services commented on ARTEMIS-2823: -- Commit b39c9c9221718cda4a0d3689c9e739e690ec5c5e in activemq-artemis's branch refs/heads/master from franz1981 [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=b39c9c9 ] ARTEMIS-2823 Apply default datasource configs if not overridden > Improve JDBC connection management > -- > > Key: ARTEMIS-2823 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2823 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker >Reporter: Mikko >Assignee: Francesco Nigro >Priority: Major > Time Spent: 16h 20m > Remaining Estimate: 0h > > I have a case where the whole clustering reliability and HA must rely on HA > capabilities of clustered database, and running on top of application server > is not an option. > The current JDBC store implementation is rather bare bones on the connection > management side. JDBC driver is used directly with no management layer. At > startup, the broker just opens couple of direct connections to database and > expects them to be available forever. This is something that cannot be > expected in HA production environment. So, similarly to the discussion linked > below, in our case we lose the db connection after one hour, and all the > brokers need to be restared to get new connections: > [http://activemq.2283324.n4.nabble.com/Artemis-does-not-reconnect-to-MySQL-after-connection-timeout-td4751956.html] > > This is something that could be resolved by simply using JDBC4 isValid > checks, but proper connection handling and pooling through datasource would > be preferrable. > I have implemented a solution for this by using DBCP2 datasource. Our test > cluster has been successfully running this forked version since the release > of Artemis 2.13.0. I will prepare of pull request if this is seen to be > something that can be useful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (ARTEMIS-2823) Improve JDBC connection management
[ https://issues.apache.org/jira/browse/ARTEMIS-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on ARTEMIS-2823 started by Francesco Nigro. > Improve JDBC connection management > -- > > Key: ARTEMIS-2823 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2823 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker >Reporter: Mikko >Assignee: Francesco Nigro >Priority: Major > Time Spent: 16h 20m > Remaining Estimate: 0h > > I have a case where the whole clustering reliability and HA must rely on HA > capabilities of clustered database, and running on top of application server > is not an option. > The current JDBC store implementation is rather bare bones on the connection > management side. JDBC driver is used directly with no management layer. At > startup, the broker just opens couple of direct connections to database and > expects them to be available forever. This is something that cannot be > expected in HA production environment. So, similarly to the discussion linked > below, in our case we lose the db connection after one hour, and all the > brokers need to be restared to get new connections: > [http://activemq.2283324.n4.nabble.com/Artemis-does-not-reconnect-to-MySQL-after-connection-timeout-td4751956.html] > > This is something that could be resolved by simply using JDBC4 isValid > checks, but proper connection handling and pooling through datasource would > be preferrable. > I have implemented a solution for this by using DBCP2 datasource. Our test > cluster has been successfully running this forked version since the release > of Artemis 2.13.0. I will prepare of pull request if this is seen to be > something that can be useful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[
https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223686#comment-17223686
]
Robbie Gemmell edited comment on ARTEMIS-2969 at 10/30/20, 2:47 PM:
The latest fix means the broker does notice the connection going down
immediately which is good, however it also introduces a new problem behaviour.
It seems like each time it disconnects it creates an extra additional new
connection on trying to reconnect. E.g in testing, I've got one
server-connection configured, but after a number of restarts on the peer I now
have traffic logs froming from 4 separate broker connections. As soon as I
discconnected it, it logged twice that it was doing 'retry 2 of -1' (i.e it
skipped logging retry 1). I stopped and started the router a few more times and
ended up with yet more connection logs. I can see the live traffic going for
them all on a protocol trace (broker + router) so they really are actually
there.
E.g on shutting the router down in this state, it logged 4 connections:
{noformat}
2020-10-30 14:35:19.403646 + ROUTER_CORE (info) Router Core thread exited
2020-10-30 14:35:19.403807 + ROUTER_CORE (info) Finalizing core module:
streaming_link_scruber
2020-10-30 14:35:19.403834 + ROUTER_CORE (info) Finalizing core module:
stuck_delivery_detection
2020-10-30 14:35:19.403842 + ROUTER_CORE (info) Finalizing core module:
address_lookup_client
2020-10-30 14:35:19.403875 + SERVER (info) [C1] Closing connection on
shutdown
2020-10-30 14:35:19.403915 + SERVER (info) [C2] Closing connection on
shutdown
2020-10-30 14:35:19.403925 + SERVER (info) [C3] Closing connection on
shutdown
2020-10-30 14:35:19.403929 + SERVER (info) [C4] Closing connection on
shutdown
[0x23cd9b0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x23cd9b0]: IO:FRAME: <- EOS
[0x2396bd0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x2396bd0]: IO:FRAME: <- EOS
[0x231b3b0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x231b3b0]: IO:FRAME: <- EOS
[0x2314620]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
{noformat}
While the broker was logging:
{noformat}
2020-10-30 14:36:04,423 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 46 of -1
***
2020-10-30 14:36:04,424 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 47 of -1
***
2020-10-30 14:36:04,426 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 48 of -1
***
2020-10-30 14:36:04,427 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 49 of -1
***
2020-10-30 14:36:04,429 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 50 of -1
***
{noformat}
It appears the (now 5) different connections end up sharing the same reconnect
counter (which perhaps also explains the initial 'retry 2' thing I noted
seeing).
was (Author: gemmellr):
The latest fix means the broker does notice the connection going down
immediately which is good,
[jira] [Commented] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[
https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223686#comment-17223686
]
Robbie Gemmell commented on ARTEMIS-2969:
-
The latest fix means the broker does notice the connection going down
immediately which is good, however it also introduces a new problem behaviour.
It seems like each time it disconnects it creates an extra additional new
connection on trying to reconnect. E.g in testing, I've got one
server-connection configured, but after a number of restarts on the peer I now
have traffic logs froming from 4 separate broker connections. As soon as I
discconnected it, it logged twice that it was doing 'retry 2 of -1' (i.e it
skipped logging retry 1). I stopped and started the router a few more times and
ended up with yet more connection logs. I can see the live traffic going for
them all on a protocol trace (broker + router) so they really are actually
there.
E.g on shutting the router down in this state, it logged:
{noformat}
2020-10-30 14:35:19.403646 + ROUTER_CORE (info) Router Core thread exited
2020-10-30 14:35:19.403807 + ROUTER_CORE (info) Finalizing core module:
streaming_link_scruber
2020-10-30 14:35:19.403834 + ROUTER_CORE (info) Finalizing core module:
stuck_delivery_detection
2020-10-30 14:35:19.403842 + ROUTER_CORE (info) Finalizing core module:
address_lookup_client
2020-10-30 14:35:19.403875 + SERVER (info) [C1] Closing connection on
shutdown
2020-10-30 14:35:19.403915 + SERVER (info) [C2] Closing connection on
shutdown
2020-10-30 14:35:19.403925 + SERVER (info) [C3] Closing connection on
shutdown
2020-10-30 14:35:19.403929 + SERVER (info) [C4] Closing connection on
shutdown
[0x23cd9b0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x23cd9b0]: IO:FRAME: <- EOS
[0x2396bd0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x2396bd0]: IO:FRAME: <- EOS
[0x231b3b0]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
[0x231b3b0]: IO:FRAME: <- EOS
[0x2314620]: AMQP:FRAME:0 -> @close(24) [error=@error(29)
[condition=:"amqp:connection:framing-error", description="connection aborted"]]
{noformat}
While the broker was logging:
{noformat}
2020-10-30 14:36:04,423 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 46 of -1
***
2020-10-30 14:36:04,424 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 47 of -1
***
2020-10-30 14:36:04,426 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 48 of -1
***
2020-10-30 14:36:04,427 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 49 of -1
***
2020-10-30 14:36:04,429 INFO
[org.apache.activemq.artemis.protocol.amqp.logger] AMQ111002:
***
Retrying Server AMQP Connection my-router on localhost:6000 retry 50 of -1
***
{noformat}
It appears the (now 5) different connections end up sharing the same reconnect
counter (which perhaps also explains the initial 'retry 2' thing I noted
seeing).
> broker doesnt handle server-connection being severed
>
>
> Key: ARTEMIS-2969
> URL:
[jira] [Reopened] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[ https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell reopened ARTEMIS-2969: - > broker doesnt handle server-connection being severed > > > Key: ARTEMIS-2969 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2969 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The broker doesnt notice or handle a server-connection being severed / timing > out. > Start a broker, having instructed it to connect to a router (i.e 'peer' > configuration). Stop the router (CTRL-C). Start the router again. Broker > doesnt seem to notice the server-connection is gone and doesnt reconnect. It > actually sits generating heartbeat frames until its 60sec actual-timeout is > reached...at which point proton generates a close frame indicating the > timeout. The broker still doesnt notice the connection is gone, and still > doesnt reconnect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2860) Console -- Delete/Create/Brows/Send buttons don't appear until after a page refresh
[ https://issues.apache.org/jira/browse/ARTEMIS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223678#comment-17223678 ] Domenico Francesco Bruscino commented on ARTEMIS-2860: -- This issue should be fixed by ARTEMIS-2838. It will be included in the next release 2.16. > Console -- Delete/Create/Brows/Send buttons don't appear until after a page > refresh > --- > > Key: ARTEMIS-2860 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2860 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Web Console >Affects Versions: 2.14.0 >Reporter: BENJAMIN Lee WARRICK >Priority: Minor > > I recently upgraded to Artemis 2.14.0. Now when I go to an address or queue > in the console, I have to refresh the page or the Delete/Create/Brows/Send > buttons don't appear. > Windows 10 > Chrome 83.0.4103.116 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AMQ-6899) Runtime Configuration Plugin throwing NPE at startup
[
https://issues.apache.org/jira/browse/AMQ-6899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223672#comment-17223672
]
Matt Pavlovich commented on AMQ-6899:
-
This is caused by the spring-beans.xsd not being present in the activemq-osgi
jar here:
https://github.com/apache/activemq/blob/master/activemq-runtime-config/src/main/java/org/apache/activemq/plugin/RuntimeConfigurationBroker.java#L236
{noformat}
schemas.add(new
StreamSource(getClass().getResource("/org/springframework/beans/factory/xml/spring-beans-3.0.xsd").toExternalForm()));
{noformat}
> Runtime Configuration Plugin throwing NPE at startup
>
>
> Key: AMQ-6899
> URL: https://issues.apache.org/jira/browse/AMQ-6899
> Project: ActiveMQ
> Issue Type: Bug
> Components: Plugin
>Affects Versions: 5.13.3
> Environment: Karaf
> both Windows and Linux (CentOS 7)
>Reporter: Peter Berkman
>Assignee: Matt Pavlovich
>Priority: Minor
>
> NOTE: running in Karaf
>
> Here is the plugin declaration from activemq.xml - note: have tried by with
> and without the checkPeriod - both throw the same NPE:
> {code:java}
>
>
>
>
>
>
> {code}
> The configuration seems to load fine, but this exception is thrown on every
> startup.
>
>
> {code:java}
> 20180213 16:34:25.556 [INFO ] Broker Starting Thread |
> 150:org.apache.activemq.activemq-osgi |
> org.apache.activemq.plugin.AbstractRuntimeConfigurationBroker | Unexpected
> exception during load of: URL
> [file:/D:/Nextgate/products/ngs/etc/activemq.xml]
> java.lang.NullPointerException
> at
> org.apache.activemq.plugin.RuntimeConfigurationBroker.getSchema(RuntimeConfigurationBroker.java:232)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.plugin.RuntimeConfigurationBroker.loadConfiguration(RuntimeConfigurationBroker.java:178)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.plugin.RuntimeConfigurationBroker.start(RuntimeConfigurationBroker.java:82)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.broker.BrokerService$6.start(BrokerService.java:2324)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:732)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.broker.BrokerService.access$300(BrokerService.java:139)[150:org.apache.activemq.activemq-osgi:5.13.3]
> at
> org.apache.activemq.broker.BrokerService$3.run(BrokerService.java:711)[150:org.apache.activemq.activemq-osgi:5.13.3]
> 20180213 16:34:25.568 [INFO ] Broker Starting Thread |
> 150:org.apache.activemq.activemq-osgi |
> org.apache.activemq.transport.TransportServerThreadSupport | Listening for
> connections at:
> tcp://127.0.0.1:8206?transport.threadName=1000=104857600
>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[ https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223665#comment-17223665 ] Clebert Suconic commented on ARTEMIS-2969: -- the outcome of this is nice, we now have a complete solution.. dealing with the TTL and the disconnects.. thanks for raising this. > broker doesnt handle server-connection being severed > > > Key: ARTEMIS-2969 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2969 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The broker doesnt notice or handle a server-connection being severed / timing > out. > Start a broker, having instructed it to connect to a router (i.e 'peer' > configuration). Stop the router (CTRL-C). Start the router again. Broker > doesnt seem to notice the server-connection is gone and doesnt reconnect. It > actually sits generating heartbeat frames until its 60sec actual-timeout is > reached...at which point proton generates a close frame indicating the > timeout. The broker still doesnt notice the connection is gone, and still > doesnt reconnect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2937) AMQP Server Connectivity
[ https://issues.apache.org/jira/browse/ARTEMIS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223660#comment-17223660 ] ASF subversion and git services commented on ARTEMIS-2937: -- Commit f3dde91b911e73cc9733a06fe00a72ec7d2249c0 in activemq-artemis's branch refs/heads/master from Clebert Suconic [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=f3dde91 ] ARTEMIS-2969 / ARTEMIS-2937 Dealing with instant disconnects on amqp broker connections > AMQP Server Connectivity > > > Key: ARTEMIS-2937 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2937 > Project: ActiveMQ Artemis > Issue Type: New Feature > Components: AMQP >Reporter: Clebert Suconic >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 35h 20m > Remaining Estimate: 0h > > This feature adds server side connectivity. > > It is possible to link two brokers directly using AMQP with this feature, and > have a Queue transferring messages to another broker directly. > > For this we would have options called > > it would also be possible to use qpid-dispatch as an intermediary between > clients and the brokers (or eventually between brokers), on that case the > option will be > it would also be possible to use between two brokers, bringing the possibility of using it for Disaster & > Recovery and Failover. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[ https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223659#comment-17223659 ] ASF subversion and git services commented on ARTEMIS-2969: -- Commit f3dde91b911e73cc9733a06fe00a72ec7d2249c0 in activemq-artemis's branch refs/heads/master from Clebert Suconic [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=f3dde91 ] ARTEMIS-2969 / ARTEMIS-2937 Dealing with instant disconnects on amqp broker connections > broker doesnt handle server-connection being severed > > > Key: ARTEMIS-2969 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2969 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The broker doesnt notice or handle a server-connection being severed / timing > out. > Start a broker, having instructed it to connect to a router (i.e 'peer' > configuration). Stop the router (CTRL-C). Start the router again. Broker > doesnt seem to notice the server-connection is gone and doesnt reconnect. It > actually sits generating heartbeat frames until its 60sec actual-timeout is > reached...at which point proton generates a close frame indicating the > timeout. The broker still doesnt notice the connection is gone, and still > doesnt reconnect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[ https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223652#comment-17223652 ] Clebert Suconic commented on ARTEMIS-2969: -- I have missed implementing channelInactive on the new Channel class.. this PR is dealing with that : https://github.com/apache/activemq-artemis/pull/3323 > broker doesnt handle server-connection being severed > > > Key: ARTEMIS-2969 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2969 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The broker doesnt notice or handle a server-connection being severed / timing > out. > Start a broker, having instructed it to connect to a router (i.e 'peer' > configuration). Stop the router (CTRL-C). Start the router again. Broker > doesnt seem to notice the server-connection is gone and doesnt reconnect. It > actually sits generating heartbeat frames until its 60sec actual-timeout is > reached...at which point proton generates a close frame indicating the > timeout. The broker still doesnt notice the connection is gone, and still > doesnt reconnect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARTEMIS-2784) Could not decode properties for CoreMessage ... java.lang.IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/ARTEMIS-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noah Zucker resolved ARTEMIS-2784.
--
Resolution: Resolved
Happy to report that upgrading to the current Artemis MQ client library
resolved this issue!
> Could not decode properties for CoreMessage ...
> java.lang.IndexOutOfBoundsException
> ---
>
> Key: ARTEMIS-2784
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2784
> Project: ActiveMQ Artemis
> Issue Type: Bug
>Affects Versions: 2.12.0
>Reporter: Noah Zucker
>Priority: Major
>
> Our application is observing a large number of messages from Artemis
> experiencing the following error
> Server Version: ArtemisMQ 2.12.0
> Client Version: org.hornetq hornetq-core 2.2.7.Final
> {code}
> Could not decode properties for
> CoreMessage[messageID=0,durable=true,userID=null,priority=4, ti
> mestamp=1590674773446,expiration=0,address=null, propertiesLocation=145:
> java.lang.IndexOutOfBoundsException
> activemq-artemis 2020-05-28 14:09:41,364 WARN
> [org.apache.activemq.artemis.core.message.impl.CoreMessage] Failed message
> has messageID=0 and the following buffer:
> activemq-artemis +-+
> activemq-artemis | 0 1 2 3 4 5 6 7 8 9 a b c d e f |
> activemq-artemis
> ++-++
> activemq-artemis || 00 00 00 7d 00 00 00 66 00 66 7b 20 22 74 22 20
> |...}...f.f{ "t" |
> activemq-artemis |0010| 3a 20 22 63 6f 6d 2e 6e 6f 76 75 73 2e 61 6e 61
> |: "com.novus.ana|
> activemq-artemis |0020| 6c 79 74 69 63 73 2e 75 74 69 6c 2e 74 61 73 6b
> |lytics.util.task|
> activemq-artemis |0030| 73 2e 41 73 79 6e 63 43 6f 6e 73 75 6d 65 72 4d
> |s.AsyncConsumerM|
> activemq-artemis |0040| 65 73 73 61 67 65 22 20 2c 20 22 6d 73 67 22 20
> |essage" , "msg" |
> activemq-artemis |0050| 3a 20 22 66 65 61 74 75 72 65 46 6c 61 67 43 61
> |: "featureFlagCa|
> activemq-artemis |0060| 63 68 65 2e 69 6e 76 61 6c 69 64 61 74 65 22 7d
> |che.invalidate"}|
> activemq-artemis |0070| 00 00 00 f2 00 00 00 00 00 00 00 00 00 00 03 ff
> ||
> activemq-artemis |0080| 00 00 00 00 00 00 00 00 00 00 01 72 5b 9a d5 c6
> |...r[...|
> activemq-artemis |0090| 04 01 00 00 00 01 00 00 00 16 5f 00 48 00 51 00
> |.._.H.Q.|
> activemq-artemis |00a0| 5f 00 44 00 55 00 50 00 4c 00 5f 00 49 00 44 00
> |_.D.U.P.L._.I.D.|
> activemq-artemis |00b0| 0a 00 00 00 30 35 00 65 00 63 00 66 00 63 00 35
> |05.e.c.f.c.5|
> activemq-artemis |00c0| 00 35 00 35 00 64 00 39 00 31 00 33 00 62 00 32
> |.5.5.d.9.1.3.b.2|
> activemq-artemis |00d0| 00 34 00 65 00 66 00 30 00 32 00 32 00
> |.4.e.f.0.2.2. |
> activemq-artemis
> ++-++
> {code}
> Our client-side is also having trouble reading incoming text messages in its
> onMessage callback. We encounter this with frequency: {{Unable to parse
> string bodybuffer from Msg 'class
> org.hornetq.core.client.impl.ClientMessageImpl' having id=4530593 !!!
> Details: java.lang.IndexOutOfBoundsException: Not enough readable bytes -
> Need 24, maximum is 20}}
> Apologies for being open ended on this, please let me know what additional
> details to provide.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARTEMIS-2971) outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms
[ https://issues.apache.org/jira/browse/ARTEMIS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated ARTEMIS-2971: Description: The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoing 'server-connection' feature to have the broker establish connections to other servers. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it should at least be supported if not the default. was: The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoing 'server-connection' feature to have the broker establish connections to other servers. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. > outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL > mechanisms > - > > Key: ARTEMIS-2971 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2971 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > > The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms > for connecting AMQP clients to the broker. However, it doesnt support either > for the outgoing 'server-connection' feature to have the broker establish > connections to other servers. This means an Artemis broker may be unable to > connect to another Artemis broker using this feature, depending on its > configuration. > The broker currently only supports using PLAIN if a user+pass is supplied for > the outgoing connection, or using a raw AMQP (no SASL) connection if no > user+pass is supplied. This means the broker cant connect to a server unless > it either offers PLAIN, or accepts bare AMQP without SASL auth (which some > dont, e.g I think ActiveMQ 5 by default at least). > > EXTERNAL is going to be fairly desirable for such use cases. Using SASL > ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical > default, but it should at least be supported if not the default. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARTEMIS-2971) outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms
[ https://issues.apache.org/jira/browse/ARTEMIS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated ARTEMIS-2971: Description: The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoing 'server-connection' feature to have the broker establish connections to other servers. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. was: The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoingserver-connection feature to have the broker establish connections. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. > outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL > mechanisms > - > > Key: ARTEMIS-2971 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2971 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > > The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms > for connecting AMQP clients to the broker. However, it doesnt support either > for the outgoing 'server-connection' feature to have the broker establish > connections to other servers. This means an Artemis broker may be unable to > connect to another Artemis broker using this feature, depending on its > configuration. > The broker currently only supports using PLAIN if a user+pass is supplied for > the outgoing connection, or using a raw AMQP (no SASL) connection if no > user+pass is supplied. This means the broker cant connect to a server unless > it either offers PLAIN, or accepts bare AMQP without SASL auth (which some > dont, e.g I think ActiveMQ 5 by default at least). > > EXTERNAL is going to be fairly desirable for such use cases. Using SASL > ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical > default, but it shoul at least be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARTEMIS-2971) outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms
[ https://issues.apache.org/jira/browse/ARTEMIS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated ARTEMIS-2971: Description: The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoingserver-connection feature to have the broker establish connections. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. was: The broker supports using ANONYMOUS and EXTERNAL SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoingserver-connection feature to have the broker establish connections. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. > outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL > mechanisms > - > > Key: ARTEMIS-2971 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2971 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > > The broker supports using ANONYMOUS and EXTERNAL (and PLAIN) SASL mechanisms > for connecting AMQP clients to the broker. However, it doesnt support either > for the outgoingserver-connection feature to have the broker establish > connections. This means an Artemis broker may be unable to connect to another > Artemis broker using this feature, depending on its configuration. > The broker currently only supports using PLAIN if a user+pass is supplied for > the outgoing connection, or using a raw AMQP (no SASL) connection if no > user+pass is supplied. This means the broker cant connect to a server unless > it either offers PLAIN, or accepts bare AMQP without SASL auth (which some > dont, e.g I think ActiveMQ 5 by default at least). > > EXTERNAL is going to be fairly desirable for such use cases. Using SASL > ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical > default, but it shoul at least be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-1730) Server leaks memory when messages are expired without an expiry-address
[ https://issues.apache.org/jira/browse/ARTEMIS-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223633#comment-17223633 ] ASF subversion and git services commented on ARTEMIS-1730: -- Commit 6772314488c7ff8baa9b5fe160790dc8de588e66 in activemq-artemis's branch refs/heads/master from Shrikant Chavan [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=6772314 ] ARTEMIS-1730 Adding Restart Sequence of brokers on doc > Server leaks memory when messages are expired without an expiry-address > --- > > Key: ARTEMIS-1730 > URL: https://issues.apache.org/jira/browse/ARTEMIS-1730 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Affects Versions: 2.4.0 >Reporter: Ilkka Virolainen >Assignee: Justin Bertram >Priority: Major > Fix For: 2.5.0 > > > When messages are being expired from an address that has an empty > expiry-address they should be dropped. At the moment what happens is that the > broker logs a message indicating that the message is being dropped: > AMQ222146: Message has expired. No bindings for Expiry Address so dropping > it > However, the messages are never acknowledged so they end up showing as in > delivery. ExpiredCount for the queue is never incremented while > DeliveringCount is. This results in increased memory consumption and as the > amount of expiring messages increase the broker eventually runs out of memory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2937) AMQP Server Connectivity
[ https://issues.apache.org/jira/browse/ARTEMIS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223625#comment-17223625 ] ASF subversion and git services commented on ARTEMIS-2937: -- Commit dff2ed36383f1e52304eff2705bf43e01e3c3f10 in activemq-artemis's branch refs/heads/master from Clebert Suconic [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=dff2ed3 ] ARTEMIS-2937 Implementing skip create and skip delete on Mirror Source > AMQP Server Connectivity > > > Key: ARTEMIS-2937 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2937 > Project: ActiveMQ Artemis > Issue Type: New Feature > Components: AMQP >Reporter: Clebert Suconic >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 35h 20m > Remaining Estimate: 0h > > This feature adds server side connectivity. > > It is possible to link two brokers directly using AMQP with this feature, and > have a Queue transferring messages to another broker directly. > > For this we would have options called > > it would also be possible to use qpid-dispatch as an intermediary between > clients and the brokers (or eventually between brokers), on that case the > option will be > it would also be possible to use between two brokers, bringing the possibility of using it for Disaster & > Recovery and Failover. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2960) Update httpcomponents
[ https://issues.apache.org/jira/browse/ARTEMIS-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223632#comment-17223632 ] ASF subversion and git services commented on ARTEMIS-2960: -- Commit ed2c936f24223cc837d50efb7c0807b915a671dd in activemq-artemis's branch refs/heads/master from Domenico Francesco Bruscino [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=ed2c936 ] ARTEMIS-2960 Update httpcomponents > Update httpcomponents > - > > Key: ARTEMIS-2960 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2960 > Project: ActiveMQ Artemis > Issue Type: Task >Reporter: Domenico Francesco Bruscino >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update httpcore version to 4.4.13 and httpclient version to 4.5.13. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2971) outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms
[ https://issues.apache.org/jira/browse/ARTEMIS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223624#comment-17223624 ] Robbie Gemmell commented on ARTEMIS-2971: - Related, if the broker tries to connect with user+pass configured to a server that doesnt support PLAIN, it appears to go into an infinite loop and so generates a StackOverflowError before failing the attempt as result. It then repeats this each reconnect-interval. > outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL > mechanisms > - > > Key: ARTEMIS-2971 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2971 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > > The broker supports using ANONYMOUS and EXTERNAL SASL mechanisms for > connecting AMQP clients to the broker. However, it doesnt support either for > the outgoingserver-connection feature to have the broker establish > connections. This means an Artemis broker may be unable to connect to another > Artemis broker using this feature, depending on its configuration. > The broker currently only supports using PLAIN if a user+pass is supplied for > the outgoing connection, or using a raw AMQP (no SASL) connection if no > user+pass is supplied. This means the broker cant connect to a server unless > it either offers PLAIN, or accepts bare AMQP without SASL auth (which some > dont, e.g I think ActiveMQ 5 by default at least). > > EXTERNAL is going to be fairly desirable for such use cases. Using SASL > ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical > default, but it shoul at least be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARTEMIS-2971) outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms
Robbie Gemmell created ARTEMIS-2971: --- Summary: outgoing 'server-connection' doesnt support ANONYMOUS or EXTERNAL SASL mechanisms Key: ARTEMIS-2971 URL: https://issues.apache.org/jira/browse/ARTEMIS-2971 Project: ActiveMQ Artemis Issue Type: Bug Affects Versions: 2.16.0 Reporter: Robbie Gemmell Assignee: Clebert Suconic The broker supports using ANONYMOUS and EXTERNAL SASL mechanisms for connecting AMQP clients to the broker. However, it doesnt support either for the outgoingserver-connection feature to have the broker establish connections. This means an Artemis broker may be unable to connect to another Artemis broker using this feature, depending on its configuration. The broker currently only supports using PLAIN if a user+pass is supplied for the outgoing connection, or using a raw AMQP (no SASL) connection if no user+pass is supplied. This means the broker cant connect to a server unless it either offers PLAIN, or accepts bare AMQP without SASL auth (which some dont, e.g I think ActiveMQ 5 by default at least). EXTERNAL is going to be fairly desirable for such use cases. Using SASL ANONYMOUS for connections without user+pass/EXTERNAL woud be a more typical default, but it shoul at least be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARTEMIS-2970) Implement a test validating Connection timeout with broker connection that does not depend on qpid-dispatch
[ https://issues.apache.org/jira/browse/ARTEMIS-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated ARTEMIS-2970: Summary: Implement a test validating Connection timeout with broker connection that does not depend on qpid-dispatch (was: Implement a test validating Connection timeout with broker connection that does not depend on pid-dispatch) > Implement a test validating Connection timeout with broker connection that > does not depend on qpid-dispatch > --- > > Key: ARTEMIS-2970 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2970 > Project: ActiveMQ Artemis > Issue Type: Task >Reporter: Clebert Suconic >Assignee: Clebert Suconic >Priority: Critical > Fix For: 2.17.0 > > > This is a must do task before we release 2.17.0 > > During 2.16.0 I added a test to validate timeouts that depends on > qpid-dispatch, however the test is optional and for dev purpose only. I need > a more consistent test as part of the test suite. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2969) broker doesnt handle server-connection being severed
[ https://issues.apache.org/jira/browse/ARTEMIS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223615#comment-17223615 ] Robbie Gemmell commented on ARTEMIS-2969: - The changes are an improvement, but the behaviour out of the box is still pretty poor and should definitely be improved further. The broker now eventually notices nothing was sent after the actual 60sec timeout (it advertised a 30sec idle-timeout, half the real period, in keeping with the AMQP spec) and then chops the connection locally as a result, sparking a reconnect after the reconnect-delay. Total time to notice and reconnect, around 61 seconds (60 timeout + 1 reconnect delay). In my case this was well over 60seconds after the peer was restored. The broker should really notice the connection being severed essentially instantly (which netty will; or at least does in another component I had connected to the same peer at the same point), giving at most time before reconnecting (assuming the peer is back). > broker doesnt handle server-connection being severed > > > Key: ARTEMIS-2969 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2969 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.16.0 >Reporter: Robbie Gemmell >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The broker doesnt notice or handle a server-connection being severed / timing > out. > Start a broker, having instructed it to connect to a router (i.e 'peer' > configuration). Stop the router (CTRL-C). Start the router again. Broker > doesnt seem to notice the server-connection is gone and doesnt reconnect. It > actually sits generating heartbeat frames until its 60sec actual-timeout is > reached...at which point proton generates a close frame indicating the > timeout. The broker still doesnt notice the connection is gone, and still > doesnt reconnect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2937) AMQP Server Connectivity
[ https://issues.apache.org/jira/browse/ARTEMIS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223583#comment-17223583 ] ASF subversion and git services commented on ARTEMIS-2937: -- Commit 7a5f325b721fcbb7169cca1a0d0dc5de9f6d06cc in activemq-artemis's branch refs/heads/master from Robbie Gemmell [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=7a5f325 ] ARTEMIS-2937: remove a couple of unecessary lines for alt stores, left over by mistake (c error) > AMQP Server Connectivity > > > Key: ARTEMIS-2937 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2937 > Project: ActiveMQ Artemis > Issue Type: New Feature > Components: AMQP >Reporter: Clebert Suconic >Assignee: Clebert Suconic >Priority: Major > Fix For: 2.16.0 > > Time Spent: 35h 20m > Remaining Estimate: 0h > > This feature adds server side connectivity. > > It is possible to link two brokers directly using AMQP with this feature, and > have a Queue transferring messages to another broker directly. > > For this we would have options called > > it would also be possible to use qpid-dispatch as an intermediary between > clients and the brokers (or eventually between brokers), on that case the > option will be > it would also be possible to use between two brokers, bringing the possibility of using it for Disaster & > Recovery and Failover. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARTEMIS-2860) Console -- Delete/Create/Brows/Send buttons don't appear until after a page refresh
[ https://issues.apache.org/jira/browse/ARTEMIS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223581#comment-17223581 ] ncasaux commented on ARTEMIS-2860: -- I confirm the issue, impossible to get the "Delete","Send" or "Create" buttons when I pick an address. I have to refresh the page to get them > Console -- Delete/Create/Brows/Send buttons don't appear until after a page > refresh > --- > > Key: ARTEMIS-2860 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2860 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Web Console >Affects Versions: 2.14.0 >Reporter: BENJAMIN Lee WARRICK >Priority: Minor > > I recently upgraded to Artemis 2.14.0. Now when I go to an address or queue > in the console, I have to refresh the page or the Delete/Create/Brows/Send > buttons don't appear. > Windows 10 > Chrome 83.0.4103.116 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AMQ-7514) Replace racially charged terms throughout source code, comments and documentation
[ https://issues.apache.org/jira/browse/AMQ-7514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223562#comment-17223562 ] Scott M Messner commented on AMQ-7514: -- Regardless of the the racial charge of the 'master' and 'slave' terminology – it's actually quite incorrect for what it implies and the actual usage. For example: I use [https://activemq.apache.org/jdbc-master-slave] . I think the fitting term is 'active' and 'passive'. I can maybe help in refactoring the doc, but I admit, I've very little experience in open source contributions. Do I just check out the doc, modify, push, and make a pull-request ? Is there a procedure written somewhere for getting started making contributions ? > Replace racially charged terms throughout source code, comments and > documentation > - > > Key: AMQ-7514 > URL: https://issues.apache.org/jira/browse/AMQ-7514 > Project: ActiveMQ > Issue Type: Task >Reporter: Bruce Snyder >Priority: Major > > Given the racial charged nature of certain terms in today's world, we must > pull together to create a plan for changing any such terms throughout all the > ActiveMQ projects and in the git repos themselves. > > Example: [https://activemq.apache.org/masterslave.html] > > Here are just a few terms that should be changed: * The following terms are > being targeted for change: > * > ** 'master' and 'slave' should be replaced with the terms 'live' and 'backup' > ** 'whitelist' and 'blacklist' should be replaced with the terms 'allowlist' > and 'denylist' > * Rename all the git 'master' branches to the term 'main' > Proposal notes from activemq-dev mailing list > Phase 1: > 1. Deprecate terms such as ‘master’ and ’slave > 2. log.warn any configuration change notifications > 3. Provide compatibility under the covers for deprecated terms > 4. Provide any openwire compatibility changes b/w ActiveMQ 5 and Artemis > 5. Notify users in an announcement and provide a conversion HOWTO > Phase 2: > 1. Remove terminology as part of a major or minor release (SEMVER where ‘y’ > in ‘x.y.z’ is minor version number) > New terms: > a. For shared storage: ‘active’ and ’standby’ > b. For replication: ‘primary’ and ‘replica' > c. For 'white list' and 'blacklist': 'allow list' and 'deny list' > For example: > ‘master’ -> ‘active’ > ’slave’ -> ’standby' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506631=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506631 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 10:38 Start Date: 30/Oct/20 10:38 Worklog Time Spent: 10m Work Description: uomik commented on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719477325 Correct. With default pool configuration, we had issues. But by using more aggressive settings everything works fine. And all this because our database closes connections after one hour, being active or not. This is not a general issue, I suppose. The extra pool settings are: - timeBetweenEvictionRunsMillis: "90" - numTestsPerEvictionRun: "3" - testWhileIdle: "true" - testOnReturn: "true" - testOnBorrow: "true" - maxWaitMillis: "6" - minIdle: "0" - maxConnLifetimeMillis: "180" - removeAbandonedOnMaintenance: "true" And yes, the snapshot we are using was made before ARTEMIS-2941. Unfortunately we will have to wait until the official 2.16.0 release until update, for "organisational" reasons. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506631) Time Spent: 3h 40m (was: 3.5h) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 3h 40m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506616 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:44 Start Date: 30/Oct/20 09:44 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false`, `poolPreparedStatements` is `true` and `maxIdle` is 8. My expectations are: 1. if a page/large-message load would happen, the number of connections could grow unbounded (along with the related prepared statements) 2. Given that `testWhileIdle` is `false`, there is no object evictor that would test any idle connection until borrowed again But > that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs That seems more a pool "hidden" logic: given that a connection has some prepared statement it cannot be assumed as idle, despite none is checking that the connection is still valid. That's correct? I'm going to run some test to verify that. I don't see yet the correlation with lock renew failures if you're using the latest master: see https://issues.apache.org/jira/browse/ARTEMIS-2941 for more info. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506616) Time Spent: 3.5h (was: 3h 20m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 3.5h > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506608 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:33 Start Date: 30/Oct/20 09:33 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of connections could grow unbounded (along with the related prepared statements) 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max number of parallel pages/large msg requested and won't change in background until some of these will be borrowed and ack as invalid 2. Nodes in HA pairs should not suicide due to temporary connectivity loss unless the expiration timeout is exausted or there is some large msg/paging/journal activity (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506608) Time Spent: 3h 20m (was: 3h 10m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 3h 20m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506607=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506607 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:32 Start Date: 30/Oct/20 09:32 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of connections could grow unbounded (along with the related prepared statements) 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max number of parallel pages/large msg requested and won't change in background until some of these will be borrowed and ack as invalid 2. Nodes in HA pairs should not restart due to temporary connectivity loss unless the expiration timeout is exausted or there is some large msg/paging/journal activity (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506607) Time Spent: 3h 10m (was: 3h) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 3h 10m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506606 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:31 Start Date: 30/Oct/20 09:31 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of connections could grow unbounded (along with the related prepared statements) 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max number of parallel pages/large msg requested and won't change in background until some of these will be borrowed and ack as invalid 2. Nodes in HA pairs should restart due to connection loss unless live AND DBMS activity on journal or slave and the expiration timeout is exausted (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506606) Time Spent: 3h (was: 2h 50m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 3h > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506603=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506603 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:28 Start Date: 30/Oct/20 09:28 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of connections could grow unbounded (along with the related prepared statements) 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max thread pool size ie number of parallel pages/large msg requested and won't decrease until some of these be borrowed and ack as invalid 2. Nodes in HA pairs should restart due to connection loss unless live AND DBMS activity on journal or slave and the expiration timeout is exausted (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506603) Time Spent: 2h 50m (was: 2h 40m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506602=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506602 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:28 Start Date: 30/Oct/20 09:28 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of live connections could grow unbounded (along with the related prepared statements) 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max thread pool size ie number of parallel pages/large msg requested and won't decrease until some of these be borrowed and ack as invalid 2. Nodes in HA pairs should restart due to connection loss unless live AND DBMS activity on journal or slave and the expiration timeout is exausted (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506602) Time Spent: 2h 40m (was: 2.5h) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506600=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506600 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:26 Start Date: 30/Oct/20 09:26 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of live connections could grow over `maxTotal` along with the related prepared statements. 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max thread pool size ie number of parallel pages/large msg requested and won't decrease until some of these be borrowed and ack as invalid 2. Nodes in HA pairs should restart due to connection loss unless live AND DBMS activity on journal or slave and the expiration timeout is exausted (see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506600) Time Spent: 2.5h (was: 2h 20m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2.5h > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506598=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506598 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:24 Start Date: 30/Oct/20 09:24 Worklog Time Spent: 10m Work Description: franz1981 commented on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of live connections could grow over `maxTotal` along with the related prepared statements. 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max thread pool size ie number of parallel pages/large msg requested and won't decrease until some of these be borrowed and ack as invalid 2. > The nodes randomly reboot themselves due to connection issues when renewing live locks This shouldn't happen using the latest master, see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) -> are you using the latest master? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506598) Time Spent: 2h 10m (was: 2h) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2h 10m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506599=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506599 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 09:24 Start Date: 30/Oct/20 09:24 Worklog Time Spent: 10m Work Description: franz1981 edited a comment on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719442315 HI @uomik! Sorry I've missed your first comment on > It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs We're talking about default pool configuration right? It means `maxTotal` is infinite, `testWhileIdle` is `false` and `poolPreparedStatements` is `true`. My expectations are: 1. if a page/large-message load would happen, the number of live connections could grow over `maxTotal` along with the related prepared statements. 2. Given that ``testWhileIdle` is `false`, there is no object evictor that would test these connections until borrowed again: that's the only chance to acknowledge a network connectivity loss (and evict the related prepared statements too) and try to replace the invalid connection with a new one (and some new pooled statements too) It means 2 things to me: 1. the number of available alive connections (+ prepared statements) will grow to accomodate the max thread pool size ie number of parallel pages/large msg requested and won't decrease until some of these be borrowed and ack as invalid 2. > The nodes randomly reboot themselves due to connection issues when renewing live locks This shouldn't happen using the latest master, see [ARTEMIS-2941 Improve JDBC HA connection resiliency](https://issues.apache.org/jira/browse/ARTEMIS-2941) -> are you using the latest master? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506599) Time Spent: 2h 20m (was: 2h 10m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2955) commons-dbcp2 performance issue with Derby Embedded DBMS
[ https://issues.apache.org/jira/browse/ARTEMIS-2955?focusedWorklogId=506589=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506589 ] ASF GitHub Bot logged work on ARTEMIS-2955: --- Author: ASF GitHub Bot Created on: 30/Oct/20 08:55 Start Date: 30/Oct/20 08:55 Worklog Time Spent: 10m Work Description: uomik commented on pull request #3308: URL: https://github.com/apache/activemq-artemis/pull/3308#issuecomment-719428788 > Setting poolPreparedStatements = true has proven to be a little problematic. The random database connection losses have made a comeback. This indeed was again an issue with our database. It seems – quite obviously – that open prepared statements keep the connection active and prevent them to appear on idle connection eviction runs. The fix was to simply set maximum live time for db connections. Now everything works like a charm :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 506589) Time Spent: 2h (was: 1h 50m) > commons-dbcp2 performance issue with Derby Embedded DBMS > > > Key: ARTEMIS-2955 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2955 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Tests >Affects Versions: 2.16.0 >Reporter: Francesco Nigro >Assignee: Francesco Nigro >Priority: Major > Fix For: 2.16.0 > > Attachments: image-2020-10-20-09-08-45-390.png, > image-2020-10-20-09-10-10-644.png, pooling_off.svg, pooling_on.svg, > screenshot-1.png, screenshot-2.png > > Time Spent: 2h > Remaining Estimate: 0h > > The test suite has shown an increase in duration of 30 minutes going from to > 2:30 to 3 hours: it seems related to integration paging tests running on > Embedded Derby with the changes of > [ARTEMIS-2823|https://issues.apache.org/jira/browse/ARTEMIS-2823]. > After some profiling sessions on > org/apache/activemq/artemis/tests/integration/paging/PagingTest.testQueueRemoveAll > it seems that using commons-dbcp2 with Embedded Derby isn't working as > expected: > !image-2020-10-20-09-08-45-390.png! > while, if we switch to > [EmbeddedDataSource|https://db.apache.org/derby/docs/10.13/publishedapi/org/apache/derby/jdbc/EmbeddedDataSource.html] > we have > !image-2020-10-20-09-10-10-644.png! > By not using commons-dbcp2 we get roughtly a ~10X improvement in performance: > it seems that commons-dbcp2 is forcing to setup from the ground the prepared > statement each time, while EmbeddedDataSource nope. Specifically it seems > related to Derby GenericActivationHolder. > I suggest to disable commons-dbcp2 for Derby and investigate if it could > happen in a real broker too. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
