Re: [VOTE] Release Apache Qpid Proton-J 0.20.0
+1 * Ran Apache RAT check * Verified checksums and signatures on artefacts * Manually retested SCRAM-SHA SASL authentication using staged Proton-J artefacts with Qpid JMS Client (master) against the Java Broker (master). I ran this test twice, firstly with the Broker configured to use SaslOutcome to send the final challenge (new behaviour) and again with it switched to the old behaviour. * Tested staged Proton-J artefacts with Qpid JMS Client 0.23.0 against Qpid Broker for Java (master) using the Joram test suite All tests passed. On 2 August 2017 at 17:41, Timothy Bish wrote: > On 08/02/2017 10:33 AM, Robbie Gemmell wrote: >> >> Hi folks, >> >> I have put together a first spin for a Qpid Proton-J 0.20.0 release, >> please test it and vote accordingly. >> >> The source and binary archives can be grabbed from: >> https://dist.apache.org/repos/dist/dev/qpid/proton-j/0.20.0-rc1/ >> >> (Note: the .sha checksum files are generated using SHA512) >> >> The maven artifacts are staged for now at: >> https://repository.apache.org/content/repositories/orgapacheqpid- >> >> The JIRAs currently assigned are: >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313720&version=12341280 >> >> Regards, >> Robbie >> >> P.S. If you want to test things out using maven with your own build >> you can temporarily add this to your poms to access the staging repo: >> >> >> >>staging >> >> https://repository.apache.org/content/repositories/orgapacheqpid- >> >> >> >> The dependency for proton-j would then be: >> >> >> org.apache.qpid >> proton-j >> 0.20.0 >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org >> For additional commands, e-mail: users-h...@qpid.apache.org >> >> > +1 > > * Reviewed bin and src archives for proper license and notice files > * Validated signatures and checksums > * Built from source and ran the tests. > * Built Qpid JMS using staged binary and ran the tests > * Built Artemis using staged binary and ran test broker AMQP tests. > > > > -- > Tim Bish > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: [Proton 0.16.0] [Dispatch Router 0.7.0] [Java Broker 6.0.4] [JMS 0.11.1] JMS sender's connection is being closed by the dispatch router
Hi Adel, It seems that your error might be coming from this piece of proton code - https://github.com/apache/qpid-proton/blob/master/proton-c/ src/core/transport.c#L2538 This means that the socket was closed while in the middle of AMQP communication. Thanks. On Wed, Aug 2, 2017 at 2:57 PM, Ganesh Murthy wrote: > Hi Adel, > Looking at the trace, there seems to be a framing error (ERROR > amqp:connection:framing-error connection aborted) but it is not immediately > clear what caused it. Sometimes, some other problem might manifest as a > framing error, so I am not really sure. Are you able to replicate this in a > non-Solaris environment? > > Can you please share your qpid-jms producer source code? > > Thanks. > > On Wed, Aug 2, 2017 at 12:42 PM, Adel Boutros > wrote: > >> Hello, >> >> >> As stated in a separate email, we are trying to compile Proton/Dispatch >> router on Solaris using GCC4.9.2. The code compiles fine and all the unit >> tests are green. >> >> >> However, we have noticed a weird failure in our of our feature tests. The >> test is composed of a JMS producer (0.11.1) connected to a dispatch router >> which is backed by 2 Qpid Java Broker (6.0.4). >> >> The sender is sending around 500 messages in asynchronous mode ( >> jms.forceAsyncSend=true). >> >> It seems that while the JMS producer is sending the message, at some >> random stage, the connection is dropped. The error seems to be coming from >> Proton which seems to be used by the Dispatch Router for AMQP connections >> if I am not mistaken. >> >> >> I have attached the logs of the Producer(Logging in debug mode on >> org.apache.qpid) and Dispatch Router (Log level set to Trace + >> PN_TRACE_FRM=1 + PN_TRACE_RAW=1) >> >> >> Could you please provide us hints or help to analyze this problem? >> >> >> Regards, >> >> Adel >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org >> For additional commands, e-mail: users-h...@qpid.apache.org >> > >
Re: Docker image for qpid-cpp broker
Going forward, I think it is important to allow the persistent data from a broker to be stored in a docker volume. As this container spec currently stands it is really only useful for a broker with no persistent queues, as there is no way to keep the queued data if the container is stopped. Andrew On Wed, 2017-07-26 at 11:54 -0400, Irina Boverman wrote: > Correction: Fedora 26 is latest. > > On Wed, Jul 26, 2017 at 11:53 AM, Irina Boverman > > wrote: > > > I have created a docker image and posted it here: > > https://hub.docker.com/r/irinabov/docker-qpid-cpp/ > > > > Source is here: > > https://github.com/irinabov/docker-qpid-cpp > > -- > > Reviews, feedback and comments are welcome. > > This image is based on Fedora 25 and builds proton 0.17.0 and qpid- > > cpp > > 1.36.0 from apache git repo. > > -- > > Regards, Irina. > > - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: [Proton-c C++ 0.16.0] [Solaris Sparc] [GCC 4.9.2] Assuming threads will propage exceptions
Briefly - Can you try master. This should be fixed in the upcoming
0.18.
Andrew
On Wed, 2017-07-26 at 10:51 +, Adel Boutros wrote:
> Hello,
>
>
> I hope you missed my mails about Solaris 😉
>
>
> We are currently testing whether we can compile on Solaris using GCC
> 4.9.2. We were successful in doing so for Solaris x86 but we have hit
> a problem in Sparc.
>
>
> It seems related to certain assumptions made around how threads
> propagates exceptions:
>
>
> The failing test is "examples/cpp/example_test.py: test_ssl_bad_name"
>
> In examples/cpp/ssl.cpp [1], it spawns another thread and wait for it
> to throw an exception.
>
>
> However, if you check the thread documentation[2], it states the
> propagation of exceptions is not guaranteed[3].
>
>
> So I was wondering if the assumption that Proton makes is not a big
> dangerous and should be changed?
>
>
> Simpler Example
>
> -
> #include
> #include
> #include
>
> class example_cert_error : public std::runtime_error
> {
> public:
> explicit example_cert_error(const std::string& s) :
> std::runtime_error(s) {}
> };
>
> int main(int argc, char** argv) {
> try {
> std::thread th([](){throw example_cert_error("my error"); });
> } catch (const example_cert_error& ce) {
> std::cout << "Caught: " << ce.what() << std::endl;
> }
> return 0;
> }
>
>
>
> Solaris X86
>
> ---
> $ g++49 -std=c++11 throw.cpp -o throw
> $ ./throw
> Caught: my error
>
> Solaris Sparc
> -
> $ g++49 -std=c++11 throw.cpp -o throw
> $ ./throw
> terminate called without an active exception
> Abort (core dumped)
>
>
> [1]: https://github.com/apache/qpid-proton/blob/0.16.x/examples/cpp/s
> sl.cpp#L178
>
> [2]: http://en.cppreference.com/w/cpp/thread/thread
>
> [3]: The top-level function may communicate its return value or an
> exception to the caller
>
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-proton C++ on_sendable race condition
On Wed, 2017-08-02 at 16:06 +0100, Daniel Pocock wrote: > ... > > If you can wait a couple of weeks the new C++ IO code should get > > merged > > to master and the 0.28 release made. > > > > This should fix the problem you are having. > > > > If you continue having problems then, or you want help trying the > > new > > code on the branch. Let me know > > Do you mean 0.28 or 0.18? Do you have any idea how soon it will be > released? That was a type - I mean 0.18. We're on the release path now, but there are a number of bugs/issues still to go. If you can try the current master branch, that will give us very useful feedback. > > Ideally I would like to try the Debian package when it becomes > available >From our perspective it would really help for you to test the code before it's released, so that there is the best chance the released code has no bugs that affect you. > but I don't mind building a local package manually from the branch to > test it if necessary. > > Regards, > > Daniel Thanks Andrew - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: [Proton 0.16.0] [Dispatch Router 0.7.0] [Java Broker 6.0.4] [JMS 0.11.1] JMS sender's connection is being closed by the dispatch router
Hi Adel, Looking at the trace, there seems to be a framing error (ERROR amqp:connection:framing-error connection aborted) but it is not immediately clear what caused it. Sometimes, some other problem might manifest as a framing error, so I am not really sure. Are you able to replicate this in a non-Solaris environment? Can you please share your qpid-jms producer source code? Thanks. On Wed, Aug 2, 2017 at 12:42 PM, Adel Boutros wrote: > Hello, > > > As stated in a separate email, we are trying to compile Proton/Dispatch > router on Solaris using GCC4.9.2. The code compiles fine and all the unit > tests are green. > > > However, we have noticed a weird failure in our of our feature tests. The > test is composed of a JMS producer (0.11.1) connected to a dispatch router > which is backed by 2 Qpid Java Broker (6.0.4). > > The sender is sending around 500 messages in asynchronous mode ( > jms.forceAsyncSend=true). > > It seems that while the JMS producer is sending the message, at some > random stage, the connection is dropped. The error seems to be coming from > Proton which seems to be used by the Dispatch Router for AMQP connections > if I am not mistaken. > > > I have attached the logs of the Producer(Logging in debug mode on > org.apache.qpid) and Dispatch Router (Log level set to Trace + > PN_TRACE_FRM=1 + PN_TRACE_RAW=1) > > > Could you please provide us hints or help to analyze this problem? > > > Regards, > > Adel > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org >
Re: QPID-7867 [Java Broker] Authentication using self-signed expired certificates
ad a) This would solve the issue when the "peers only" truststore is used. The point that without the "peers only" store you can easily circumvent this is valid, but I don't think that this new feature would make the situation any worse. Perhaps the code it self can be included directly in the PeerTrustManager (whatever the actual class name is) to minimise the possible impact on other paths in the code. ad b.1) This sounds a bit strange to me. A bit unpredictable. ad b.2) This should do the trick and from the user perspective seems to provide cleaner solution than b.2. ad c) Alerting is of course nice. But it doesn't really solve the problem completely. On Wed, Aug 2, 2017 at 6:01 PM, Rob Godfrey wrote: > On 2 August 2017 at 17:43, Lorenz Quack wrote: > > > Hi all, > > > > tl;dr > > = > > I think overall if it would come to a vote right now I would vote like > > this: > > a) -1 > > b.1) -1 > > b.2) +0 > > c) +1 > > > > > I think I'd vote for implementing option b.2), or option a) but only for > "peers only" truststores (since if you have enabled peers-only you are > explicitly saying that your certificate chain will only ever have one > entry). > I'm not in favour of option c) since it seems like if you are doing > peers-only and you have checked the validity of the certificate when it was > added, then you would expect the trust store to enforce that validity. > > -- Rob > > > > reasoning follows inline: > > > > On Wed, 2017-08-02 at 15:13 +0100, Keith W wrote: > > > If we were to add a feature to help the use-case, we'd need to decide > > > on the scope. > > > > > > The alternatives I see: > > > > > > (a) validate the expiration of self-signed certificates used for > > > authentication purposes only > > > > > > > I don't like this. It creates a false sense of security. > > My understanding is that this option means that we would validate > > incoming self-signed peer certificates explicitly. > > However, as a malicious attacker I think this would be easily > > circumvented. > > Imagine the following scenario: Eve has a self-signed certificate "A" > > which is in the broker's truststore. Unfortunately, "A" has expired > > and the broker (thanks to this new feature) refuses the connection. > > Eve now creates a new valid certificate "B" signed by "A". When the > > broker receives "B" it will check that it is valid (which it is) and > > that its certificate chain is trusted. It will immediately find the > > trust anchor (the expired certificate "A") in its trust store and > > accept the connection. > > So I am -1 on this. > > > > > (b) broaden the feature. Disallow all expired trust anchors.This > > > which would include (a) but would also catch cases where, say a, a > > > private CA is used carelessly: the root-CA certificate is allowed to > > > expire before certificates issued from it. > > > > > > > I am okay with this. It is not required by RFC5280 but it is > > permitted. It would provide some additional security for people with > > poorly maintained truststores (e.g., old JVM). > > I am +0 on this. > > > > > (c) do nothing more. QPID-7869 has already added altering for out of > > > date certificates within the trust stores. perhaps this is enough. > > > > > > > /s/altering/alerting/ > > IMHO, this is acceptable. > > I am +0 on this. > > > > > I think any new feature would be optional and turned off by default. > > > This is for the sake of backward compatibility. > > > > > > Turning to implementation: > > > > > > For (a) we would simply change the > > > X509TrustManagers#checkClientTrusted to call > > > X509Certificate#checkValidity() on all certificates in the peer's > > > chain. The CertificateExpiredException would halt the connection. > > > > > > For (b), I see two implementation options: > > > > > > 1) Exclude expired certificates from the truststore. Currently the > > > TrustManagers are generated once and reused by the Ports.This would > > > need to change: either TrustManagers could be generated on-the-fly for > > > each connection (not sure about the cost), or the TrustManagers could > > > be periodically refreshed (there would be a race, of course). > > > > > > > I don't like the race condition. I think if we say we don't allow > > expired certificates this should be exact. I also don't like that > > this would result in "certificate_unknown" exceptions but as it turns > > out we don't seem to have an alternative to this execption :( > > > > > 2) Have X509TrustManagers#checkClientTrusted check the TrustAnchor's > > > validity. Unfortunately, the JCA doesn't expose the full > > > certification path to the application until after the TLS handshake is > > > completed. To have X509TrustManagers#checkClientTrusted check the > > > TrustAnchor's validity, it is necessary to recompute the certification > > > path. I have checked this approach is feasible (proof of concept > > > patch attached to QPID-7868). > > > > > > > Sounds like it could work (I have not looked at the
Re: [VOTE] Release Apache Qpid Proton-J 0.20.0
On 08/02/2017 10:33 AM, Robbie Gemmell wrote: Hi folks, I have put together a first spin for a Qpid Proton-J 0.20.0 release, please test it and vote accordingly. The source and binary archives can be grabbed from: https://dist.apache.org/repos/dist/dev/qpid/proton-j/0.20.0-rc1/ (Note: the .sha checksum files are generated using SHA512) The maven artifacts are staged for now at: https://repository.apache.org/content/repositories/orgapacheqpid- The JIRAs currently assigned are: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313720&version=12341280 Regards, Robbie P.S. If you want to test things out using maven with your own build you can temporarily add this to your poms to access the staging repo: staging https://repository.apache.org/content/repositories/orgapacheqpid- The dependency for proton-j would then be: org.apache.qpid proton-j 0.20.0 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org +1 * Reviewed bin and src archives for proper license and notice files * Validated signatures and checksums * Built from source and ran the tests. * Built Qpid JMS using staged binary and ran the tests * Built Artemis using staged binary and ran test broker AMQP tests. -- Tim Bish - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: [VOTE] Release Apache Qpid Proton-J 0.20.0
+1 (non binding) verified ActiveMQ Artemis use of the new setInitialRemoteMaxFrameSize using the staging repo On Wed, 2 Aug 2017 at 16:30 Robbie Gemmell wrote: > On 2 August 2017 at 15:33, Robbie Gemmell > wrote: > > Hi folks, > > > > I have put together a first spin for a Qpid Proton-J 0.20.0 release, > > please test it and vote accordingly. > > > > The source and binary archives can be grabbed from: > > https://dist.apache.org/repos/dist/dev/qpid/proton-j/0.20.0-rc1/ > > > > (Note: the .sha checksum files are generated using SHA512) > > > > The maven artifacts are staged for now at: > > https://repository.apache.org/content/repositories/orgapacheqpid- > > > > The JIRAs currently assigned are: > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313720&version=12341280 > > > > Regards, > > Robbie > > > > P.S. If you want to test things out using maven with your own build > > you can temporarily add this to your poms to access the staging repo: > > > > > > > > staging > > > https://repository.apache.org/content/repositories/orgapacheqpid- > > > > > > > > > The dependency for proton-j would then be: > > > > > > org.apache.qpid > > proton-j > > 0.20.0 > > > > > Adding my +1. > > I checked things over as follows: > - Verified the signatures and checksum files. > - Checked the LICENCE and NOTICE files in the release archives. > - Ran the source build and tests. > - Used mvn apache-rat:check to verify the source release licence headers. > - Checked the bin convenience archive contained the expected bits. > - Used the staging repo to run the Qpid JMS client master build and tests. > - Used the staging repo to run the ActiveMQ 5 master build and AMQP tests. > - Ran the JMS client HelloWorld example against Qpid Broker-J 6.1.4 and >and ActiveMQ 5 master brokers. > > Robbie > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > >
Re: QPID-7867 [Java Broker] Authentication using self-signed expired certificates
On 2 August 2017 at 17:43, Lorenz Quack wrote: > Hi all, > > tl;dr > = > I think overall if it would come to a vote right now I would vote like > this: > a) -1 > b.1) -1 > b.2) +0 > c) +1 > > I think I'd vote for implementing option b.2), or option a) but only for "peers only" truststores (since if you have enabled peers-only you are explicitly saying that your certificate chain will only ever have one entry). I'm not in favour of option c) since it seems like if you are doing peers-only and you have checked the validity of the certificate when it was added, then you would expect the trust store to enforce that validity. -- Rob > reasoning follows inline: > > On Wed, 2017-08-02 at 15:13 +0100, Keith W wrote: > > If we were to add a feature to help the use-case, we'd need to decide > > on the scope. > > > > The alternatives I see: > > > > (a) validate the expiration of self-signed certificates used for > > authentication purposes only > > > > I don't like this. It creates a false sense of security. > My understanding is that this option means that we would validate > incoming self-signed peer certificates explicitly. > However, as a malicious attacker I think this would be easily > circumvented. > Imagine the following scenario: Eve has a self-signed certificate "A" > which is in the broker's truststore. Unfortunately, "A" has expired > and the broker (thanks to this new feature) refuses the connection. > Eve now creates a new valid certificate "B" signed by "A". When the > broker receives "B" it will check that it is valid (which it is) and > that its certificate chain is trusted. It will immediately find the > trust anchor (the expired certificate "A") in its trust store and > accept the connection. > So I am -1 on this. > > > (b) broaden the feature. Disallow all expired trust anchors.This > > which would include (a) but would also catch cases where, say a, a > > private CA is used carelessly: the root-CA certificate is allowed to > > expire before certificates issued from it. > > > > I am okay with this. It is not required by RFC5280 but it is > permitted. It would provide some additional security for people with > poorly maintained truststores (e.g., old JVM). > I am +0 on this. > > > (c) do nothing more. QPID-7869 has already added altering for out of > > date certificates within the trust stores. perhaps this is enough. > > > > /s/altering/alerting/ > IMHO, this is acceptable. > I am +0 on this. > > > I think any new feature would be optional and turned off by default. > > This is for the sake of backward compatibility. > > > > Turning to implementation: > > > > For (a) we would simply change the > > X509TrustManagers#checkClientTrusted to call > > X509Certificate#checkValidity() on all certificates in the peer's > > chain. The CertificateExpiredException would halt the connection. > > > > For (b), I see two implementation options: > > > > 1) Exclude expired certificates from the truststore. Currently the > > TrustManagers are generated once and reused by the Ports.This would > > need to change: either TrustManagers could be generated on-the-fly for > > each connection (not sure about the cost), or the TrustManagers could > > be periodically refreshed (there would be a race, of course). > > > > I don't like the race condition. I think if we say we don't allow > expired certificates this should be exact. I also don't like that > this would result in "certificate_unknown" exceptions but as it turns > out we don't seem to have an alternative to this execption :( > > > 2) Have X509TrustManagers#checkClientTrusted check the TrustAnchor's > > validity. Unfortunately, the JCA doesn't expose the full > > certification path to the application until after the TLS handshake is > > completed. To have X509TrustManagers#checkClientTrusted check the > > TrustAnchor's validity, it is necessary to recompute the certification > > path. I have checked this approach is feasible (proof of concept > > patch attached to QPID-7868). > > > > Sounds like it could work (I have not looked at the PoC which is on > QPID-7869 and not QPID-7868). > The down side of this would be more (non-trivial) code which can contain > security relevant bugs and needs to be understood and maintained. > > Kind regards, > Lorenz > > > I want to mention what the client would see,. With the CPP Broker/NSS > > a TLS alert "certificate_expired" goes over the wire. Unfortunately, > > Oracle's coding of the JCA means that if a > > X509TrustManager#checkClientTrusted throws an CertificateException or > > sub-class, the TLS alert that goes over the wire is always > > "certificate_unknown" regardless of underlying cause. The code in > > question is sun.security.ssl.ServerHandshaker#clientCertificate(). > > The class is final and the whole piece seems to be unpluggable.I > > don't see a reasonable way around this. > > > > I am in favour of feature (b) as this feels more generally useful. I > > prefer implementation option
C++ management library 0.2.0 for C++ broker released
Hi all, For anyone who might find this useful, I've just released version 0.2.0 of our open C++ broker management library which now includes support for managing inter-broker routes (mirroring qpid-route functionality). Source: https://github.com/fourceu/fourc-qpid-manager/releases/tag/v0.2.0 Background: http://qpid.2158936.n2.nabble.com/C-broker-management-from-a-C-library-td7653957.html Regards -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Dispatch Router load balancing config questions
Thank you all very much. i will upgrade to dispatch router 0.8.0 and forget about XA transactions. i was thinking that all of these brokers out there support XA transactions and i am now realizing that they support XA on protocols that are not AMQP 1.0. in the past few days i have also studied more about XA and realize that for all of my clients they do not need transactions or that the local transactions will be just great. On Wed, Aug 2, 2017 at 4:43 AM Robbie Gemmell wrote: > On 1 August 2017 at 22:20, Dan Langford wrote: > > ( github gist with all the config and data from the Original Post > > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 ) > > > > YES i was under pretty light load. a couple hundred or thousand messages > at > > a time were all getting funneled into the local broker. You are right > once > > i got 7 or 8 simultaneous connections all pushing in a couple million > > message i started to see the load overflow to the other router and > broker. > > about 10% of messages in my setup were overflowing. thank you for your > > patience. I think this config is going to work great. i do think i just > > need a small adjustment to how i think about load balancing. when the > cost > > was the same to each broker then i would get approx 50/50 split (that was > > when each router connected to each broker). with this setup where the > local > > cost is less its more like "balancing out once under load" and not the > > traditional "balance evenly to avoid load". but now that i understand > this > > its fine and i know what behavior to expect. > > > > so I do have some producers and consumers that need to use session > > transactions. we have seen those work fine over amqp1.0 when connected > to a > > QPID Broker or Artemis broker. but with the config you see here > connecting > > to a qpid dispatch router (directly or through our VIP) I cannot create a > > session with "transacted=true". I get a NullPointerException > > > > javax.jms.JMSException: java.lang.NullPointerException > > at > > > org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86) > > at > > > org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108) > > at > org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609) > > at > > > org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125) > > at org.apache.qpid.jms.JmsSession.(JmsSession.java:143) > > at > org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299) > > at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149) > > ... > > Caused by: java.io.IOException: java.lang.NullPointerException > > at > > > org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45) > > at > > > org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117) > > at > > > org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928) > > at > > > org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93) > > at > > > org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > at > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.lang.NullPointerException > > at > > > org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93) > > ... 10 more > > > > I've raised https://issues.apache.org/jira/browse/QPIDJMS-307 to > investigate this and improve the handling. Though it did fail to > create the session, as it should have given Gordon's earlier > explanation, it didn't do it very well so there is definitely a client > bug there. Possibly also a router one if my hunch of what happened is > accurate. > > > are transactions expected to work with qpid dispatch router? on that note > > do you think i could also get XA transactions working? would i need to > use > > a link route? i dont know for sure but it sounds like a link route is a > > little more low level that an autolink. > > > > thank you > > > > On Tue, Aug 1, 2017 at 6:47 AM Ted Ross wrote: > > > >> Dan, > >> > >> There's one issue with your configuration which doesn't affect the load > >> balancing but will cause problems with receiving messages from the > >> brokers. In the address.prefix, you use "foo.#". This is
Re: QPID-7867 [Java Broker] Authentication using self-signed expired certificates
Hi all, tl;dr = I think overall if it would come to a vote right now I would vote like this: a) -1 b.1) -1 b.2) +0 c) +1 reasoning follows inline: On Wed, 2017-08-02 at 15:13 +0100, Keith W wrote: > If we were to add a feature to help the use-case, we'd need to decide > on the scope. > > The alternatives I see: > > (a) validate the expiration of self-signed certificates used for > authentication purposes only > I don't like this. It creates a false sense of security. My understanding is that this option means that we would validate incoming self-signed peer certificates explicitly. However, as a malicious attacker I think this would be easily circumvented. Imagine the following scenario: Eve has a self-signed certificate "A" which is in the broker's truststore. Unfortunately, "A" has expired and the broker (thanks to this new feature) refuses the connection. Eve now creates a new valid certificate "B" signed by "A". When the broker receives "B" it will check that it is valid (which it is) and that its certificate chain is trusted. It will immediately find the trust anchor (the expired certificate "A") in its trust store and accept the connection. So I am -1 on this. > (b) broaden the feature. Disallow all expired trust anchors.This > which would include (a) but would also catch cases where, say a, a > private CA is used carelessly: the root-CA certificate is allowed to > expire before certificates issued from it. > I am okay with this. It is not required by RFC5280 but it is permitted. It would provide some additional security for people with poorly maintained truststores (e.g., old JVM). I am +0 on this. > (c) do nothing more. QPID-7869 has already added altering for out of > date certificates within the trust stores. perhaps this is enough. > /s/altering/alerting/ IMHO, this is acceptable. I am +0 on this. > I think any new feature would be optional and turned off by default. > This is for the sake of backward compatibility. > > Turning to implementation: > > For (a) we would simply change the > X509TrustManagers#checkClientTrusted to call > X509Certificate#checkValidity() on all certificates in the peer's > chain. The CertificateExpiredException would halt the connection. > > For (b), I see two implementation options: > > 1) Exclude expired certificates from the truststore. Currently the > TrustManagers are generated once and reused by the Ports.This would > need to change: either TrustManagers could be generated on-the-fly for > each connection (not sure about the cost), or the TrustManagers could > be periodically refreshed (there would be a race, of course). > I don't like the race condition. I think if we say we don't allow expired certificates this should be exact. I also don't like that this would result in "certificate_unknown" exceptions but as it turns out we don't seem to have an alternative to this execption :( > 2) Have X509TrustManagers#checkClientTrusted check the TrustAnchor's > validity. Unfortunately, the JCA doesn't expose the full > certification path to the application until after the TLS handshake is > completed. To have X509TrustManagers#checkClientTrusted check the > TrustAnchor's validity, it is necessary to recompute the certification > path. I have checked this approach is feasible (proof of concept > patch attached to QPID-7868). > Sounds like it could work (I have not looked at the PoC which is on QPID-7869 and not QPID-7868). The down side of this would be more (non-trivial) code which can contain security relevant bugs and needs to be understood and maintained. Kind regards, Lorenz > I want to mention what the client would see,. With the CPP Broker/NSS > a TLS alert "certificate_expired" goes over the wire. Unfortunately, > Oracle's coding of the JCA means that if a > X509TrustManager#checkClientTrusted throws an CertificateException or > sub-class, the TLS alert that goes over the wire is always > "certificate_unknown" regardless of underlying cause. The code in > question is sun.security.ssl.ServerHandshaker#clientCertificate(). > The class is final and the whole piece seems to be unpluggable.I > don't see a reasonable way around this. > > I am in favour of feature (b) as this feels more generally useful. I > prefer implementation option 2). > > Thoughts? Alternatives? > Keith. > > > On 2 August 2017 at 11:50, Keith W wrote: > > > > Hello > > > > Martin Krasa raised JIRA QPID-7867 [1] on 21st July. As the JIRA > > possibly eluded to a potential security issue, the initial discussion > > was held in private on the Qpid private / Apache security lists. We > > have now reached a point where there is a agreement that there is no > > security issue as such. We might however added a feature to the Java > > Broker to benefit this use-case. > > > > I don't wish to cross-post the private conversation verbatim. Instead > > I will try and summerise the issue and the background as I understand > > it. > > > >
Re: QPID-7867 [Java Broker] Authentication using self-signed expired certificates
Correcting two typos. On 2 August 2017 at 15:13, Keith W wrote: > If we were to add a feature to help the use-case, we'd need to decide > on the scope. > > The alternatives I see: > > (a) validate the expiration of self-signed certificates used for > authentication purposes only > > (b) broaden the feature. Disallow all expired trust anchors.This > which would include (a) but would also catch cases where, say a, a > private CA is used carelessly: the root-CA certificate is allowed to > expire before certificates issued from it. > > (c) do nothing more. QPID-7869 has already added altering for out of > date certificates within the trust stores. perhaps this is enough. typo: added alerting > > I think any new feature would be optional and turned off by default. > This is for the sake of backward compatibility. > > Turning to implementation: > > For (a) we would simply change the > X509TrustManagers#checkClientTrusted to call > X509Certificate#checkValidity() on all certificates in the peer's > chain. The CertificateExpiredException would halt the connection. > > For (b), I see two implementation options: > > 1) Exclude expired certificates from the truststore. Currently the > TrustManagers are generated once and reused by the Ports.This would > need to change: either TrustManagers could be generated on-the-fly for > each connection (not sure about the cost), or the TrustManagers could > be periodically refreshed (there would be a race, of course). > > 2) Have X509TrustManagers#checkClientTrusted check the TrustAnchor's > validity. Unfortunately, the JCA doesn't expose the full > certification path to the application until after the TLS handshake is > completed. To have X509TrustManagers#checkClientTrusted check the > TrustAnchor's validity, it is necessary to recompute the certification > path. I have checked this approach is feasible (proof of concept > patch attached to QPID-7868). typo: patch attached to QPID-7869 > > I want to mention what the client would see,. With the CPP Broker/NSS > a TLS alert "certificate_expired" goes over the wire. Unfortunately, > Oracle's coding of the JCA means that if a > X509TrustManager#checkClientTrusted throws an CertificateException or > sub-class, the TLS alert that goes over the wire is always > "certificate_unknown" regardless of underlying cause. The code in > question is sun.security.ssl.ServerHandshaker#clientCertificate(). > The class is final and the whole piece seems to be unpluggable.I > don't see a reasonable way around this. > > I am in favour of feature (b) as this feels more generally useful. I > prefer implementation option 2). > > Thoughts? Alternatives? > Keith. > > > On 2 August 2017 at 11:50, Keith W wrote: >> Hello >> >> Martin Krasa raised JIRA QPID-7867 [1] on 21st July. As the JIRA >> possibly eluded to a potential security issue, the initial discussion >> was held in private on the Qpid private / Apache security lists. We >> have now reached a point where there is a agreement that there is no >> security issue as such. We might however added a feature to the Java >> Broker to benefit this use-case. >> >> I don't wish to cross-post the private conversation verbatim. Instead >> I will try and summerise the issue and the background as I understand >> it. >> >> use-case/issue: >> >> The use-case is a development one, It involves a mutually >> authenticated TLS connection for messaging from the Qpid JMS Client to >> the Java Broker. Both client and server were using self signed >> certificates and those certificates had expired The complaint was >> that connection succeeded even though the certificates had expired. >> The user was comparing behaviour with the CPP Broker which disallowed >> such connections. >> >> background: >> >> Both the Java Broker and Qpid JMS Client rely on the inbuilt features >> of Java JCA for the purpose of establishing trust during a TLS >> handshake. Java treats certificates in the truststore store as >> trust anchors. During an TLS handshake, the Java builds a certificate >> chain from the certificate(s) provided by the peer over the wire to a >> trust anchor in the store. If the chain can be formed, the >> certificates that comprise the chain are subject to checks (such as an >> expiration), but these checks are *not* applied to the trust anchor >> itself. This behaviour seems to me to consistent with RFC5280 [4]. >> >> In use-cases involving self-signed certificates, the certificate is >> installed in the peer's truststore. and as such the self signed >> certificate is itself the trust anchor. When the remote side >> connects and presents the self signed cert, this matches the trust >> anchor stored and connection forms without expiration exception. >> >> I hope this is reasonable summary. Please feel free to correct or clarify. >> I plan to post again later today with my ideas for changes we could >> make to the Java Broker to help this use-case. >> >> Keith,. >> >> [
Re: [VOTE] Release Apache Qpid Proton-J 0.20.0
On 2 August 2017 at 15:33, Robbie Gemmell wrote: > Hi folks, > > I have put together a first spin for a Qpid Proton-J 0.20.0 release, > please test it and vote accordingly. > > The source and binary archives can be grabbed from: > https://dist.apache.org/repos/dist/dev/qpid/proton-j/0.20.0-rc1/ > > (Note: the .sha checksum files are generated using SHA512) > > The maven artifacts are staged for now at: > https://repository.apache.org/content/repositories/orgapacheqpid- > > The JIRAs currently assigned are: > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313720&version=12341280 > > Regards, > Robbie > > P.S. If you want to test things out using maven with your own build > you can temporarily add this to your poms to access the staging repo: > > > > staging > > https://repository.apache.org/content/repositories/orgapacheqpid- > > > > The dependency for proton-j would then be: > > > org.apache.qpid > proton-j > 0.20.0 > Adding my +1. I checked things over as follows: - Verified the signatures and checksum files. - Checked the LICENCE and NOTICE files in the release archives. - Ran the source build and tests. - Used mvn apache-rat:check to verify the source release licence headers. - Checked the bin convenience archive contained the expected bits. - Used the staging repo to run the Qpid JMS client master build and tests. - Used the staging repo to run the ActiveMQ 5 master build and AMQP tests. - Ran the JMS client HelloWorld example against Qpid Broker-J 6.1.4 and and ActiveMQ 5 master brokers. Robbie - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-proton C++ on_sendable race condition
On 29/06/17 19:28, Andrew Stitcher wrote: > On Tue, 2017-06-27 at 12:00 +0200, Daniel Pocock wrote: >> reSIProcate has a class QpidProtonThread[1] for sending to a topic. >> >> The class runs the container in a thread, the method >> QpidProtonThread::thread() is the code executed in the thread when it >> starts. >> > I'm assuming this is an application method as I don't recognise that > name. Correct, the method is in my code[1] on Github >> Another thread is calling the sendMessage(..) method. That method >> uses >> event_loop()->inject(...) to have the doSend(...) method called >> safely. >> This appears to work for a while. > Probably accidentally! The cross thread functionality was experimental > (and rather broken) in the current (0.17) release. >> I notice that when the event loop calls my void_function0 class >> operator(), which calls doSend(...), it is not calling it in the >> container thread, it is being called in the thread that invoked >> sendMessage(...). Is that expected behaviour? > That is the expected, but broken, behaviour - I said that released code > was experimental! > >> Eventually, the on_sendable() method is called. I see it is being >> called in the container's thread. That method also calls >> doSend(...). >> At this point, the other thread becomes stuck. > I expect it's a deadlock > . >> I was thinking about simply removing the on_sendable() >> method. However, >> given that I am using the inject method, should I not be having a >> problem like this at all? > If you can wait a couple of weeks the new C++ IO code should get merged > to master and the 0.28 release made. > > This should fix the problem you are having. > > If you continue having problems then, or you want help trying the new > code on the branch. Let me know Do you mean 0.28 or 0.18? Do you have any idea how soon it will be released? Ideally I would like to try the Debian package when it becomes available but I don't mind building a local package manually from the branch to test it if necessary. Regards, Daniel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
visiting DebConf, Montreal next week?
Will anybody from the qpid community be at DebConf? I'll be there from Saturday Regards, Daniel - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
[VOTE] Release Apache Qpid Proton-J 0.20.0
Hi folks, I have put together a first spin for a Qpid Proton-J 0.20.0 release, please test it and vote accordingly. The source and binary archives can be grabbed from: https://dist.apache.org/repos/dist/dev/qpid/proton-j/0.20.0-rc1/ (Note: the .sha checksum files are generated using SHA512) The maven artifacts are staged for now at: https://repository.apache.org/content/repositories/orgapacheqpid- The JIRAs currently assigned are: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313720&version=12341280 Regards, Robbie P.S. If you want to test things out using maven with your own build you can temporarily add this to your poms to access the staging repo: staging https://repository.apache.org/content/repositories/orgapacheqpid- The dependency for proton-j would then be: org.apache.qpid proton-j 0.20.0 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: QPID-7867 [Java Broker] Authentication using self-signed expired certificates
If we were to add a feature to help the use-case, we'd need to decide on the scope. The alternatives I see: (a) validate the expiration of self-signed certificates used for authentication purposes only (b) broaden the feature. Disallow all expired trust anchors.This which would include (a) but would also catch cases where, say a, a private CA is used carelessly: the root-CA certificate is allowed to expire before certificates issued from it. (c) do nothing more. QPID-7869 has already added altering for out of date certificates within the trust stores. perhaps this is enough. I think any new feature would be optional and turned off by default. This is for the sake of backward compatibility. Turning to implementation: For (a) we would simply change the X509TrustManagers#checkClientTrusted to call X509Certificate#checkValidity() on all certificates in the peer's chain. The CertificateExpiredException would halt the connection. For (b), I see two implementation options: 1) Exclude expired certificates from the truststore. Currently the TrustManagers are generated once and reused by the Ports.This would need to change: either TrustManagers could be generated on-the-fly for each connection (not sure about the cost), or the TrustManagers could be periodically refreshed (there would be a race, of course). 2) Have X509TrustManagers#checkClientTrusted check the TrustAnchor's validity. Unfortunately, the JCA doesn't expose the full certification path to the application until after the TLS handshake is completed. To have X509TrustManagers#checkClientTrusted check the TrustAnchor's validity, it is necessary to recompute the certification path. I have checked this approach is feasible (proof of concept patch attached to QPID-7868). I want to mention what the client would see,. With the CPP Broker/NSS a TLS alert "certificate_expired" goes over the wire. Unfortunately, Oracle's coding of the JCA means that if a X509TrustManager#checkClientTrusted throws an CertificateException or sub-class, the TLS alert that goes over the wire is always "certificate_unknown" regardless of underlying cause. The code in question is sun.security.ssl.ServerHandshaker#clientCertificate(). The class is final and the whole piece seems to be unpluggable.I don't see a reasonable way around this. I am in favour of feature (b) as this feels more generally useful. I prefer implementation option 2). Thoughts? Alternatives? Keith. On 2 August 2017 at 11:50, Keith W wrote: > Hello > > Martin Krasa raised JIRA QPID-7867 [1] on 21st July. As the JIRA > possibly eluded to a potential security issue, the initial discussion > was held in private on the Qpid private / Apache security lists. We > have now reached a point where there is a agreement that there is no > security issue as such. We might however added a feature to the Java > Broker to benefit this use-case. > > I don't wish to cross-post the private conversation verbatim. Instead > I will try and summerise the issue and the background as I understand > it. > > use-case/issue: > > The use-case is a development one, It involves a mutually > authenticated TLS connection for messaging from the Qpid JMS Client to > the Java Broker. Both client and server were using self signed > certificates and those certificates had expired The complaint was > that connection succeeded even though the certificates had expired. > The user was comparing behaviour with the CPP Broker which disallowed > such connections. > > background: > > Both the Java Broker and Qpid JMS Client rely on the inbuilt features > of Java JCA for the purpose of establishing trust during a TLS > handshake. Java treats certificates in the truststore store as > trust anchors. During an TLS handshake, the Java builds a certificate > chain from the certificate(s) provided by the peer over the wire to a > trust anchor in the store. If the chain can be formed, the > certificates that comprise the chain are subject to checks (such as an > expiration), but these checks are *not* applied to the trust anchor > itself. This behaviour seems to me to consistent with RFC5280 [4]. > > In use-cases involving self-signed certificates, the certificate is > installed in the peer's truststore. and as such the self signed > certificate is itself the trust anchor. When the remote side > connects and presents the self signed cert, this matches the trust > anchor stored and connection forms without expiration exception. > > I hope this is reasonable summary. Please feel free to correct or clarify. > I plan to post again later today with my ideas for changes we could > make to the Java Broker to help this use-case. > > Keith,. > > [1] https://issues.apache.org/jira/browse/QPID-7867 > [2] > https://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/CryptoSpec.html > [3] > https://docs.oracle.com/javase/8/docs/api/java/security/cert/package-summary.html > [4] http://www.ietf.org/rfc/rfc5280.txt -
TPM based SSL/TLS authentication
Hi, I was wondering if it is possible configure QPID so that when it uses SSL it is able to utilize the TPM for public key decryption. I found this interesting blog post where someone was able to use openSSL to use the TPM to establish a connection between a simple host and client: https://blog.habets.se/2012/02/TPM-backed-SSL.html I was hoping to do something similar with the SSL authentication in QPID. If it's not currently possible, are there any future plans for this? Thanks, Andy
QPID-7867 [Java Broker] Authentication using self-signed expired certificates
Hello Martin Krasa raised JIRA QPID-7867 [1] on 21st July. As the JIRA possibly eluded to a potential security issue, the initial discussion was held in private on the Qpid private / Apache security lists. We have now reached a point where there is a agreement that there is no security issue as such. We might however added a feature to the Java Broker to benefit this use-case. I don't wish to cross-post the private conversation verbatim. Instead I will try and summerise the issue and the background as I understand it. use-case/issue: The use-case is a development one, It involves a mutually authenticated TLS connection for messaging from the Qpid JMS Client to the Java Broker. Both client and server were using self signed certificates and those certificates had expired The complaint was that connection succeeded even though the certificates had expired. The user was comparing behaviour with the CPP Broker which disallowed such connections. background: Both the Java Broker and Qpid JMS Client rely on the inbuilt features of Java JCA for the purpose of establishing trust during a TLS handshake. Java treats certificates in the truststore store as trust anchors. During an TLS handshake, the Java builds a certificate chain from the certificate(s) provided by the peer over the wire to a trust anchor in the store. If the chain can be formed, the certificates that comprise the chain are subject to checks (such as an expiration), but these checks are *not* applied to the trust anchor itself. This behaviour seems to me to consistent with RFC5280 [4]. In use-cases involving self-signed certificates, the certificate is installed in the peer's truststore. and as such the self signed certificate is itself the trust anchor. When the remote side connects and presents the self signed cert, this matches the trust anchor stored and connection forms without expiration exception. I hope this is reasonable summary. Please feel free to correct or clarify. I plan to post again later today with my ideas for changes we could make to the Java Broker to help this use-case. Keith,. [1] https://issues.apache.org/jira/browse/QPID-7867 [2] https://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/CryptoSpec.html [3] https://docs.oracle.com/javase/8/docs/api/java/security/cert/package-summary.html [4] http://www.ietf.org/rfc/rfc5280.txt - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Dispatch Router questions
On 27 July 2017 at 02:56, Dan Langford wrote: > Thanks For the reply. (I realize my diagrams had a small typo. Too many Ra > nodes whoops some should be Rb) I do have queues with that name on both > brokers. I'll make sure it's set to balanced. The "sharded" behavior you > went on to described is exactly our intent. We want high availability of > the service not necessarily high availability of the messages. We are ok > communicating to our users that a node may go down and those messages may > be unavailable until we get it back up. > > That being said we want to feel confident that if a node going down is a > fatal data corruption issue like a corrupt disk we definitely don't want to > lose those messages which have been persisted. We could configure Artemis > to store messages in a backed up DB or we could snapshot our VM regularly > but the approach we are investigating first is using an Artemis cluster > with colocation replication. That way in theory if we completely destroy a > disk we can via Artemis promote the backup to be primary and they will be > served again. > > (We tried using Artemis load balancing but it cannot load balance AMQP at > the moment) > > Another question crept up about transactions: using the Qpid-JMS-client we > cannot interact with our cluster here if we have session transacted=true. > Should we be able to do transacted sessions via an autoLink? Or do we have > to use a linkRoute? I may not know the difference > > As we work further on this we will explore XATransactions. I know we will > need to use another lib like the old Qpid client for Java but should we > expect XA transactions to work through dispatch Router? > Further to Gordon's reply on the other thread, note that the router only supports AMQP 1.0 so you can't use the older JMS client with it as that only supports the AMQP 0-x protocols. > Thanks so much for your time and thought > On Wed, Jul 26, 2017 at 1:10 PM Ted Ross wrote: > >> On Fri, Jul 21, 2017 at 7:12 PM, Dan Langford >> wrote: >> >> > On Thu, Jul 20, 2017 at 9:58 AM Ted Ross wrote: >> > >> > > On Wed, Jul 19, 2017 at 7:36 PM, Dan Langford >> > > wrote: >> > > >> > > > > - Can I configure QDR to autoLink in and out ANY/ALL addresses? >> > > > No. There is no way currently for QDR to know what queues are >> present >> > on >> > > > its connected brokers. It would not be difficult to write a program >> to >> > > > synchronize autolinks to existing queues. >> > > >> > >> > You are right it wouldn't be that difficult. Also with artemis I can turn >> > on autocreation of queues and then then use QDR as the spot to manage >> what >> > queues can exist. Not bad. What about synchronizing autoLink config >> across >> > routers in a QDR network? are messages to the $management queue broadcast >> > throughout the cluster? i could always resend the necessary messages >> > through the _topo address namespace to get it to the other routers. >> > >> > >> > > > > - Artemis doesn't support vhosts. Can I configure connections to >> > > vhost:Foo >> > > > > address:bar actually be address:Foo.bar when the message goes back >> to >> > > the >> > > > > broker? >> > > > Yes. There is a multi-tenancy feature for listeners that does >> exactly >> > > what >> > > > you are asking for. If you add the attribute "multiTenant: yes" to >> the >> > > > configuration of a listener in the qdrouterd.conf file, clients >> > connected >> > > > via that listener will have their addresses annotated as vhost/addr >> in >> > > the >> > > > router. >> > > >> > >> > ok this is going to be perfect. i am starting to feel more comfortable >> > with everything in this config file >> > >> > > > - Can I configure QDR to pass auth through to the broker and let the >> > > broker >> > > > > decide is the user is authenticated and authorized? Inversely can I >> > > > > configure QDR to be the only determinate of auth? >> > > > Presently, QDR expects to be the sole determiner of authentic >> identity. >> > >> > > There is an open request to add a SASL proxy that might be used to >> allow >> > > > the broker to do authentication on behalf of the router, but that >> > hasn't >> > > > made it into master yet. >> > > >> > >> > this is one part that has me a little stuck. QDR is the sole determiner >> of >> > auth identity. but QDR delegates to a cyrus sasl config right? and cyrus >> > sasl has some local DB options or sql or ldap or it can delegate to >> > kerberos or pam and i am just starting to feel a little lost in all my >> auth >> > option because its been a long time since i have been through all that. i >> > will figure it out well enough. i kind of wish there was a way i could >> send >> > a message in through $management to add a new user/pass to the sasldb but >> > ill figure something out. >> > >> > also, in regards to auth where is it that i specify what users have >> access >> > to what addresses? it looks like that might be in the config in >> > vhost>groups but then i see a policy
Re: Dispatch Router load balancing config questions
On 1 August 2017 at 22:20, Dan Langford wrote: > ( github gist with all the config and data from the Original Post > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 ) > > YES i was under pretty light load. a couple hundred or thousand messages at > a time were all getting funneled into the local broker. You are right once > i got 7 or 8 simultaneous connections all pushing in a couple million > message i started to see the load overflow to the other router and broker. > about 10% of messages in my setup were overflowing. thank you for your > patience. I think this config is going to work great. i do think i just > need a small adjustment to how i think about load balancing. when the cost > was the same to each broker then i would get approx 50/50 split (that was > when each router connected to each broker). with this setup where the local > cost is less its more like "balancing out once under load" and not the > traditional "balance evenly to avoid load". but now that i understand this > its fine and i know what behavior to expect. > > so I do have some producers and consumers that need to use session > transactions. we have seen those work fine over amqp1.0 when connected to a > QPID Broker or Artemis broker. but with the config you see here connecting > to a qpid dispatch router (directly or through our VIP) I cannot create a > session with "transacted=true". I get a NullPointerException > > javax.jms.JMSException: java.lang.NullPointerException > at > org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86) > at > org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108) > at org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609) > at > org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125) > at org.apache.qpid.jms.JmsSession.(JmsSession.java:143) > at org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299) > at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149) > ... > Caused by: java.io.IOException: java.lang.NullPointerException > at > org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45) > at > org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117) > at > org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928) > at > org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93) > at > org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.NullPointerException > at > org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93) > ... 10 more > I've raised https://issues.apache.org/jira/browse/QPIDJMS-307 to investigate this and improve the handling. Though it did fail to create the session, as it should have given Gordon's earlier explanation, it didn't do it very well so there is definitely a client bug there. Possibly also a router one if my hunch of what happened is accurate. > are transactions expected to work with qpid dispatch router? on that note > do you think i could also get XA transactions working? would i need to use > a link route? i dont know for sure but it sounds like a link route is a > little more low level that an autolink. > > thank you > > On Tue, Aug 1, 2017 at 6:47 AM Ted Ross wrote: > >> Dan, >> >> There's one issue with your configuration which doesn't affect the load >> balancing but will cause problems with receiving messages from the >> brokers. In the address.prefix, you use "foo.#". This is a pure prefix >> and it should simply be "foo". The wildcards are coming in the next >> release but are not implemented in the code you are using. >> >> Regarding your actual question. I assume that you are testing this >> configuration under light load (i.e. sending one message at a time). >> >> The way that the balancing works is that it will route to the consumer >> (broker) with the fewest outstanding deliveries + inter-router cost. This >> means that it will favor the local broker over the remote one if there are >> no in-flight deliveries. The default (and minimum) cost for an >> inter-router connection is 1. You can set it to a higher value in the >> listener or connector. >> >>
Re: qpid-broker-j-6.1.4 examples will not run
On 2 August 2017 at 07:50, Rob Godfrey wrote: > On 2 August 2017 at 00:15, d...@ucar.edu wrote: > >> Sorry for being so dense, but... >> >> Clearly, I am assuming something about the broker >> distribution that is not true. Specifially, I am assuming >> that the qpid broker distribution is self contained. >> [Judging from the replies, this assumption may be false.] >> >> > Which distribution are you talking about? For 6.1.4 there is a combined > "Qpid for Java 6.1.4" *source* distribution (containing both the Qpid > Broker for Java and Qpid JMS for AMQP 0-9-1/0-10 source code), and a "Qpid > Broker for Java 6.1.4" binary distribution. There is no *Qpid Broker for > Java* distribution which includes examples. > > For historical reasons the source code for the Broker for Java and the JMS > for AMQP 0-x client were in the same repository, and released as a combined > source package. These two components have now been separated, and so the > 6.1.x line is the last line where this will be true. > > >> So, my confusion comes from the above assumption >> plus the fact that the qpid broker distribution comes >> with a subdirectory of examples. >> > > The *broker* distribution does not have an examples directory. The > combined broker and JMS for AMQP 0-x source distribution *does* have > examples. These examples are for the JMS for AMQP 0-x component, and could > be used against any broker or service that offers AMQP 0-9-1 or AMQP 0-10 > (such as the Qpid C++ broker, or (with some caveats) RabbitMQ). > > >> I again assume that if I follow the directions, >> I can execute those examples. without recourse to >> any other code. Instead, those examples fail to work for me. >> >> What am I missing? >> > > An AMQP broker should work with any clients which support the same AMQP > version. Unless there are other reasons to do so (such as the shared > codebase) there is no reason why a broker distribution should include an > AMQP client. > > Now, I agree that if you follow instructions in the source bundle for > running examples, they should work - so we should look into this; however > since these examples are for the AMQP 0-x client, and in another of you > mails you said you needed AMQP 1.0, the fact that this may be broken > doesn't seem like it should actually matter to you. You would seem to want > to get an AMQP 1.0 client (e.g. the separate Qpid JMS client for AMQP 1.0) > and run any examples from the client which uses the AMQP 1.0 protocol. > > -- Rob > As Rob said, it sounds like you are using the combined source release. I just tried the examples from there and didn't find any issue with them, though I am running things on Linux so its possible there are still Windows+Cygwin specific issues you are hitting. I grabbed the source release, built the bits, and changed to the broker modules target dir (where its binary package is built) as follows: wget http://mirror.ox.ac.uk/sites/rsync.apache.org/qpid/java/6.1.4/qpid-java-6.1.4.tar.gz wget http://www.apache.org/dist/qpid/java/6.1.4/qpid-java-6.1.4.tar.gz.sha sha512sum -c qpid-java-6.1.4.tar.gz.sha tar -xzvf qpid-java-6.1.4.tar.gz cd qpid-java-6.1.4/ mvn clean package -DskipTests cd broker/target/ I started the broker like so: tar -xzvf qpid-broker-6.1.4-bin.tar.gz cd qpid-broker/6.1.4/ export QPID_WORK=`pwd` bin/qpid-server In another terminal, I ran the AMQP 0-x JMS examples you have been trying along the lines of the following: cd /qpid-java-6.1.4/client/example/ mvn clean package dependency:copy-dependencies -DincludeScope=runtime -DskipTests java -cp "target/classes/:target/dependency/*" org.apache.qpid.example.Hello I'd expect the second and third group of instructions to be very similar to what you would do with the individual broker and AMQP 0-x JMS client binary distributions. That said, you've said you actually want AMQP 1.0 so you should grab the newer client and follow its example instructions instead. Essentially: start the broker, create the queue [1], then build and run the clients examples with something like: mvn clean package dependency:copy-dependencies -DincludeScope=runtime -DskipTests java -DUSER=guest -DPASSWORD=guest -cp "target/classes/:target/dependency/*" org.apache.qpid.jms.example.HelloWorld [1] This can be done via the brokers web management which defaults to http://localhost:8080, default user detail examples are in etc/passwd, e.g try admin:admin. Double click the 'default' virthuost to open it. Hit Add Queue, enter "queue" as the name, hit Create Queue. Final aside: Rob has added the users@ mailing list to his reply as that is where the thread originally started and where discussion such as this belongs, please keep users@ in respones. Robbie - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Dispatch Router load balancing config questions
On 01/08/17 22:20, Dan Langford wrote:
are transactions expected to work with qpid dispatch router?
The router itself does not support transactions (it does not do any
storage of messages). From 0.8 you should be able to link route (local)
transaction requests to a transaction coordinator such as a broker
(using the pseudo address $coordinator) e.g. by adding something like:
linkRoute {
prefix: $coordinator
dir: in
connection: txbroker
}
However this only really works if you also ensure that all transactional
operations go to the same broker as any control/coordination messages.
on that note
do you think i could also get XA transactions working?
No, at present there is no support for XA over AMQP 1.0 in any broker.
-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
