C++ binding stream encoder/decoder - review requested.
The new commits are on the pull request https://github.com/apache/qpid-proton/pull/35/commits PROTON-865: Stream like Encoder/Decoder and AMQP Value type for C++ binding. See Encoder.h, Decoder.h, Value.h for details. Encoder/Decoder use operator << and >> to encode/decode AMQP values. Value is a variant-like type that can hold an arbitrary AMQP value. Simple types are implemented, complex types coming next.
[jira] [Commented] (PROTON-865) C++ reactor client binding
[ https://issues.apache.org/jira/browse/PROTON-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582675#comment-14582675 ] ASF subversion and git services commented on PROTON-865: Commit aaf0430d7f24b176d69f9306c69243b99e5b9e43 in qpid-proton's branch refs/heads/cjansen-cpp-client from [~aconway] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=aaf0430 ] PROTON-865: Remove duplicate UUID code. There's no need UUID generation in the binding now (container-id is not required.) If it arises later we should use proton's UUID code, not duplicate it in the binding. > C++ reactor client binding > -- > > Key: PROTON-865 > URL: https://issues.apache.org/jira/browse/PROTON-865 > Project: Qpid Proton > Issue Type: New Feature > Environment: C++ >Reporter: Cliff Jansen >Assignee: Cliff Jansen > > This JIRA tracks initial work on a C++ binding to the Proton reactor event > based model with an eye to providing an API very similar to the python > client. As stated on the Proton list, the broad goals are: > to make it easy to write simple programs > natural to build out more complicated ones > very similar API to the Python client > lean and performant > The initial introduction with accompanying HelloWorld can be found at > https://github.com/apache/qpid-proton/pull/18 > Ongoing work is proceeding in my github account in branch cpp-work > https://github.com/cliffjansen/qpid-proton/tree/cpp-work > commit 1453f57ca3f446450ef654505548f1947cb7a0f1 adds listener support, > exceptions and a logging interface. > The initial implementation will provide no thread safety guarantees, but the > bone structure should allow that to be added later with no API change. I > still hold out hope of eventually allowing multiple threads processing > separate connections concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-865) C++ reactor client binding
[ https://issues.apache.org/jira/browse/PROTON-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582674#comment-14582674 ] ASF subversion and git services commented on PROTON-865: Commit 052d466963bc342b300ea28248ecd40fbddbd523 in qpid-proton's branch refs/heads/cjansen-cpp-client from [~aconway] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=052d466 ] PROTON-865: Stream like Encoder/Decoder and AMQP Value type for C++ binding. See Encoder.h, Decoder.h, Value.h for details. Encoder/Decoder use operator << and >> to encode/decode AMQP values. Value is a variant-like type that can hold an arbitrary AMQP value. Simple types are implemented, complex types coming next. > C++ reactor client binding > -- > > Key: PROTON-865 > URL: https://issues.apache.org/jira/browse/PROTON-865 > Project: Qpid Proton > Issue Type: New Feature > Environment: C++ >Reporter: Cliff Jansen >Assignee: Cliff Jansen > > This JIRA tracks initial work on a C++ binding to the Proton reactor event > based model with an eye to providing an API very similar to the python > client. As stated on the Proton list, the broad goals are: > to make it easy to write simple programs > natural to build out more complicated ones > very similar API to the Python client > lean and performant > The initial introduction with accompanying HelloWorld can be found at > https://github.com/apache/qpid-proton/pull/18 > Ongoing work is proceeding in my github account in branch cpp-work > https://github.com/cliffjansen/qpid-proton/tree/cpp-work > commit 1453f57ca3f446450ef654505548f1947cb7a0f1 adds listener support, > exceptions and a logging interface. > The initial implementation will provide no thread safety guarantees, but the > bone structure should allow that to be added later with no API change. I > still hold out hope of eventually allowing multiple threads processing > separate connections concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-911) Implement SASL security layer
Andrew Stitcher created PROTON-911: -- Summary: Implement SASL security layer Key: PROTON-911 URL: https://issues.apache.org/jira/browse/PROTON-911 Project: Qpid Proton Issue Type: Sub-task Reporter: Andrew Stitcher Implement SASL encryption using the Cyrus SASL library. This would allow you to use AMQP over kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-911) Implement SASL security layer
[ https://issues.apache.org/jira/browse/PROTON-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Stitcher reassigned PROTON-911: -- Assignee: Andrew Stitcher > Implement SASL security layer > - > > Key: PROTON-911 > URL: https://issues.apache.org/jira/browse/PROTON-911 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > > Implement SASL encryption using the Cyrus SASL library. > This would allow you to use AMQP over kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: something rotten in the state of... something or other
On 11/06/15 16:08 +0100, Gordon Sim wrote: On 06/11/2015 02:39 PM, Flavio Percoco wrote: On 11/06/15 09:33 -0400, Ken Giusti wrote: Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources. we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run. It's actually something that swig does itself when building the python extension, I don't think there's much we can do unless we move cproton.i somewhere else, which is not a crazy thing to do. The cmake based build generates it under the build tree. The swig command seems to have an -outdir option, maybe we could use? Unfortunately, distutils doesn't have the right abstraction to change the output dir for swig. However, this should fix the current issue: https://reviews.apache.org/r/35369/ I believe we should still find a better place to put those files but we can do that later. Cheers, Flavio I tried moving it under a different package but that ends up in the build step copying the whole package into site-packages instead of just copying the generated `cproton.py` module. That said, distutils has enough hooks to make something like this work, I'll take a look. Cheers, Flavio P.S: I was finally able to replicate this issue. -- @flaper87 Flavio Percoco pgpA13Y7IJzqC.pgp Description: PGP signature
[jira] [Created] (PROTON-910) Proton-J SASL implementation should support new SASL APIs
Andrew Stitcher created PROTON-910: -- Summary: Proton-J SASL implementation should support new SASL APIs Key: PROTON-910 URL: https://issues.apache.org/jira/browse/PROTON-910 Project: Qpid Proton Issue Type: Sub-task Reporter: Andrew Stitcher The Java Proton implementation should be able to support the basic SASL implementation APIs. This would enable removing some of the seemingly arbitrary test skips for SASL tests if we're running in Jython. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-909) Tests for new SASL features and mechanisms
[ https://issues.apache.org/jira/browse/PROTON-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582485#comment-14582485 ] Andrew Stitcher commented on PROTON-909: There are now tests that exercise nearly all of the SASL functionality in either basic or extended SASL implementations. * One area that is hard to test is the basic client support for PLAIN (because there is no basic server support for PLAIN) > Tests for new SASL features and mechanisms > -- > > Key: PROTON-909 > URL: https://issues.apache.org/jira/browse/PROTON-909 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-867) Enable PLAIN SASL mech iff connection is encrypted
[ https://issues.apache.org/jira/browse/PROTON-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Stitcher resolved PROTON-867. Resolution: Fixed Fix Version/s: 0.10 This issue has been resolved with the changes to support PROTON-866 > Enable PLAIN SASL mech iff connection is encrypted > -- > > Key: PROTON-867 > URL: https://issues.apache.org/jira/browse/PROTON-867 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > Fix For: 0.10 > > > Currently PLAIN is never offered by Proton acting as a server because the > SASL layer doesn't know when encryption is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PROTON-867) Enable PLAIN SASL mech iff connection is encrypted
[ https://issues.apache.org/jira/browse/PROTON-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582476#comment-14582476 ] Andrew Stitcher edited comment on PROTON-867 at 6/11/15 8:28 PM: - It would be helpful to the user if there was a log message when PLAIN was ruled out because the connection is not encrypted. However there is no general logging system in Proton (yet) that would be able to support this. was (Author: astitcher): It would be helpful to the user if there was a log message when PLAIN was ruled out because the connection is not encrypted. > Enable PLAIN SASL mech iff connection is encrypted > -- > > Key: PROTON-867 > URL: https://issues.apache.org/jira/browse/PROTON-867 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > > Currently PLAIN is never offered by Proton acting as a server because the > SASL layer doesn't know when encryption is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-867) Enable PLAIN SASL mech iff connection is encrypted
[ https://issues.apache.org/jira/browse/PROTON-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582476#comment-14582476 ] Andrew Stitcher commented on PROTON-867: It would be helpful to the user if there was a log message when PLAIN was ruled out because the connection is not encrypted. > Enable PLAIN SASL mech iff connection is encrypted > -- > > Key: PROTON-867 > URL: https://issues.apache.org/jira/browse/PROTON-867 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > > Currently PLAIN is never offered by Proton acting as a server because the > SASL layer doesn't know when encryption is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-866) Implement SASL external with TLS client authentication
[ https://issues.apache.org/jira/browse/PROTON-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Stitcher resolved PROTON-866. Resolution: Fixed Fix Version/s: 0.10 > Implement SASL external with TLS client authentication > -- > > Key: PROTON-866 > URL: https://issues.apache.org/jira/browse/PROTON-866 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > Fix For: 0.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-334) SASL Implementation for Proton C
[ https://issues.apache.org/jira/browse/PROTON-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582471#comment-14582471 ] ASF subversion and git services commented on PROTON-334: Commit d7df5760f979a2e0503b272638067493fd5b9e7b in qpid-proton's branch refs/heads/master from [~astitcher] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=d7df576 ] PROTON-334: Add capability to detect extended SASL support > SASL Implementation for Proton C > > > Key: PROTON-334 > URL: https://issues.apache.org/jira/browse/PROTON-334 > Project: Qpid Proton > Issue Type: Wish > Components: proton-c >Reporter: Ted Ross >Assignee: Andrew Stitcher > > It would be desirable to have the ability to use a plug-in module for SASL in > Proton. The following implementations could then be developed: > 1) A portable stand-alone plugin that does ANONYMOUS, PLAIN, and EXTERNAL > 2) A Cyrus-Sasl based plugin for Linux > 3) A Windows plugin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-909) Tests for new SASL features and mechanisms
[ https://issues.apache.org/jira/browse/PROTON-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582472#comment-14582472 ] ASF subversion and git services commented on PROTON-909: Commit 990b11e8a3e3b7a63c891b42e22ea3d180278e55 in qpid-proton's branch refs/heads/master from [~astitcher] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=990b11e ] PROTON-909: Tests for Cyrus SASL mechs > Tests for new SASL features and mechanisms > -- > > Key: PROTON-909 > URL: https://issues.apache.org/jira/browse/PROTON-909 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-909) Tests for new SASL features and mechanisms
[ https://issues.apache.org/jira/browse/PROTON-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Stitcher reassigned PROTON-909: -- Assignee: Andrew Stitcher > Tests for new SASL features and mechanisms > -- > > Key: PROTON-909 > URL: https://issues.apache.org/jira/browse/PROTON-909 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-909) Tests for new SASL features and mechanisms
Andrew Stitcher created PROTON-909: -- Summary: Tests for new SASL features and mechanisms Key: PROTON-909 URL: https://issues.apache.org/jira/browse/PROTON-909 Project: Qpid Proton Issue Type: Sub-task Reporter: Andrew Stitcher -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Proton-c Null Messages
I used valgrind and the segfault isn't directly proton's fault, the problem is we are getting back null messages from pn_messenger_get, and proton seg-faults when I call pn_message_get_content_type on that message. -- View this message in context: http://qpid.2158936.n2.nabble.com/Proton-c-Null-Messages-tp7625967p7626329.html Sent from the Apache Qpid Proton mailing list archive at Nabble.com.
[jira] [Commented] (PROTON-866) Implement SASL external with TLS client authentication
[ https://issues.apache.org/jira/browse/PROTON-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582406#comment-14582406 ] ASF subversion and git services commented on PROTON-866: Commit 8108bd760914c362dde575ed700b744f5d415a3d in qpid-proton's branch refs/heads/master from [~astitcher] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=8108bd7 ] PROTON-866: Add support for EXTERNAL mechanism to default SASL impl > Implement SASL external with TLS client authentication > -- > > Key: PROTON-866 > URL: https://issues.apache.org/jira/browse/PROTON-866 > Project: Qpid Proton > Issue Type: Sub-task > Components: proton-c >Reporter: Andrew Stitcher >Assignee: Andrew Stitcher > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Proton-c Null Messages
On 06/09/2015 07:54 PM, Gordon Sim wrote: On 06/09/2015 06:40 PM, logty wrote: When I run the client I get: [0x5351db0]:0 <- @transfer(20) [handle=0, delivery-id=0, delivery-tag=b"", message-format=0, settled=true, more=true] (16363) "\x00Sp\xc0\x07\x05B..." My guess would be that it is the delivery tag being null (or empty, can't tell which) that is the problem. From the spec: "This field MUST be specified for the first transfer of a multi-transfer message and can only be omitted for continuation transfers." [section 2.7.5] So I think that whatever is sending that frame has a bug. Proton-c has a bug too of course, since it shouldn't segfault but should close the connection with a framing-error or similar. Can you get a backtrace for the segfault? (I've been unable to reproduce so far).
[jira] [Commented] (PROTON-908) Use swig as a build dependency when possible
[ https://issues.apache.org/jira/browse/PROTON-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582035#comment-14582035 ] Flavio Percoco commented on PROTON-908: --- review board: https://reviews.apache.org/r/35360/ > Use swig as a build dependency when possible > > > Key: PROTON-908 > URL: https://issues.apache.org/jira/browse/PROTON-908 > Project: Qpid Proton > Issue Type: Bug >Reporter: Flavio Percoco > Attachments: > 0001-PROTON-908-Remove-install-runtime-dependency-on-swig.patch > > > python-qpid-proton depends on swig to generate the C/python wrappers for > proton-c. > It is possible to soften this dependency by making it a build (sdist in > python jargon) dependency instead of a runtime/install dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: something rotten in the state of... something or other
On 06/11/2015 02:39 PM, Flavio Percoco wrote: On 11/06/15 09:33 -0400, Ken Giusti wrote: Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources. we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run. It's actually something that swig does itself when building the python extension, I don't think there's much we can do unless we move cproton.i somewhere else, which is not a crazy thing to do. The cmake based build generates it under the build tree. The swig command seems to have an -outdir option, maybe we could use? I tried moving it under a different package but that ends up in the build step copying the whole package into site-packages instead of just copying the generated `cproton.py` module. That said, distutils has enough hooks to make something like this work, I'll take a look. Cheers, Flavio P.S: I was finally able to replicate this issue.
[jira] [Updated] (PROTON-908) Use swig as a build dependency when possible
[ https://issues.apache.org/jira/browse/PROTON-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Percoco updated PROTON-908: -- Attachment: 0001-PROTON-908-Remove-install-runtime-dependency-on-swig.patch > Use swig as a build dependency when possible > > > Key: PROTON-908 > URL: https://issues.apache.org/jira/browse/PROTON-908 > Project: Qpid Proton > Issue Type: Bug >Reporter: Flavio Percoco > Attachments: > 0001-PROTON-908-Remove-install-runtime-dependency-on-swig.patch > > > python-qpid-proton depends on swig to generate the C/python wrappers for > proton-c. > It is possible to soften this dependency by making it a build (sdist in > python jargon) dependency instead of a runtime/install dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-908) Use swig as a build dependency when possible
Flavio Percoco created PROTON-908: - Summary: Use swig as a build dependency when possible Key: PROTON-908 URL: https://issues.apache.org/jira/browse/PROTON-908 Project: Qpid Proton Issue Type: Bug Reporter: Flavio Percoco python-qpid-proton depends on swig to generate the C/python wrappers for proton-c. It is possible to soften this dependency by making it a build (sdist in python jargon) dependency instead of a runtime/install dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-904) Remove dependency on libuuid
[ https://issues.apache.org/jira/browse/PROTON-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti closed PROTON-904. - Resolution: Fixed Fix Version/s: 0.10 > Remove dependency on libuuid > > > Key: PROTON-904 > URL: https://issues.apache.org/jira/browse/PROTON-904 > Project: Qpid Proton > Issue Type: Bug >Reporter: Flavio Percoco >Assignee: Ken Giusti > Fix For: 0.10 > > > The current proton-c version depends on libuuid for, well, generating uuids. > The unfortunate thing of this dependency is that it's currently just required > by the messenger and it's just being used for building the messengers name. > It's really unfortunate to require this library and headers to be present for > such a small case. > It'd be possible to remove this dependency by adding a built-in uuid4 > generator since uuid4 is based on random bytes generation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-904) Remove dependency on libuuid
[ https://issues.apache.org/jira/browse/PROTON-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti reassigned PROTON-904: - Assignee: Ken Giusti > Remove dependency on libuuid > > > Key: PROTON-904 > URL: https://issues.apache.org/jira/browse/PROTON-904 > Project: Qpid Proton > Issue Type: Bug >Reporter: Flavio Percoco >Assignee: Ken Giusti > Fix For: 0.10 > > > The current proton-c version depends on libuuid for, well, generating uuids. > The unfortunate thing of this dependency is that it's currently just required > by the messenger and it's just being used for building the messengers name. > It's really unfortunate to require this library and headers to be present for > such a small case. > It'd be possible to remove this dependency by adding a built-in uuid4 > generator since uuid4 is based on random bytes generation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-904) Remove dependency on libuuid
[ https://issues.apache.org/jira/browse/PROTON-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582003#comment-14582003 ] ASF subversion and git services commented on PROTON-904: Commit 060b1aa8a0631ce3bdd05bc09374e2d684ccd201 in qpid-proton's branch refs/heads/master from [~flaper87] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=060b1aa ] PROTON-904 Remove the dependency on uuid Instead of relying on libuuid for uuid generation, let proton-c have a built-in uuid4 generator to generate a somewhat random name. > Remove dependency on libuuid > > > Key: PROTON-904 > URL: https://issues.apache.org/jira/browse/PROTON-904 > Project: Qpid Proton > Issue Type: Bug >Reporter: Flavio Percoco > > The current proton-c version depends on libuuid for, well, generating uuids. > The unfortunate thing of this dependency is that it's currently just required > by the messenger and it's just being used for building the messengers name. > It's really unfortunate to require this library and headers to be present for > such a small case. > It'd be possible to remove this dependency by adding a built-in uuid4 > generator since uuid4 is based on random bytes generation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Proton-c Null Messages
On 06/11/2015 01:54 PM, aconway wrote: On Thu, 2015-06-11 at 13:40 +0100, Gordon Sim wrote: If a name field is populated with an empty string, that to me is the same as not supplying a name. An empty string is a legal encoding, but in my view it does not supply a value at all. (It is not like say 0 which may be the default but is clearly a value in its own right). It is exactly like 0, a perfectly legal value that is often abused to mean something special. It is treated as meaning 'empty' i.e. 'there is nothing here'. I don't consider that abuse myself. If a string is a sequence of chars, the empty string is not a sequence of chars, there are no chars specified in sequence, therefore it clearly is 'special'. The distinction between an empty string and null is an artificial one and in my view anything for that relies on that difference to convey something logically significant is poorly designed. "" is a legal string literal in every language I know of. It can be used as a key in a map or hash table. It can be compared with other strings. There is no string operation in any language I know that will throw NotAString if you apply it to "". Those are fair points and I withdraw the point I made about not being a 'value', since that has a specific technical meaning and as you point out the empty string would clearly qualify. However I still believe that logically an empty string and null are two different ways of not supplying any information. I don't consider either as acceptable if an item of data is stated as being required. However, from the practical point of view... This is very practical. Interoperability is about agreeing on a type system. A "type" defines a range of legal values. The AMQP type system includes empty string and 0-length binaries as legal values for those types. We absolutely cannot treat any legal value in an exceptional way unless that is clearly mandated by the spec. The wording in the spec could certainly be more precise on the point. To me it is still clear, but we will not reach agreement on that. In any case there is no harm in handling an empty string as a valid delivery tag (as long as it is unique). I have no objection to that. The only part I personally consider important is that proton-c doesn't crash.
Re: something rotten in the state of... something or other
On 11/06/15 09:33 -0400, Ken Giusti wrote: Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources. we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run. It's actually something that swig does itself when building the python extension, I don't think there's much we can do unless we move cproton.i somewhere else, which is not a crazy thing to do. I tried moving it under a different package but that ends up in the build step copying the whole package into site-packages instead of just copying the generated `cproton.py` module. That said, distutils has enough hooks to make something like this work, I'll take a look. Cheers, Flavio P.S: I was finally able to replicate this issue. - Original Message - From: "Gordon Sim" To: proton@qpid.apache.org Cc: "Flavio Percoco" , flape...@gmail.com Sent: Wednesday, June 10, 2015 12:56:51 PM Subject: Re: something rotten in the state of... something or other On 06/10/2015 03:24 PM, Flavio Percoco wrote: > On 09/06/15 12:30 -0400, Ken Giusti wrote: >> A betting man would wager it has something to do with the recent >> changes to the python setup.py. >> >> I'll have a look into it. >> >> - Original Message - >>> From: "Gordon Sim" >>> To: proton@qpid.apache.org >>> Sent: Tuesday, June 9, 2015 11:57:25 AM >>> Subject: something rotten in the state of... something or other >>> >>> I've recently started seeing errors[1] when running tests due to left >>> over artefacts of previous builds. This happens even for a completely >>> clean build directory, as some of the offending artefacts seem to be >>> created in the source tree. >>> >>> Jython seems to be trying and failing to load cproton. With a completely >>> clean source and build tree, everything passes, but it is kind of >>> annoying to have to rely on that. Is anyone else seeing anything >>> similar? Any ideas as to the cause (I've only seen it happening quite >>> recently) or possible cures? > > I haven't been able to replicate this but I'm afraid it might be my > fault. Does it happen to you every time? Pretty much, yes. On more digging, it appears that the proton-c/bindings/python/cproton.py file in the source tree is causing the problem. It seems to get generated when running the python-tox-tests, and once its there the proton-java tests fail with the error pasted previously. If I delete that file and also delete the .pyc, .class and .pyo objects in the source tree, then the java tests pass again. -- -K -- @flaper87 Flavio Percoco pgprbmItS_BQR.pgp Description: PGP signature
Re: something rotten in the state of... something or other
Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources. we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run. - Original Message - > From: "Gordon Sim" > To: proton@qpid.apache.org > Cc: "Flavio Percoco" , flape...@gmail.com > Sent: Wednesday, June 10, 2015 12:56:51 PM > Subject: Re: something rotten in the state of... something or other > > On 06/10/2015 03:24 PM, Flavio Percoco wrote: > > On 09/06/15 12:30 -0400, Ken Giusti wrote: > >> A betting man would wager it has something to do with the recent > >> changes to the python setup.py. > >> > >> I'll have a look into it. > >> > >> - Original Message - > >>> From: "Gordon Sim" > >>> To: proton@qpid.apache.org > >>> Sent: Tuesday, June 9, 2015 11:57:25 AM > >>> Subject: something rotten in the state of... something or other > >>> > >>> I've recently started seeing errors[1] when running tests due to left > >>> over artefacts of previous builds. This happens even for a completely > >>> clean build directory, as some of the offending artefacts seem to be > >>> created in the source tree. > >>> > >>> Jython seems to be trying and failing to load cproton. With a completely > >>> clean source and build tree, everything passes, but it is kind of > >>> annoying to have to rely on that. Is anyone else seeing anything > >>> similar? Any ideas as to the cause (I've only seen it happening quite > >>> recently) or possible cures? > > > > I haven't been able to replicate this but I'm afraid it might be my > > fault. Does it happen to you every time? > > Pretty much, yes. On more digging, it appears that the > proton-c/bindings/python/cproton.py file in the source tree is causing > the problem. It seems to get generated when running the > python-tox-tests, and once its there the proton-java tests fail with the > error pasted previously. > > If I delete that file and also delete the .pyc, .class and .pyo objects > in the source tree, then the java tests pass again. > -- -K
[jira] [Commented] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581918#comment-14581918 ] Ken Giusti commented on PROTON-905: --- reviewboard link: https://reviews.apache.org/r/35355/ > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Proton-c Null Messages
On Thu, 2015-06-11 at 13:40 +0100, Gordon Sim wrote: > On 06/11/2015 01:11 PM, aconway wrote: > > I disagree. An empty string is a perfectly legal value for a > > string. If > > the spec wants to assign special meaning to particular values of a > > property that needs to be stated. Of course, like you, I personally > > would not use an empty string as an identifier but as an > > implementor of > > an inter-operable spec I think we have to take the large view: > > *any* > > legal value of a parameter has to be considered equal unless the > > spec > > clearly states otherwise. > > If a name field is populated with an empty string, that to me is the > same as not supplying a name. An empty string is a legal encoding, > but > in my view it does not supply a value at all. (It is not like say 0 > which may be the default but is clearly a value in its own right). > It is exactly like 0, a perfectly legal value that is often abused to mean something special. "" is a legal string literal in every language I know of. It can be used as a key in a map or hash table. It can be compared with other strings. There is no string operation in any language I know that will throw NotAString if you apply it to "". > However, from the practical point of view... This is very practical. Interoperability is about agreeing on a type system. A "type" defines a range of legal values. The AMQP type system includes empty string and 0-length binaries as legal values for those types. We absolutely cannot treat any legal value in an exceptional way unless that is clearly mandated by the spec. > [...] > > Quote me the spec, this is a mater of law not opinion ;) > > I suspect that the sending of an empty string for a multi-frame > message > is entirely unintentional on the part of Apollo. I suspect it is a > bug > in Apollo or in the proton-j version/path it uses. That should be > confirmed and an appropriate JIRA raised That's fine for Apollo but irrelevant for proton. The first law of interoperability is "be strict with what you send, be forgiving with what you receive." To me that means that we should never *send* an empty delivery tag, but we should accept one unless the spec clearly states that it is illegal for anyone to ever send one. I see no such clear statement. . > > Proton-c should also not crash on receiving an empty (or null) > delivery > id. Beyond that I'm not overly concerned how it handles the empty > string > case.
Re: Proton-c Null Messages
On 06/11/2015 01:11 PM, aconway wrote: I disagree. An empty string is a perfectly legal value for a string. If the spec wants to assign special meaning to particular values of a property that needs to be stated. Of course, like you, I personally would not use an empty string as an identifier but as an implementor of an inter-operable spec I think we have to take the large view: *any* legal value of a parameter has to be considered equal unless the spec clearly states otherwise. If a name field is populated with an empty string, that to me is the same as not supplying a name. An empty string is a legal encoding, but in my view it does not supply a value at all. (It is not like say 0 which may be the default but is clearly a value in its own right). However, from the practical point of view... [...] Quote me the spec, this is a mater of law not opinion ;) I suspect that the sending of an empty string for a multi-frame message is entirely unintentional on the part of Apollo. I suspect it is a bug in Apollo or in the proton-j version/path it uses. That should be confirmed and an appropriate JIRA raised. Proton-c should also not crash on receiving an empty (or null) delivery id. Beyond that I'm not overly concerned how it handles the empty string case.
Re: Proton-c Null Messages
On Wed, 2015-06-10 at 16:32 +0100, Gordon Sim wrote: > On 06/10/2015 04:01 PM, aconway wrote: > > On Tue, 2015-06-09 at 19:54 +0100, Gordon Sim wrote: > > > On 06/09/2015 06:40 PM, logty wrote: > > > > When I run the client I get: > > > > > > > > [0x5351db0]:0 <- @transfer(20) [handle=0, delivery-id=0, > > > > delivery > > > > -tag=b"", > > > > message-format=0, settled=true, more=true] (16363) > > > > "\x00Sp\xc0\x07\x05B..." > > > > > > My guess would be that it is the delivery tag being null (or > > > empty, > > > can't tell which) that is the problem. From the spec: > > > > > > "This field MUST be specified for the first transfer of > > > a multi-transfer message and can only be omitted for > > > continuation transfers." [section 2.7.5] > > > > > > So I think that whatever is sending that frame has a bug. Proton > > > -c > > > has a > > > bug too of course, since it shouldn't segfault but should close > > > the > > > connection with a framing-error or similar. > > > > It says the field must be specified, it does not say it must not be > > an > > empty binary value. Is the field really missing or is proton > > choking on > > a 0-length delivery tag? > > I'm not sure the distinction between null and an empty value is very > useful here. The intent is that the delivery is clearly identified. I > > would argue that a 'zero byte identifier' doesn't meet the spirit of > the > law there. I disagree. An empty string is a perfectly legal value for a string. If the spec wants to assign special meaning to particular values of a property that needs to be stated. Of course, like you, I personally would not use an empty string as an identifier but as an implementor of an inter-operable spec I think we have to take the large view: *any* legal value of a parameter has to be considered equal unless the spec clearly states otherwise. > > It shouldn't, which might explain why rabbit is OK with > > it. > > I don't think RabbitMQ is ever seeing that frame. I believe that > frame > is emitted by ApolloMQ to the receiving client. Rabbit, Apollo, whoever. If somebody is using the empty string as a delivery tag and the spec does not clearly state "you must never use the empty string as a delivery tag" then we should accept it. > I agree that proton should not choke on a zero byte delivery tag (or > indeed on a non-existent delivery tag). But I do think it's a bug to > send such a frame. Quote me the spec, this is a mater of law not opinion ;)
Re: Strange behaviour for pn_messenger_send on CentOS 6
Thanks - at least I know I'm not going mad! Just raised https://issues.apache.org/jira/browse/PROTON-907 for tracking. Cheers, Frank On Wed, Jun 10, 2015 at 2:52 PM, Darryl L. Pierce wrote: > On Wed, Jun 10, 2015 at 02:17:23PM +0100, Frank Quinn wrote: > > You can recreate this on a CentOS 6.6 box by installing > qpid-proton-c-devel > > using yum from EPEL, then compiling the example application that comes > with > > it in /usr/share/proton/examples/messenger/send.c. > > > > There is no broker with these example applications - it's point to point. > > > > There is a matching recv.c application there too. If you start recv > first, > > it works fine, but if you don't, the send application hangs which I > believe > > is new behaviour. > > > > Again, this does not happen on my Fedora laptop - only on CentOS 6. > > I'm able to recreate this on RHEL6 as well using the RPMs from EPEL. I > see the connection aborted notice in the trace and then it fails to > exit. It appears to be repeatedly calling pn_transport_closed(). The > stack trace I see on EL6 is: > > (gdb) backtrace > #0 pn_transport_closed (transport=0x804fa28) > at > /home/mcpierce/Programming/Proton/proton-c/src/transport/transport.c:2802 > #1 0x00140ea8 in pni_connection_capacity (ctx=0x804f920) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:166 > #2 pni_connection_update (ctx=0x804f920) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:196 > #3 pni_conn_modified (ctx=0x804f920) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:225 > #4 0x00141071 in pn_messenger_process_transport (messenger=0x804cb40, > event=0x805c520) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1201 > #5 0x00141134 in pn_messenger_process_events (messenger=0x804cb40) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1252 > #6 0x00141f83 in pni_connection_readable (sel=0x804f958) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:261 > #7 0x00143425 in pn_selectable_readable (selectable=0x804f958) > at /home/mcpierce/Programming/Proton/proton-c/src/selectable.c:204 > #8 0x00141483 in pn_messenger_process (messenger=0x804cb40) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1310 > #9 0x001415c8 in pn_messenger_tsync (messenger=0x804cb40, > predicate=0x13da40 , timeout=-1) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1379 > #10 0x00141a97 in pn_messenger_sync (messenger=0x804cb40, > predicate=0x13da40 ) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1410 > #11 0x00141c8c in pn_messenger_send (messenger=0x804cb40, n=-1) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:2119 > #12 0x08048e59 in main (argc=-72537468, argv=0xb7ffe000) > at /home/mcpierce/Programming/Proton/examples/c/messenger/send.c:102 > > The same backtrace on F22 is: > > (gdb) backtrace > #0 pn_transport_closed (transport=transport@entry=0x60ac40) > at > /home/mcpierce/Programming/Proton/proton-c/src/transport/transport.c:2801 > #1 0x77bbf0a8 in pni_connection_capacity (sel=0x60aaf0) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:166 > #2 pni_connection_update (sel=0x60aaf0) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:196 > #3 pni_conn_modified (ctx=0x60aa80) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:225 > #4 0x77bbf135 in pn_messenger_process_transport > (messenger=messenger@entry=0x605970, > event=event@entry=0x61a240) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1201 > #5 0x77bbf27b in pn_messenger_process_events > (messenger=messenger@entry=0x605970) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1252 > #6 0x77bbf6d9 in pni_connection_readable (sel=0x60aaf0) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:261 > #7 0x77bbf7e8 in pn_messenger_process (messenger=messenger@entry > =0x605970) > at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1310 > #8 0x77bbf94f in pn_messenger_tsync (messenger=0x605970, > predicate=0x77bbc420 , > timeout=-1) at > /home/mcpierce/Programming/Proton/proton-c/src/messenger/messenger.c:1379 > #9 0x00401158 in main (argc=, argv=) > at /home/mcpierce/Programming/Proton/examples/c/messenger/send.c:102 > > There's definitely a difference in the runtime behavior of Fedora vs. > RHEL in this case. > > -- > Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. > Delivering value year after year. > Red Hat ranks #1 in value among software vendors. > http://www.redhat.com/promo/vendor/ > >
[jira] [Created] (PROTON-907) Qpid Proton Point to Point Hang on CentOS 6 pn_messenger_send
Frank Quinn created PROTON-907: -- Summary: Qpid Proton Point to Point Hang on CentOS 6 pn_messenger_send Key: PROTON-907 URL: https://issues.apache.org/jira/browse/PROTON-907 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.9.1, 0.8 Environment: CentOS 6 (both VM and native 64-bit) and RHEL 6 Reporter: Frank Quinn Priority: Critical See thread at http://qpid.2158936.n2.nabble.com/Strange-behaviour-for-pn-messenger-send-on-CentOS-6-td7625846.html. Key points: * pn_messenger_send will hang on CentOS 6 if the destination is not yet up * Works fine on Fedora 21 and 22 (by 'fine', i mean it will attempt to send, fail and move on) * Can be recreated by running the send.c application when recv.c is not yet running * Proton burns CPU as it hangs This effectively deadlocks our application. So far, I’ve tried compiling qpid proton c myself (both 0.8 and 0.9.1), setting pn_messenger_send timeout to 1 (it was previously -1), turning off iptables entirely and disabling selinux and rebooting but no luck. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
