ProtonJ compilation and test failures
The recent changes on Proton-J seemed to have created some issues: https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1032/console The module currently requries Java 7 to compile, which is a slightly out of sync with the compiler source+target still being set to Java 6 (which the above job is using). Once using Java 8 to do the maven build locally, the python tests then failed with: proton_tests.utils.SyncRequestResponseTest.test_request_response fail Error during test: Traceback (most recent call last): File "/home/gemmellr/workspace/proton/tests/python/proton-test", line 360, in run phase() File "/home/gemmellr/workspace/proton/tests/python/proton_tests/utils.py", line 89, in test_request_response connection = BlockingConnection(server.url, timeout=self.timeout) File "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", line 195, in __init__ self.wait(lambda: not (self.conn.state & Endpoint.REMOTE_UNINIT), File "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", line 229, in wait container_timeout = self.container.timeout File "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", line 104, in _get_timeout return millis2timeout(pn_reactor_get_timeout(self._impl)) File "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/__init__.py", line 2337, in millis2timeout if millis == PN_MILLIS_MAX: return None NameError: global name 'PN_MILLIS_MAX' is not defined I notice that the TravisCI job did pass: https://travis-ci.org/apache/qpid-proton/builds/69665060 I guess the main difference is it ran via cmake so the proton-c build was performed before the proton-j tests were run. Robbie
Re: qpid-proton git commit: PROTON_927: only write out values for expiry and creation time if these are non-zero, which passes at present for a test that they have been set
This seems to be resulting in segfaults running the tests on Windows: https://ci.appveyor.com/project/ke4qqq/qpid-proton/build/0.10-SNAPSHOT-master.122 1: proton_tests.message.CodecTest.testRoundTrip pass 1: proton_tests.message.CodecTest.testRoundTripWithTimes ... 1/8 Test #1: python-test ..***Exception: SegFault 19.63 sec On 3 July 2015 at 11:10, wrote: > Repository: qpid-proton > Updated Branches: > refs/heads/master fc38e86a6 -> 57b0f34ab > > > PROTON_927: only write out values for expiry and creation time if these are > non-zero, which passes at present for a test that they have been set > > > Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo > Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/57b0f34a > Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/57b0f34a > Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/57b0f34a > > Branch: refs/heads/master > Commit: 57b0f34ab9904209fdc67aa066c72e0f80b3e966 > Parents: fc38e86 > Author: Gordon Sim > Authored: Wed Jul 1 18:31:08 2015 +0100 > Committer: Gordon Sim > Committed: Fri Jul 3 10:34:57 2015 +0100 > > -- > proton-c/src/message/message.c | 4 +++- > tests/python/proton_tests/message.py | 13 + > 2 files changed, 16 insertions(+), 1 deletion(-) > -- > > > http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/57b0f34a/proton-c/src/message/message.c > -- > diff --git a/proton-c/src/message/message.c b/proton-c/src/message/message.c > index b24a994..847c314 100644 > --- a/proton-c/src/message/message.c > +++ b/proton-c/src/message/message.c > @@ -792,7 +792,7 @@ int pn_message_data(pn_message_t *msg, pn_data_t *data) > pn_data_exit(data); >} > > - err = pn_data_fill(data, "DL[CzSSSCssttSIS]", PROPERTIES, > + err = pn_data_fill(data, "DL[CzSSSCss?t?tSIS]", PROPERTIES, > msg->id, > pn_string_size(msg->user_id), > pn_string_get(msg->user_id), > pn_string_get(msg->address), > @@ -802,6 +802,8 @@ int pn_message_data(pn_message_t *msg, pn_data_t *data) > pn_string_get(msg->content_type), > pn_string_get(msg->content_encoding), > msg->expiry_time, > + msg->expiry_time, > + msg->creation_time, > msg->creation_time, > pn_string_get(msg->group_id), > msg->group_sequence, > > http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/57b0f34a/tests/python/proton_tests/message.py > -- > diff --git a/tests/python/proton_tests/message.py > b/tests/python/proton_tests/message.py > index 2f7cb4f..899cbab 100644 > --- a/tests/python/proton_tests/message.py > +++ b/tests/python/proton_tests/message.py > @@ -129,3 +129,16 @@ class CodecTest(Test): > assert self.msg.address == msg2.address, (self.msg.address, msg2.address) > assert self.msg.subject == msg2.subject, (self.msg.subject, msg2.subject) > assert self.msg.body == msg2.body, (self.msg.body, msg2.body) > + > + def testRoundTripWithTimes(self): > +self.msg.expiry_time = 987654321; > +self.msg.creation_time = 123456789; > +self.msg.body = 'Hello World!' > + > +data = self.msg.encode() > + > +msg2 = Message() > +msg2.decode(data) > + > +assert self.msg.expiry_time == msg2.expiry_time, (self.msg.expiry_time, > msg2.expiry_time) > +assert self.msg.creation_time == msg2.creation_time, > (self.msg.creation_time, msg2.creation_time) > > > - > To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org > For additional commands, e-mail: commits-h...@qpid.apache.org >
Re: qpid-proton git commit: PROTON_927: only write out values for expiry and creation time if these are non-zero, which passes at present for a test that they have been set
On 07/06/2015 10:52 AM, Robbie Gemmell wrote: This seems to be resulting in segfaults running the tests on Windows: https://ci.appveyor.com/project/ke4qqq/qpid-proton/build/0.10-SNAPSHOT-master.122 1: proton_tests.message.CodecTest.testRoundTrip pass 1: proton_tests.message.CodecTest.testRoundTripWithTimes ... 1/8 Test #1: python-test ..***Exception: SegFault 19.63 sec I'll have a look and either fix or back it out.
Re: ProtonJ compilation and test failures
I just ran a maven-only clean build locally with no problems. You should have PN_MILLIS_MAX defined in proton-j/src/main/resources/ctypes.py, and this should be imported from proton-j/src/main/resources/cproton.py. Can you verify that this is as expected? --Rafael On Mon, Jul 6, 2015 at 5:50 AM, Robbie Gemmell wrote: > The recent changes on Proton-J seemed to have created some issues: > https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1032/console > > The module currently requries Java 7 to compile, which is a slightly > out of sync with the compiler source+target still being set to Java 6 > (which the above job is using). > > Once using Java 8 to do the maven build locally, the python tests then > failed with: > > proton_tests.utils.SyncRequestResponseTest.test_request_response > fail > Error during test: Traceback (most recent call last): > File "/home/gemmellr/workspace/proton/tests/python/proton-test", > line 360, in run > phase() > File > "/home/gemmellr/workspace/proton/tests/python/proton_tests/utils.py", > line 89, in test_request_response > connection = BlockingConnection(server.url, timeout=self.timeout) > File > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", > line 195, in __init__ > self.wait(lambda: not (self.conn.state & Endpoint.REMOTE_UNINIT), > File > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", > line 229, in wait > container_timeout = self.container.timeout > File > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", > line 104, in _get_timeout > return millis2timeout(pn_reactor_get_timeout(self._impl)) > File > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/__init__.py", > line 2337, in millis2timeout > if millis == PN_MILLIS_MAX: return None > NameError: global name 'PN_MILLIS_MAX' is not defined > > I notice that the TravisCI job did pass: > https://travis-ci.org/apache/qpid-proton/builds/69665060 > > I guess the main difference is it ran via cmake so the proton-c build > was performed before the proton-j tests were run. > > Robbie >
[jira] [Commented] (PROTON-927) absolute-expiry-time and creation-time are encoded as 0 if not set
[ https://issues.apache.org/jira/browse/PROTON-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614915#comment-14614915 ] ASF subversion and git services commented on PROTON-927: Commit 23a4e3bbb77e14cff6366d7a87aa804f84baa7f6 in qpid-proton's branch refs/heads/master from [~gsim] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=23a4e3b ] PROTON-927: reverse previous change pending investigation into segfault on windows in the added test > absolute-expiry-time and creation-time are encoded as 0 if not set > -- > > Key: PROTON-927 > URL: https://issues.apache.org/jira/browse/PROTON-927 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Gordon Sim >Assignee: Gordon Sim > Fix For: 0.10 > > > They should instead be encoded as null (since there is no default). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (PROTON-927) absolute-expiry-time and creation-time are encoded as 0 if not set
[ https://issues.apache.org/jira/browse/PROTON-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gordon Sim reopened PROTON-927: --- Assignee: (was: Gordon Sim) The new test was reported to have segfaulted on windows: https://ci.appveyor.com/project/ke4qqq/qpid-proton/build/0.10-SNAPSHOT-master.122 Reverting the change. > absolute-expiry-time and creation-time are encoded as 0 if not set > -- > > Key: PROTON-927 > URL: https://issues.apache.org/jira/browse/PROTON-927 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Gordon Sim > Fix For: 0.10 > > > They should instead be encoded as null (since there is no default). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-927) absolute-expiry-time and creation-time are encoded as 0 if not set
[ https://issues.apache.org/jira/browse/PROTON-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gordon Sim updated PROTON-927: -- Fix Version/s: (was: 0.10) > absolute-expiry-time and creation-time are encoded as 0 if not set > -- > > Key: PROTON-927 > URL: https://issues.apache.org/jira/browse/PROTON-927 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Gordon Sim > > They should instead be encoded as null (since there is no default). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: qpid-proton git commit: PROTON_927: only write out values for expiry and creation time if these are non-zero, which passes at present for a test that they have been set
On 07/06/2015 11:04 AM, Gordon Sim wrote: On 07/06/2015 10:52 AM, Robbie Gemmell wrote: This seems to be resulting in segfaults running the tests on Windows: https://ci.appveyor.com/project/ke4qqq/qpid-proton/build/0.10-SNAPSHOT-master.122 1: proton_tests.message.CodecTest.testRoundTrip pass 1: proton_tests.message.CodecTest.testRoundTripWithTimes ... 1/8 Test #1: python-test ..***Exception: SegFault 19.63 sec I'll have a look and either fix or back it out. I couldn't see anything wrong in the change. I suspect it may be related to the test. However I have reverted the commit in its entirety for now.
Re: ProtonJ compilation and test failures
All the ProtonJInterop tests fail for me, and the python-test then hangs. The error for each is something like: 2: proton_tests.reactor_interop.ReactorInteropTest. \ 2: Error: Could not find or load main class org.apache.qpid.proton.ProtonJInterop 2: test_protonc_to_protonj_1 ... fail 2: Error during test: Traceback (most recent call last): 2: File "/home/gordon/projects/proton-git/tests/python/proton-test", line 360, in run 2: phase() 2: File "/home/gordon/projects/proton-git/tests/python/proton_tests/reactor_interop.py", line 141, in test_protonc_to_protonj_1 2: self.protonc_to_protonj(1) 2: File "/home/gordon/projects/proton-git/tests/python/proton_tests/reactor_interop.py", line 124, in protonc_to_protonj 2: assert(java_thread.result == 0) 2: AssertionError 2: proton_tests.reactor_interop.ReactorInteropTest. \ 2: Error: Could not find or load main class org.apache.qpid.proton.ProtonJInterop
Re: ProtonJ compilation and test failures
Can you try doing an mvn clean and seeing if it is still an issue? A class entirely missing like that is usually due to mvn not recompiling everything that is impacted by a given change. --Rafael On Mon, Jul 6, 2015 at 8:11 AM, Gordon Sim wrote: > All the ProtonJInterop tests fail for me, and the python-test then hangs. > The error for each is something like: > > 2: proton_tests.reactor_interop.ReactorInteropTest. \ >> 2: Error: Could not find or load main class >> org.apache.qpid.proton.ProtonJInterop >> 2: test_protonc_to_protonj_1 >> ... fail >> 2: Error during test: Traceback (most recent call last): >> 2: File "/home/gordon/projects/proton-git/tests/python/proton-test", >> line 360, in run >> 2: phase() >> 2: File >> "/home/gordon/projects/proton-git/tests/python/proton_tests/reactor_interop.py", >> line 141, in test_protonc_to_protonj_1 >> 2: self.protonc_to_protonj(1) >> 2: File >> "/home/gordon/projects/proton-git/tests/python/proton_tests/reactor_interop.py", >> line 124, in protonc_to_protonj >> 2: assert(java_thread.result == 0) >> 2: AssertionError >> 2: proton_tests.reactor_interop.ReactorInteropTest. \ >> 2: Error: Could not find or load main class >> org.apache.qpid.proton.ProtonJInterop >> > >
Re: ProtonJ compilation and test failures
Were you running it after having previously used the cmake build in the same terminal? I do indeed have the definition in ctypes, with the cproton file importing everything from ctypes. The maven build failed when I ran it directly in my git-clean'ed checkout. It then passed when run indirectly via cmake and make test. It then passed if run directly again. However as it turns out, it also passed in the same terminal after I git-clean'ed the checkout again. It failed if I run it again directly in the same checkout but using a new terminal where I hadn't used the cmake build. Robbie On 6 July 2015 at 11:57, Rafael Schloming wrote: > I just ran a maven-only clean build locally with no problems. > > You should have PN_MILLIS_MAX defined in > proton-j/src/main/resources/ctypes.py, and this should be imported from > proton-j/src/main/resources/cproton.py. Can you verify that this is as > expected? > > --Rafael > > On Mon, Jul 6, 2015 at 5:50 AM, Robbie Gemmell > wrote: > >> The recent changes on Proton-J seemed to have created some issues: >> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1032/console >> >> The module currently requries Java 7 to compile, which is a slightly >> out of sync with the compiler source+target still being set to Java 6 >> (which the above job is using). >> >> Once using Java 8 to do the maven build locally, the python tests then >> failed with: >> >> proton_tests.utils.SyncRequestResponseTest.test_request_response >> fail >> Error during test: Traceback (most recent call last): >> File "/home/gemmellr/workspace/proton/tests/python/proton-test", >> line 360, in run >> phase() >> File >> "/home/gemmellr/workspace/proton/tests/python/proton_tests/utils.py", >> line 89, in test_request_response >> connection = BlockingConnection(server.url, timeout=self.timeout) >> File >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", >> line 195, in __init__ >> self.wait(lambda: not (self.conn.state & Endpoint.REMOTE_UNINIT), >> File >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", >> line 229, in wait >> container_timeout = self.container.timeout >> File >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", >> line 104, in _get_timeout >> return millis2timeout(pn_reactor_get_timeout(self._impl)) >> File >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/__init__.py", >> line 2337, in millis2timeout >> if millis == PN_MILLIS_MAX: return None >> NameError: global name 'PN_MILLIS_MAX' is not defined >> >> I notice that the TravisCI job did pass: >> https://travis-ci.org/apache/qpid-proton/builds/69665060 >> >> I guess the main difference is it ran via cmake so the proton-c build >> was performed before the proton-j tests were run. >> >> Robbie >>
Re: ProtonJ compilation and test failures
Can you do a git pull and give it another shot? I believe what is happening is that when maven launches the jython tests, it doesn't seem to include the jython shim in the class path. For some reason, this isn't an issue of the .class files that jython generates are hanging around in the source tree from a previous build, or if the JYTHONPATH environment variable is set to include the appropriate path. I'm not sure how long this has been the case, but I suspect it may actually predate the reactor changes. I think they just triggered the latent issue for you by adding another file that did not already have a generated .class file sitting around in the tree. I've modified JythonTest to add the shim to the classpath and it now works for me without needing any help from a prior cmake build. --Rafael On Mon, Jul 6, 2015 at 8:25 AM, Robbie Gemmell wrote: > Were you running it after having previously used the cmake build in > the same terminal? > > I do indeed have the definition in ctypes, with the cproton file > importing everything from ctypes. The maven build failed when I ran it > directly in my git-clean'ed checkout. It then passed when run > indirectly via cmake and make test. It then passed if run directly > again. However as it turns out, it also passed in the same terminal > after I git-clean'ed the checkout again. It failed if I run it again > directly in the same checkout but using a new terminal where I hadn't > used the cmake build. > > Robbie > > On 6 July 2015 at 11:57, Rafael Schloming wrote: > > I just ran a maven-only clean build locally with no problems. > > > > You should have PN_MILLIS_MAX defined in > > proton-j/src/main/resources/ctypes.py, and this should be imported from > > proton-j/src/main/resources/cproton.py. Can you verify that this is as > > expected? > > > > --Rafael > > > > On Mon, Jul 6, 2015 at 5:50 AM, Robbie Gemmell > > > wrote: > > > >> The recent changes on Proton-J seemed to have created some issues: > >> > https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1032/console > >> > >> The module currently requries Java 7 to compile, which is a slightly > >> out of sync with the compiler source+target still being set to Java 6 > >> (which the above job is using). > >> > >> Once using Java 8 to do the maven build locally, the python tests then > >> failed with: > >> > >> proton_tests.utils.SyncRequestResponseTest.test_request_response > > >> fail > >> Error during test: Traceback (most recent call last): > >> File "/home/gemmellr/workspace/proton/tests/python/proton-test", > >> line 360, in run > >> phase() > >> File > >> "/home/gemmellr/workspace/proton/tests/python/proton_tests/utils.py", > >> line 89, in test_request_response > >> connection = BlockingConnection(server.url, timeout=self.timeout) > >> File > >> > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", > >> line 195, in __init__ > >> self.wait(lambda: not (self.conn.state & Endpoint.REMOTE_UNINIT), > >> File > >> > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", > >> line 229, in wait > >> container_timeout = self.container.timeout > >> File > >> > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", > >> line 104, in _get_timeout > >> return millis2timeout(pn_reactor_get_timeout(self._impl)) > >> File > >> > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/__init__.py", > >> line 2337, in millis2timeout > >> if millis == PN_MILLIS_MAX: return None > >> NameError: global name 'PN_MILLIS_MAX' is not defined > >> > >> I notice that the TravisCI job did pass: > >> https://travis-ci.org/apache/qpid-proton/builds/69665060 > >> > >> I guess the main difference is it ran via cmake so the proton-c build > >> was performed before the proton-j tests were run. > >> > >> Robbie > >> >
Re: ProtonJ compilation and test failures
On 07/06/2015 01:24 PM, Rafael Schloming wrote: Can you try doing an mvn clean and seeing if it is still an issue? I see the same thing after mvn clean
Re: ProtonJ compilation and test failures
That seemed to do the trick. Running the maven build in a clean checkout in clean terminal now works, so long as you use a version of Java able to build the recent source; the compilation issue is still there otherwise: https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-c/ https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/ Robbie On 6 July 2015 at 13:55, Rafael Schloming wrote: > Can you do a git pull and give it another shot? > > I believe what is happening is that when maven launches the jython tests, > it doesn't seem to include the jython shim in the class path. For some > reason, this isn't an issue of the .class files that jython generates are > hanging around in the source tree from a previous build, or if the > JYTHONPATH environment variable is set to include the appropriate path. > > I'm not sure how long this has been the case, but I suspect it may actually > predate the reactor changes. I think they just triggered the latent issue > for you by adding another file that did not already have a generated .class > file sitting around in the tree. > > I've modified JythonTest to add the shim to the classpath and it now works > for me without needing any help from a prior cmake build. > > --Rafael > > > > On Mon, Jul 6, 2015 at 8:25 AM, Robbie Gemmell > wrote: > >> Were you running it after having previously used the cmake build in >> the same terminal? >> >> I do indeed have the definition in ctypes, with the cproton file >> importing everything from ctypes. The maven build failed when I ran it >> directly in my git-clean'ed checkout. It then passed when run >> indirectly via cmake and make test. It then passed if run directly >> again. However as it turns out, it also passed in the same terminal >> after I git-clean'ed the checkout again. It failed if I run it again >> directly in the same checkout but using a new terminal where I hadn't >> used the cmake build. >> >> Robbie >> >> On 6 July 2015 at 11:57, Rafael Schloming wrote: >> > I just ran a maven-only clean build locally with no problems. >> > >> > You should have PN_MILLIS_MAX defined in >> > proton-j/src/main/resources/ctypes.py, and this should be imported from >> > proton-j/src/main/resources/cproton.py. Can you verify that this is as >> > expected? >> > >> > --Rafael >> > >> > On Mon, Jul 6, 2015 at 5:50 AM, Robbie Gemmell > > >> > wrote: >> > >> >> The recent changes on Proton-J seemed to have created some issues: >> >> >> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1032/console >> >> >> >> The module currently requries Java 7 to compile, which is a slightly >> >> out of sync with the compiler source+target still being set to Java 6 >> >> (which the above job is using). >> >> >> >> Once using Java 8 to do the maven build locally, the python tests then >> >> failed with: >> >> >> >> proton_tests.utils.SyncRequestResponseTest.test_request_response >> >> >> fail >> >> Error during test: Traceback (most recent call last): >> >> File "/home/gemmellr/workspace/proton/tests/python/proton-test", >> >> line 360, in run >> >> phase() >> >> File >> >> "/home/gemmellr/workspace/proton/tests/python/proton_tests/utils.py", >> >> line 89, in test_request_response >> >> connection = BlockingConnection(server.url, timeout=self.timeout) >> >> File >> >> >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", >> >> line 195, in __init__ >> >> self.wait(lambda: not (self.conn.state & Endpoint.REMOTE_UNINIT), >> >> File >> >> >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/utils.py", >> >> line 229, in wait >> >> container_timeout = self.container.timeout >> >> File >> >> >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", >> >> line 104, in _get_timeout >> >> return millis2timeout(pn_reactor_get_timeout(self._impl)) >> >> File >> >> >> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/__init__.py", >> >> line 2337, in millis2timeout >> >> if millis == PN_MILLIS_MAX: return None >> >> NameError: global name 'PN_MILLIS_MAX' is not defined >> >> >> >> I notice that the TravisCI job did pass: >> >> https://travis-ci.org/apache/qpid-proton/builds/69665060 >> >> >> >> I guess the main difference is it ran via cmake so the proton-c build >> >> was performed before the proton-j tests were run. >> >> >> >> Robbie >> >> >>
Re: ProtonJ compilation and test failures
On 6 July 2015 at 14:17, Gordon Sim wrote: > On 07/06/2015 01:24 PM, Rafael Schloming wrote: >> >> Can you try doing an mvn clean and seeing if it is still an issue? > > > I see the same thing after mvn clean > Does cleaning the checkout as a whole make any difference? To preview what woudl be deleted: git clean -n -d -x -e "*.classpath" -e "*.project" -e "*.settings" . To actually delete things: git clean -f -d -x -e "*.classpath" -e "*.project" -e "*.settings" . The -e flags are protecting project files generated by Eclipse...if you dont use it, no need for them.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
Is this change allowing clients to skip the SASL layer when connecting
to servers that have enabled the SASL layer? If so, how is the new
default behaviour disabled?
The existing but unimplemented 'allowSkip' method previously intended
to enable such behaviour still doesn't do anything, so is there a way
to require clients use a SASL layer as would have been previously
after enabling SASL for a proton-j (and in the past a proton-c)
server?
Robbie
On 6 July 2015 at 00:45, wrote:
> implemented sasl sniffing for proton-j; this allows the reactor interop tests
> to pass
>
>
> Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo
> Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/46edaebe
> Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/46edaebe
> Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/46edaebe
>
> Branch: refs/heads/master
> Commit: 46edaebeb92fa780120f17aa8ac0b54d2feaa8b9
> Parents: bf81c44
> Author: Rafael Schloming
> Authored: Sun Jul 5 19:08:20 2015 -0400
> Committer: Rafael Schloming
> Committed: Sun Jul 5 19:33:45 2015 -0400
>
> --
> proton-c/bindings/python/proton/__init__.py | 2 +-
> .../impl/HandshakeSniffingTransportWrapper.java | 182 +++
> .../qpid/proton/engine/impl/SaslImpl.java | 12 +-
> .../qpid/proton/engine/impl/SaslSniffer.java| 53 ++
> .../SslHandshakeSniffingTransportWrapper.java | 170 +++--
> proton-j/src/main/resources/cengine.py | 26 ++-
> 6 files changed, 284 insertions(+), 161 deletions(-)
> --
>
>
> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/46edaebe/proton-c/bindings/python/proton/__init__.py
> --
> diff --git a/proton-c/bindings/python/proton/__init__.py
> b/proton-c/bindings/python/proton/__init__.py
> index 1032edf..9c75800 100644
> --- a/proton-c/bindings/python/proton/__init__.py
> +++ b/proton-c/bindings/python/proton/__init__.py
> @@ -3294,7 +3294,7 @@ A callback for trace logging. The callback is passed
> the transport and log messa
>def push(self, binary):
> n = self._check(pn_transport_push(self._impl, binary))
> if n != len(binary):
> - raise OverflowError("unable to process all bytes")
> + raise OverflowError("unable to process all bytes: %s, %s" % (n,
> len(binary)))
>
>def close_tail(self):
> self._check(pn_transport_close_tail(self._impl))
>
> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/46edaebe/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/HandshakeSniffingTransportWrapper.java
> --
> diff --git
> a/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/HandshakeSniffingTransportWrapper.java
>
> b/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/HandshakeSniffingTransportWrapper.java
> new file mode 100644
> index 000..6a5aac5
> --- /dev/null
> +++
> b/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/HandshakeSniffingTransportWrapper.java
> @@ -0,0 +1,182 @@
> +/*
> + *
> + * Licensed to the Apache Software Foundation (ASF) under one
> + * or more contributor license agreements. See the NOTICE file
> + * distributed with this work for additional information
> + * regarding copyright ownership. The ASF licenses this file
> + * to you under the Apache License, Version 2.0 (the
> + * "License"); you may not use this file except in compliance
> + * with the License. You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing,
> + * software distributed under the License is distributed on an
> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> + * KIND, either express or implied. See the License for the
> + * specific language governing permissions and limitations
> + * under the License.
> + *
> + */
> +package org.apache.qpid.proton.engine.impl;
> +
> +import java.nio.ByteBuffer;
> +
> +import org.apache.qpid.proton.engine.Transport;
> +import org.apache.qpid.proton.engine.TransportException;
> +import org.apache.qpid.proton.engine.impl.TransportWrapper;
> +
> +public abstract class HandshakeSniffingTransportWrapper TransportWrapper, T2 extends TransportWrapper>
> +implements TransportWrapper
> +{
> +
> +protected final T1 _wrapper1;
> +protected final T2 _wrapper2;
> +
> +private boolean _tail_closed = false;
> +private boolean _head_closed = false;
> +protected TransportWrapper _selectedTransportWrapper;
> +
> +private final ByteBuffer _determinationBuffer;
> +
> +protected HandshakeSniffingTransportWrapper
> +(T1 wrapper1,
> + T2 wrapper2)
> +{
> +_wrapper1 = wrapper1;
> +_wr
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On Mon, Jul 6, 2015 at 9:52 AM, Robbie Gemmell wrote: > Is this change allowing clients to skip the SASL layer when connecting > to servers that have enabled the SASL layer? If so, how is the new > default behaviour disabled? > Yes, it was necessary to allow the tests to pass. > The existing but unimplemented 'allowSkip' method previously intended > to enable such behaviour still doesn't do anything, so is there a way > to require clients use a SASL layer as would have been previously > after enabling SASL for a proton-j (and in the past a proton-c) > server? > Ah, I didn't notice that. Thanks for pointing it out. I'll wire it up and cross my fingers that the tests still pass. --Rafael
Re: ProtonJ compilation and test failures
On 07/06/2015 02:23 PM, Robbie Gemmell wrote: On 6 July 2015 at 14:17, Gordon Sim wrote: On 07/06/2015 01:24 PM, Rafael Schloming wrote: Can you try doing an mvn clean and seeing if it is still an issue? I see the same thing after mvn clean Does cleaning the checkout as a whole make any difference? Doesn't seem to, no. To preview what woudl be deleted: git clean -n -d -x -e "*.classpath" -e "*.project" -e "*.settings" . To actually delete things: git clean -f -d -x -e "*.classpath" -e "*.project" -e "*.settings" . The -e flags are protecting project files generated by Eclipse...if you dont use it, no need for them.
Re: ProtonJ compilation and test failures
Any sort of missing class really should be a compile time exception, which I think means you must have stale class files *somewhere*. You could try doing a find checkout -name "*.class" just as a sanity check. Also, it's possible something in your local maven repo is somehow coming into play, maybe blow that away and rebuild it and/or do an mvn install to be sure that remove dependencies aren't out of sync with local code? --Rafael On Mon, Jul 6, 2015 at 11:02 AM, Gordon Sim wrote: > On 07/06/2015 02:23 PM, Robbie Gemmell wrote: > >> On 6 July 2015 at 14:17, Gordon Sim wrote: >> >>> On 07/06/2015 01:24 PM, Rafael Schloming wrote: >>> Can you try doing an mvn clean and seeing if it is still an issue? >>> >>> >>> I see the same thing after mvn clean >>> >>> >> Does cleaning the checkout as a whole make any difference? >> > > Doesn't seem to, no. > > > To preview what woudl be deleted: >> git clean -n -d -x -e "*.classpath" -e "*.project" -e "*.settings" . >> >> To actually delete things: >> git clean -f -d -x -e "*.classpath" -e "*.project" -e "*.settings" . >> >> The -e flags are protecting project files generated by Eclipse...if >> you dont use it, no need for them. >> >> >
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On Mon, 2015-07-06 at 10:56 -0400, Rafael Schloming wrote: > On Mon, Jul 6, 2015 at 9:52 AM, Robbie Gemmell > wrote: > > > Is this change allowing clients to skip the SASL layer when connecting > > to servers that have enabled the SASL layer? If so, how is the new > > default behaviour disabled? > > > > Yes, it was necessary to allow the tests to pass. > > > > The existing but unimplemented 'allowSkip' method previously intended > > to enable such behaviour still doesn't do anything, so is there a way > > to require clients use a SASL layer as would have been previously > > after enabling SASL for a proton-j (and in the past a proton-c) > > server? > > > > Ah, I didn't notice that. Thanks for pointing it out. I'll wire it up and > cross my fingers that the tests still pass. Allow_skip is no longer present in the C API it is replaced with the require_auth (now on the transport object) API. So it would make more sense to implement that and remove allow_skip. > > --Rafael
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
I wired in allowSkip in a very minimal way just to restore the ability to force the old behaviour. It would be a fairly trivial to change the name of course, however it appears there are a bunch of other related changes that go along with it, e.g. adding a bunch of accessors and fixing the error behavior. Currently if you put in require authentication the java sasl layer will simply die with a TransportException if it sees a regular AMQP header, and the tests appear to expect something more graceful. I stopped there because I noticed a bunch of other unimplemented stuff and I wasn't sure how deep the bottom of the rabbit hole was. --Rafael On Mon, Jul 6, 2015 at 11:11 AM, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 10:56 -0400, Rafael Schloming wrote: > > On Mon, Jul 6, 2015 at 9:52 AM, Robbie Gemmell > > > wrote: > > > > > Is this change allowing clients to skip the SASL layer when connecting > > > to servers that have enabled the SASL layer? If so, how is the new > > > default behaviour disabled? > > > > > > > Yes, it was necessary to allow the tests to pass. > > > > > > > The existing but unimplemented 'allowSkip' method previously intended > > > to enable such behaviour still doesn't do anything, so is there a way > > > to require clients use a SASL layer as would have been previously > > > after enabling SASL for a proton-j (and in the past a proton-c) > > > server? > > > > > > > Ah, I didn't notice that. Thanks for pointing it out. I'll wire it up and > > cross my fingers that the tests still pass. > > Allow_skip is no longer present in the C API it is replaced with the > require_auth (now on the transport object) API. > > So it would make more sense to implement that and remove allow_skip. > > > > > --Rafael > > >
[jira] [Commented] (PROTON-930) add explicit AMQP 1.0 constants
[ https://issues.apache.org/jira/browse/PROTON-930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615190#comment-14615190 ] ASF subversion and git services commented on PROTON-930: Commit 3c14a0d1cf89b988529b98dc10c8f2c17abcdcf3 in qpid-proton's branch refs/heads/master from mgoulish [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=3c14a0d ] PROTON-925: use of UINT32_MAX is breaking some builds. Getting better constants will be part of PROTON-930 > add explicit AMQP 1.0 constants > --- > > Key: PROTON-930 > URL: https://issues.apache.org/jira/browse/PROTON-930 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Reporter: michael goulish >Assignee: michael goulish >Priority: Minor > Fix For: 0.10 > > > Add an include file that has explicit defined constants for every numeric > default value that is mandated by the AMQP 1.0 spec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-925) proton-c seems to treat unspecified channel-max as implying 0
[
https://issues.apache.org/jira/browse/PROTON-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615189#comment-14615189
]
ASF subversion and git services commented on PROTON-925:
Commit 3c14a0d1cf89b988529b98dc10c8f2c17abcdcf3 in qpid-proton's branch
refs/heads/master from mgoulish
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=3c14a0d ]
PROTON-925: use of UINT32_MAX is breaking some builds. Getting better
constants will be part of PROTON-930
> proton-c seems to treat unspecified channel-max as implying 0
> -
>
> Key: PROTON-925
> URL: https://issues.apache.org/jira/browse/PROTON-925
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.10
>Reporter: Gordon Sim
>Assignee: michael goulish
>Priority: Blocker
> Fix For: 0.10
>
>
> If max-channels is not specified in the open, it appears the latest proton-c
> treats that as implying the maximum is 0 though the spec states the default
> is 65535.
> This breaks compatibility with previous proton releases. E.g. the following
> is the interaction between a sender using the latest 0.10 and a receiver
> using proton 0.9.
> {noformat}
> [0x151c710]: <- AMQP
> [0x151c710]:0 <- @open(16)
> [container-id="65A6602D-5D24-4D39-9C6F-7403D98F5E15", hostname="localhost",
> channel-max=32767]
> [0x151c710]:0 <- @begin(17) [next-outgoing-id=0, incoming-window=2147483647,
> outgoing-window=1]
> [0x151c710]:1 <- @begin(17) [next-outgoing-id=0, incoming-window=2147483647,
> outgoing-window=1]
> [0x151c710]:2 <- @begin(17) [next-outgoing-id=0, incoming-window=2147483647,
> outgoing-window=1]
> [0x151c710]:0 <- @attach(18) [name="sender-xxx", handle=0, role=false,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_a",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_a",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]:1 <- @attach(18) [name="sender-xxx", handle=0, role=false,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_b",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_b",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]:2 <- @attach(18) [name="sender-xxx", handle=0, role=false,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_c",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_c",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]: -> AMQP
> [0x151c710]:0 -> @open(16)
> [container-id="abab56b0-c25e-427b-9f4f-d63da48d1973"]
> [0x151c710]:0 -> @begin(17) [remote-channel=0, next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=0]
> [0x151c710]:1 -> @begin(17) [remote-channel=1, next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=0]
> [0x151c710]:2 -> @begin(17) [remote-channel=2, next-outgoing-id=0,
> incoming-window=2147483647, outgoing-window=0]
> [0x151c710]:0 -> @attach(18) [name="sender-xxx", handle=0, role=true,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_a",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_a",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]:1 -> @attach(18) [name="sender-xxx", handle=0, role=true,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_b",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_b",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]:2 -> @attach(18) [name="sender-xxx", handle=0, role=true,
> snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue_c",
> durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue_c",
> durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
> [0x151c710]:0 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647,
> next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0,
> link-credit=341, drain=false]
> [0x151c710]:1 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647,
> next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0,
> link-credit=341, drain=false]
> [0x151c710]:2 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647,
> next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0,
> link-credit=341, drain=false]
> [0x151c710]:0 <- @close(24) [error=@error(29)
> [condition=:"amqp:connection:framing-error", description="remote channel 1 is
> above negotiated channel_max 0."]]
> [0x151c710]: <- EOS
> [0x151c710]:0 -> @close(24) []
> [0x151c710]: -> EOS
> {noformat}
--
This message was sent by Atlassian JIRA
[GitHub] qpid-proton pull request: Fix 2 Code Analysis warnings and a reall...
Github user ted-ross commented on the pull request: https://github.com/apache/qpid-proton/pull/39#issuecomment-118902728 I'd like to see this pull request move forward. On Dan's points above: 1) I agree with Andrew that using __LINE__ as an error code should be changed to simply returning an eror code (or a null result, whichever is appropriate). 2) I agree with Andrew here as well. I think it's an acceptable pattern to check inputs and exit early if there is an error condition. 3) I agree with Dan that the library should simply report the failure to allocate and not take any action on behalf of the process using it. There are a number of cases where Proton makes relatively large allocations the failure of which does not necessarily mean that the process can't recover. Taking a piecemeal approach to cleaning up all the ignored allocations is probably a good idea. If there are any API changes that will result, we should try to get those all into a single release. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: ProtonJ compilation and test failures
On 07/06/2015 04:08 PM, Rafael Schloming wrote: Any sort of missing class really should be a compile time exception, which I think means you must have stale class files *somewhere*. You could try doing a find checkout -name "*.class" just as a sanity check. I have deleted all the .class files that are generated in the source tree (and deleted the entire build directory). The class files are rebuilt for ProtonJInterop, alongside those for InteropTest and JythonTest in ./tests/target/test-classes/org/apache/qpid/proton/: ProtonJInterop$1.class ProtonJInterop.class ProtonJInterop$Recv.class ProtonJInterop$Send.class ProtonJInterop$SendHandler.class However the test run still reports that it cannot load these. Also, it's possible something in your local maven repo is somehow coming into play, maybe blow that away and rebuild it and/or do an mvn install to be sure that remove dependencies aren't out of sync with local code? I removed everything I could find that was proton related from the mvn repository, but that didn't help.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On Mon, 2015-07-06 at 11:30 -0400, Rafael Schloming wrote: > I wired in allowSkip in a very minimal way just to restore the ability to > force the old behaviour. It would be a fairly trivial to change the name of > course, I'm not sure if that really applies as the method is on a different object now (transport instead of SASL). > however it appears there are a bunch of other related changes that > go along with it, e.g. adding a bunch of accessors and fixing the error > behavior. Currently if you put in require authentication the java sasl > layer will simply die with a TransportException if it sees a regular AMQP > header, and the tests appear to expect something more graceful. Yes you will need the isAuthenticated property. Hmm, not sure about that, I think the tests just ensure that authenticated is not true, although I suppose they do rely on not getting an exception, that may be incidental behaviour though. The new APIs aren't very extensive, and should be fairly extensively documented at https://cwiki.apache.org/confluence/x/B5cWAw (this should be up to date). Bringing Proton-J in line with the new SASL APIs is https://issues.apache.org/jira/browse/PROTON-910 Andrew
Re: ProtonJ compilation and test failures
On Mon, 2015-07-06 at 16:48 +0100, Gordon Sim wrote: > On 07/06/2015 04:08 PM, Rafael Schloming wrote: > > Any sort of missing class really should be a compile time > > exception, which > > I think means you must have stale class files *somewhere*. You > > could try > > doing a find checkout -name "*.class" just as a sanity check. > > I have deleted all the .class files that are generated in the source > tree (and deleted the entire build directory). > > The class files are rebuilt for ProtonJInterop, alongside those for > InteropTest and JythonTest in > ./tests/target/test-classes/org/apache/qpid/proton/: > > ProtonJInterop$1.class > ProtonJInterop.class > ProtonJInterop$Recv.class > ProtonJInterop$Send.class > ProtonJInterop$SendHandler.class > > However the test run still reports that it cannot load these. > > > Also, it's > > possible something in your local maven repo is somehow coming into > > play, > > maybe blow that away and rebuild it and/or do an mvn install to be > > sure > > that remove dependencies aren't out of sync with local code? > > I removed everything I could find that was proton related from the > mvn > repository, but that didn't help. > Have you tried `rm -rf $HOME/.m2`? Maven stuffs it with class files from who-knows-where which can lurk for months or years before screwing up builds in a brand-new checkout. I hate maven. (and cmake, and automake. I'm not a bigot but you just can't trust build systems I'm telling you!)
Re: ProtonJ compilation and test failures
On 07/06/2015 05:22 PM, aconway wrote: On Mon, 2015-07-06 at 16:48 +0100, Gordon Sim wrote: On 07/06/2015 04:08 PM, Rafael Schloming wrote: Any sort of missing class really should be a compile time exception, which I think means you must have stale class files *somewhere*. You could try doing a find checkout -name "*.class" just as a sanity check. I have deleted all the .class files that are generated in the source tree (and deleted the entire build directory). The class files are rebuilt for ProtonJInterop, alongside those for InteropTest and JythonTest in ./tests/target/test-classes/org/apache/qpid/proton/: ProtonJInterop$1.class ProtonJInterop.class ProtonJInterop$Recv.class ProtonJInterop$Send.class ProtonJInterop$SendHandler.class However the test run still reports that it cannot load these. Also, it's possible something in your local maven repo is somehow coming into play, maybe blow that away and rebuild it and/or do an mvn install to be sure that remove dependencies aren't out of sync with local code? I removed everything I could find that was proton related from the mvn repository, but that didn't help. Have you tried `rm -rf $HOME/.m2`? I haven't. I was hoping to avoid that as then next time you invoke it, maven has to download half the interwebs. Maven stuffs it with class files from who-knows-where which can lurk for months or years before screwing up builds in a brand-new checkout. I hate maven. (and cmake, and automake. I'm not a bigot but you just can't trust build systems I'm telling you!) I don't like maven either, but if the build is able to get messed up by it in a way that mvn clean doesn't fix, then to me that suggests the build is a little broken somehow.
Schedule for the 0.10 release?
Hi all, I remember recently that there was a rumor that we may be able to start the release of 0.10 at the end of june. Now that my july 4th long weekend has worn off - what's left to be done before we can cut an alpha? -- -K
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On 6 July 2015 at 16:51, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 11:30 -0400, Rafael Schloming wrote: >> I wired in allowSkip in a very minimal way just to restore the ability to >> force the old behaviour. It would be a fairly trivial to change the name of >> course, > > I'm not sure if that really applies as the method is on a different > object now (transport instead of SASL). > >> however it appears there are a bunch of other related changes that >> go along with it, e.g. adding a bunch of accessors and fixing the error >> behavior. Currently if you put in require authentication the java sasl >> layer will simply die with a TransportException if it sees a regular AMQP >> header, and the tests appear to expect something more graceful. > > Yes you will need the isAuthenticated property. > The old toggle only used to define whether sasl was required or not (which it historically was once you enabled the sasl layer, and the toggle was never implemented in proton-j), whereas IIRC the new 'requireAuth' governs that but also whether ANONYMOUS is allowed or not when a SASL layer is used, is that correct? Given the older SASL API means much of the SASL work including mechansim selection happens outside/before proton-j and you get as far as making a connection, isnt that going to be a bit of a pain to implement and use? Might it be simpler to use allowSkip() until such time the wider SASL API is actually implemented, keep the default behaviour as it was previously (so people will be less likely to need to use the behaviour toggle and less likely to care what it is called), and update whichever reactor tests that needed the change to configure it that way in the meantime? > Hmm, not sure about that, I think the tests just ensure that > authenticated is not true, although I suppose they do rely on not > getting an exception, that may be incidental behaviour though. > > The new APIs aren't very extensive, and should be fairly extensively > documented at https://cwiki.apache.org/confluence/x/B5cWAw > (this should be up to date). > > Bringing Proton-J in line with the new SASL APIs is > https://issues.apache.org/jira/browse/PROTON-910 > > Andrew > >
Re: Schedule for the 0.10 release?
On 06/07/15 12:35 -0400, Ken Giusti wrote: Hi all, I remember recently that there was a rumor that we may be able to start the release of 0.10 at the end of june. Now that my july 4th long weekend has worn off - what's left to be done before we can cut an alpha? Yup, I shot a request and there seemed to be consensus. I'd really love to see the 0.10 release out this week. Thanks, Flavio -- @flaper87 Flavio Percoco pgp6Ptdj89IBU.pgp Description: PGP signature
Re: ProtonJ compilation and test failures
On 6 July 2015 at 16:48, Gordon Sim wrote: > On 07/06/2015 04:08 PM, Rafael Schloming wrote: >> >> Any sort of missing class really should be a compile time exception, which >> I think means you must have stale class files *somewhere*. You could try >> doing a find checkout -name "*.class" just as a sanity check. > > > I have deleted all the .class files that are generated in the source tree > (and deleted the entire build directory). > > The class files are rebuilt for ProtonJInterop, alongside those for > InteropTest and JythonTest in > ./tests/target/test-classes/org/apache/qpid/proton/: > > ProtonJInterop$1.class > ProtonJInterop.class > ProtonJInterop$Recv.class > ProtonJInterop$Send.class > ProtonJInterop$SendHandler.class > > However the test run still reports that it cannot load these. > Do you have anything interesting in your build dir structure, like spaces etc, that we might not? >> Also, it's >> possible something in your local maven repo is somehow coming into play, >> maybe blow that away and rebuild it and/or do an mvn install to be sure >> that remove dependencies aren't out of sync with local code? > > > I removed everything I could find that was proton related from the mvn > repository, but that didn't help. > If you are running maven from the root of the proton tree to build and test the modules then its building everything proton-related there is and shouldn't look in its local repo for those as a result. Each modules classes are loaded using the class files in the target dirs when testing, again not the local repo. If you build certain modules in isoluation of its dependencies, then it will look in the local repo (and/or remote repos, based on update schedules and last check times) to get any bits that it needs which aren't already part of the current build, in which case yes you would want to 'mvn install' those bits to the local repo if you have local changes, or allow them to be updated from remote repositories if you dont (again, based on schedule..use -U to force repo update check) or for example if you are using mismatched local and remote snapshot bits. Robbie
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: > ... > The old toggle only used to define whether sasl was required or not > (which it historically was once you enabled the sasl layer, and the > toggle was never implemented in proton-j), whereas IIRC the new > 'requireAuth' governs that but also whether ANONYMOUS is allowed or > not when a SASL layer is used, is that correct? That is true, but I think it actually more useful to be able to select authenticated or not compared to using SASL or not (because ANONYMOUS is unauthenticated but uses SASL). The C implementation does the actual enforcement when it reads the AMQP header, which would obviously be a significant change to the Java implementation, but I really do think gives a more satisfactory user result. > Given the older SASL > API means much of the SASL work including mechansim selection happens > outside/before proton-j and you get as far as making a connection, > isnt that going to be a bit of a pain to implement and use? Well I obviously disagree, else I would have implemented the C SASL support like this! TBH, I think that using the same API in Java would make sense along with an extra "plugin API" to allow implementing mechanisms (assuming there is no external Java library like Cyrus SASL that can do the work for us). But yes that is new work, probably not going to happen before Proton 0.11. > > Might it be simpler to use allowSkip() until such time the wider SASL > API is actually implemented, keep the default behaviour as it was > previously (so people will be less likely to need to use the behaviour > toggle and less likely to care what it is called), and update > whichever reactor tests that needed the change to configure it that > way in the meantime? I'm agnostic on this point really, I'd just like to ensure that the C and Java APIs actually converge in the shortest time scale. The alternative would be to stop testing them with the same test code - there are already far too many specific tests for Java or Windows in the tests. Andrew
Re: ProtonJ compilation and test failures
On Mon, 2015-07-06 at 17:31 +0100, Gordon Sim wrote: > On 07/06/2015 05:22 PM, aconway wrote: > > On Mon, 2015-07-06 at 16:48 +0100, Gordon Sim wrote: > > > On 07/06/2015 04:08 PM, Rafael Schloming wrote: > > > > Any sort of missing class really should be a compile time > > > > exception, which > > > > I think means you must have stale class files *somewhere*. You > > > > could try > > > > doing a find checkout -name "*.class" just as a sanity check. Not maven's fault (I still hate maven) missing CLASSPATH in cmake config. Fixed. 89fca58 NO-JIRA: Add missing CLASSPATH needed to run python tests in proton-c/CMakeLists.txt We could improve the setup by moving all the config.sh paths into cmake variables and using them consistently in cmake and to generate config.sh. Also the fact that tests in proton/test are driven from proton/proton -c/CMakeLists.txt is odd. If the tests live at top level (which is OK by me) then the test drivers should too. Especially when (as in this case) they are pulling code from both proton-c and proton-j. I'm not doing that right now because it requires thought, maybe later if nobody else does it.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On Mon, 2015-07-06 at 13:14 -0400, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: > > ... > > The old toggle only used to define whether sasl was required or not > > (which it historically was once you enabled the sasl layer, and the > > toggle was never implemented in proton-j), whereas IIRC the new > > 'requireAuth' governs that but also whether ANONYMOUS is allowed or > > not when a SASL layer is used, is that correct? > > That is true, but I think it actually more useful to be able to select > authenticated or not compared to using SASL or not (because ANONYMOUS is > unauthenticated but uses SASL). > > The C implementation does the actual enforcement when it reads the AMQP > header, which would obviously be a significant change to the Java > implementation, but I really do think gives a more satisfactory user > result. The reason for the complexity and the checking at AMQP header time is to allow SSL certificates as a valid form of authentication (not necessarily only used with SASL EXTERNAL). If you don't need to support that (or at least not yet) then "require authentication" can simply mean require the SASL layer but don't offer the ANONYMOUS mechanism. That is what earlier versions of the C code did*, and I think that would be relatively simple to implement in Java too. * The C code will still not offer ANONYMOUS as a possible mechanism if authentication is required. But the overall meaning of the flag is more complex than this as explained. Andrew
Re: ProtonJ compilation and test failures
On 6 July 2015 at 18:24, aconway wrote: > On Mon, 2015-07-06 at 17:31 +0100, Gordon Sim wrote: >> On 07/06/2015 05:22 PM, aconway wrote: >> > On Mon, 2015-07-06 at 16:48 +0100, Gordon Sim wrote: >> > > On 07/06/2015 04:08 PM, Rafael Schloming wrote: >> > > > Any sort of missing class really should be a compile time >> > > > exception, which >> > > > I think means you must have stale class files *somewhere*. You >> > > > could try >> > > > doing a find checkout -name "*.class" just as a sanity check. > > Not maven's fault > (I still hate maven) Poor maven, just can't find love from aconway ;) > missing CLASSPATH in cmake > config. Fixed. > > 89fca58 NO-JIRA: Add missing CLASSPATH needed to run python tests in > proton-c/CMakeLists.txt > > We could improve the setup by moving all the config.sh paths into cmake > variables and using them consistently in cmake and to generate > config.sh. > > Also the fact that tests in proton/test are driven from proton/proton > -c/CMakeLists.txt is odd. If the tests live at top level (which is OK > by me) then the test drivers should too. Especially when (as in this > case) they are pulling code from both proton-c and proton-j. > > I'm not doing that right now because it requires thought, maybe later > if nobody else does it.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
FWIW, my changes in this area really represent the minimum diff necessary to get the reactor branch to land. None of this is related to the reactor changes per se, it just so happens the reactor tests include several tests that check interop between proton-c and proton-j and these tests keep stumbling over incompatibilities that are currently quite easy to arise given the current state of the sasl implementations. While I agree 100% that the APIs should converge, at the moment I'm actually slightly more worried about the on-the-wire interop issues. More specifically, while it's bad for proton-c and proton-j APIs to look different, it's *really* bad if the default settings for one result in a configuration that won't interop with the default settings for the other. --Rafael On Mon, Jul 6, 2015 at 10:28 AM, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 13:14 -0400, Andrew Stitcher wrote: > > On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: > > > ... > > > The old toggle only used to define whether sasl was required or not > > > (which it historically was once you enabled the sasl layer, and the > > > toggle was never implemented in proton-j), whereas IIRC the new > > > 'requireAuth' governs that but also whether ANONYMOUS is allowed or > > > not when a SASL layer is used, is that correct? > > > > That is true, but I think it actually more useful to be able to select > > authenticated or not compared to using SASL or not (because ANONYMOUS is > > unauthenticated but uses SASL). > > > > The C implementation does the actual enforcement when it reads the AMQP > > header, which would obviously be a significant change to the Java > > implementation, but I really do think gives a more satisfactory user > > result. > > The reason for the complexity and the checking at AMQP header time is to > allow SSL certificates as a valid form of authentication (not > necessarily only used with SASL EXTERNAL). If you don't need to support > that (or at least not yet) then "require authentication" can simply mean > require the SASL layer but don't offer the ANONYMOUS mechanism. That is > what earlier versions of the C code did*, and I think that would be > relatively simple to implement in Java too. > > * The C code will still not offer ANONYMOUS as a possible mechanism if > authentication is required. But the overall meaning of the flag is more > complex than this as explained. > > Andrew > > >
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On 6 July 2015 at 18:14, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: >> ... >> The old toggle only used to define whether sasl was required or not >> (which it historically was once you enabled the sasl layer, and the >> toggle was never implemented in proton-j), whereas IIRC the new >> 'requireAuth' governs that but also whether ANONYMOUS is allowed or >> not when a SASL layer is used, is that correct? > > That is true, but I think it actually more useful to be able to select > authenticated or not compared to using SASL or not (because ANONYMOUS is > unauthenticated but uses SASL). > I wasn't arguing one way or another what is better here, I was just pointing out the difference given its important to this change-right-before-release and that proton-j doesn't have an updated SASL API/Impl to go along with it. > The C implementation does the actual enforcement when it reads the AMQP > header, which would obviously be a significant change to the Java > implementation, but I really do think gives a more satisfactory user > result. > >> Given the older SASL >> API means much of the SASL work including mechansim selection happens >> outside/before proton-j and you get as far as making a connection, >> isnt that going to be a bit of a pain to implement and use? > > Well I obviously disagree, else I would have implemented the C SASL > support like this! If it wasnt clear above, I only meant it might be pain to implement and use isAuthenticated right now *given the older SASL API/Impl* that currently still exists in proton-j and presumably isnt going away before the next release. > > TBH, I think that using the same API in Java would make sense along with > an extra "plugin API" to allow implementing mechanisms (assuming there > is no external Java library like Cyrus SASL that can do the work for > us). > > But yes that is new work, probably not going to happen before Proton > 0.11. >> > >> Might it be simpler to use allowSkip() until such time the wider SASL >> API is actually implemented, keep the default behaviour as it was >> previously (so people will be less likely to need to use the behaviour >> toggle and less likely to care what it is called), and update >> whichever reactor tests that needed the change to configure it that >> way in the meantime? > > I'm agnostic on this point really, I'd just like to ensure that the C > and Java APIs actually converge in the shortest time scale. My comment above was coming from an impression that the change here wouldn't be close to fully aligning them, meaning further change and probable breakage would be required down the line. If thats the case I would tend towards leaving the existing default behaviour until such time a viable attempt at converging them is actually being made, in which case such change and breakage would be better warranted and rewarded. > The > alternative would be to stop testing them with the same test code - > there are already far too many specific tests for Java or Windows in the > tests. > >From my impression jsut as many of the specific tests are C or Linux specific given they often [now] test things that simply haven't been implemented elsewhere yet :) I'll admit I have writetn some Java specific tests in actual Java on occasionsometimes because I burned a lot of time first trying to write it in python and found the test shim stuff entirely masked the problem so I actually couldn't, and others just because it was a known handful of minutes as a direct pure unit test versus indeterminate time working out how the to do it far less directly via the python tests that I simply justify at the time.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On 6 July 2015 at 18:28, Andrew Stitcher wrote: > On Mon, 2015-07-06 at 13:14 -0400, Andrew Stitcher wrote: >> On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: >> > ... >> > The old toggle only used to define whether sasl was required or not >> > (which it historically was once you enabled the sasl layer, and the >> > toggle was never implemented in proton-j), whereas IIRC the new >> > 'requireAuth' governs that but also whether ANONYMOUS is allowed or >> > not when a SASL layer is used, is that correct? >> >> That is true, but I think it actually more useful to be able to select >> authenticated or not compared to using SASL or not (because ANONYMOUS is >> unauthenticated but uses SASL). >> >> The C implementation does the actual enforcement when it reads the AMQP >> header, which would obviously be a significant change to the Java >> implementation, but I really do think gives a more satisfactory user >> result. > > The reason for the complexity and the checking at AMQP header time is to > allow SSL certificates as a valid form of authentication (not > necessarily only used with SASL EXTERNAL). If you don't need to support > that (or at least not yet) then "require authentication" can simply mean > require the SASL layer but don't offer the ANONYMOUS mechanism. That is > what earlier versions of the C code did*, and I think that would be > relatively simple to implement in Java too. > > * The C code will still not offer ANONYMOUS as a possible mechanism if > authentication is required. But the overall meaning of the flag is more > complex than this as explained. > > Andrew > Yeah, I appreciated that, I just didn't think it necessary detail for the fairly basic point I was trying to make. As mentioned in my last mail, I was really just pointing out the mismatch between the existing behaviour and what requireAuthentication and isAuthenitcated do versus the lack of the whole new SASL/auth API/impl in proton-j at this point in time, possibly days before release, which seems to be needed in order to make them actually work the way they do in proton-c. If all we do now is change the default behaviour a little now in a way that requries most people to update their existing usage, knowing we will change the same bits again in bigger way that will require most people to update their already-updated usage again, then I'd prefer we didnt change the default behaviour now as the near term reward doesnt seem that great for all the ongoing hassle from the repeated changes, thats all I was meaning.
Re: Schedule for the 0.10 release?
+1 The only issue that has me worried here is the sasl interop story between proton-c and proton-j. I can cut a release later today just to give us something to poke at, but there may still be sasl work needed. --Rafael On Mon, Jul 6, 2015 at 9:57 AM, Flavio Percoco wrote: > On 06/07/15 12:35 -0400, Ken Giusti wrote: > >> Hi all, >> >> I remember recently that there was a rumor that we may be able to start >> the release of 0.10 at the end of june. >> >> Now that my july 4th long weekend has worn off - what's left to be done >> before we can cut an alpha? >> > > Yup, I shot a request and there seemed to be consensus. I'd really > love to see the 0.10 release out this week. > > Thanks, > Flavio > > -- > @flaper87 > Flavio Percoco >
[jira] [Created] (PROTON-933) Cyrus SASL GSSAPI plugin can error if sent long buffers.
Andrew Stitcher created PROTON-933:
--
Summary: Cyrus SASL GSSAPI plugin can error if sent long buffers.
Key: PROTON-933
URL: https://issues.apache.org/jira/browse/PROTON-933
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: 0.10
Reporter: Andrew Stitcher
Assignee: Andrew Stitcher
Priority: Blocker
Cyrus SASL GSSAPI is not able to handle buffers in the advertised way -
Even if you limit the amount of data to decode on the server side to the
SASL_MAXOUTBUF value that it reports it can fail with a syslog of (for example):
{quote}
Jul 6 13:32:38 proton msgr-recv: encoded packet size too big (32828 > 32768)
{quote}
It appears that the decode routine can coalesce received buffers, and in fact
its actual requirement is that you may not _encode_ on the _client_ any more
than the value returned on the server. However the value of SASL_MAXOUTBUF that
is returned on the client is 60 bytes bigger than the value returned on the
server.
The workaround is to subract 60 bytes from the value on the returned on the
client before using it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-933) Cyrus SASL GSSAPI plugin can error if sent long buffers.
[
https://issues.apache.org/jira/browse/PROTON-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615592#comment-14615592
]
ASF subversion and git services commented on PROTON-933:
Commit c6499b687cce4102655db2d7012bc7cbc9bcd1fd in qpid-proton's branch
refs/heads/master from [~astitcher]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=c6499b6 ]
PROTON-933: Workaround for seeming bug in Cyrus SASL GSSAPI buffer size
reporting
> Cyrus SASL GSSAPI plugin can error if sent long buffers.
>
>
> Key: PROTON-933
> URL: https://issues.apache.org/jira/browse/PROTON-933
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.10
>Reporter: Andrew Stitcher
>Assignee: Andrew Stitcher
>Priority: Blocker
>
> Cyrus SASL GSSAPI is not able to handle buffers in the advertised way -
> Even if you limit the amount of data to decode on the server side to the
> SASL_MAXOUTBUF value that it reports it can fail with a syslog of (for
> example):
> {quote}
> Jul 6 13:32:38 proton msgr-recv: encoded packet size too big (32828 > 32768)
> {quote}
> It appears that the decode routine can coalesce received buffers, and in fact
> its actual requirement is that you may not _encode_ on the _client_ any more
> than the value returned on the server. However the value of SASL_MAXOUTBUF
> that is returned on the client is 60 bytes bigger than the value returned on
> the server.
> The workaround is to subract 60 bytes from the value on the returned on the
> client before using it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PROTON-933) Cyrus SASL GSSAPI plugin can error if sent long buffers.
[
https://issues.apache.org/jira/browse/PROTON-933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Stitcher resolved PROTON-933.
Resolution: Fixed
Fix Version/s: 0.10
> Cyrus SASL GSSAPI plugin can error if sent long buffers.
>
>
> Key: PROTON-933
> URL: https://issues.apache.org/jira/browse/PROTON-933
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.10
>Reporter: Andrew Stitcher
>Assignee: Andrew Stitcher
>Priority: Blocker
> Fix For: 0.10
>
>
> Cyrus SASL GSSAPI is not able to handle buffers in the advertised way -
> Even if you limit the amount of data to decode on the server side to the
> SASL_MAXOUTBUF value that it reports it can fail with a syslog of (for
> example):
> {quote}
> Jul 6 13:32:38 proton msgr-recv: encoded packet size too big (32828 > 32768)
> {quote}
> It appears that the decode routine can coalesce received buffers, and in fact
> its actual requirement is that you may not _encode_ on the _client_ any more
> than the value returned on the server. However the value of SASL_MAXOUTBUF
> that is returned on the client is 60 bytes bigger than the value returned on
> the server.
> The workaround is to subract 60 bytes from the value on the returned on the
> client before using it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-934) Build tests fail if Java is not available
Andrew Stitcher created PROTON-934: -- Summary: Build tests fail if Java is not available Key: PROTON-934 URL: https://issues.apache.org/jira/browse/PROTON-934 Project: Qpid Proton Issue Type: Bug Components: proton-c, proton-j Affects Versions: 0.10 Reporter: Andrew Stitcher Assignee: Rafael H. Schloming Priority: Blocker Due to the addition of new Proton-J interop tests. Build tests that do not have Java available will just fail, because the tests just assume that Java is available. Various Windows builds do not build the Java code, and this just makes them fail and timeout. Failing wouldn't be quite so bad, but hanging making the tests timeout means that subsequent tests are just not run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-201) Provide a C++ Messenger and Message class
[ https://issues.apache.org/jira/browse/PROTON-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Stitcher updated PROTON-201: --- Assignee: Cliff Jansen (was: Andrew Stitcher) > Provide a C++ Messenger and Message class > - > > Key: PROTON-201 > URL: https://issues.apache.org/jira/browse/PROTON-201 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Reporter: Darryl L. Pierce >Assignee: Cliff Jansen > > Provide wrapper classes for these two types similar to what's available in > Perl and Ruby. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-201) Provide a C++ Messenger and Message class
[ https://issues.apache.org/jira/browse/PROTON-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615620#comment-14615620 ] Andrew Stitcher commented on PROTON-201: [~cliffjansen], I believe this is included in your C++ API work - if not just resolve this as won't implement. > Provide a C++ Messenger and Message class > - > > Key: PROTON-201 > URL: https://issues.apache.org/jira/browse/PROTON-201 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Reporter: Darryl L. Pierce >Assignee: Cliff Jansen > > Provide wrapper classes for these two types similar to what's available in > Perl and Ruby. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Schedule for the 0.10 release?
On Mon, 2015-07-06 at 11:28 -0700, Rafael Schloming wrote: > +1 > > The only issue that has me worried here is the sasl interop story between > proton-c and proton-j. I can cut a release later today just to give us > something to poke at, but there may still be sasl work needed. I have a blocking fix to push[1], but I can't push to the repo. So hold off until I can put it back. I'll try to make it a priority for infra. Andrew [1] Currently GSSAPI support (Kerberos really) is broken seemingly due to a bug in Cyrus SASL. The patch is a workaround.
Re: [38/38] qpid-proton git commit: implemented sasl sniffing for proton-j; this allows the reactor interop tests to pass
On 6 July 2015 at 18:57, Rafael Schloming wrote: > FWIW, my changes in this area really represent the minimum diff necessary > to get the reactor branch to land. None of this is related to the reactor > changes per se, Yes, it almost seems like a change in default behaviour of a sasl enabled server to allow non-SASL connections should have a JIRA of its own :P > it just so happens the reactor tests include several tests > that check interop between proton-c and proton-j and these tests keep > stumbling over incompatibilities that are currently quite easy to arise > given the current state of the sasl implementations. > > While I agree 100% that the APIs should converge, at the moment I'm > actually slightly more worried about the on-the-wire interop issues. More > specifically, while it's bad for proton-c and proton-j APIs to look > different, it's *really* bad if the default settings for one result in a > configuration that won't interop with the default settings for the other. > > --Rafael I can agree with that. Although I do feel the defaults for both engines (independent from the defaults of any of our other code that uses them, which can do anything it likes) are now wrong as a result of this latest change ;) > > On Mon, Jul 6, 2015 at 10:28 AM, Andrew Stitcher > wrote: > >> On Mon, 2015-07-06 at 13:14 -0400, Andrew Stitcher wrote: >> > On Mon, 2015-07-06 at 17:48 +0100, Robbie Gemmell wrote: >> > > ... >> > > The old toggle only used to define whether sasl was required or not >> > > (which it historically was once you enabled the sasl layer, and the >> > > toggle was never implemented in proton-j), whereas IIRC the new >> > > 'requireAuth' governs that but also whether ANONYMOUS is allowed or >> > > not when a SASL layer is used, is that correct? >> > >> > That is true, but I think it actually more useful to be able to select >> > authenticated or not compared to using SASL or not (because ANONYMOUS is >> > unauthenticated but uses SASL). >> > >> > The C implementation does the actual enforcement when it reads the AMQP >> > header, which would obviously be a significant change to the Java >> > implementation, but I really do think gives a more satisfactory user >> > result. >> >> The reason for the complexity and the checking at AMQP header time is to >> allow SSL certificates as a valid form of authentication (not >> necessarily only used with SASL EXTERNAL). If you don't need to support >> that (or at least not yet) then "require authentication" can simply mean >> require the SASL layer but don't offer the ANONYMOUS mechanism. That is >> what earlier versions of the C code did*, and I think that would be >> relatively simple to implement in Java too. >> >> * The C code will still not offer ANONYMOUS as a possible mechanism if >> authentication is required. But the overall meaning of the flag is more >> complex than this as explained. >> >> Andrew >> >> >>
Re: Schedule for the 0.10 release?
I'd like to see something in the release to do with the session outgoing-window problems that mean the new JMS client can't currently send to ServiceBus. As mentioend elsewhere earlier, a very basic change that leaves the current default behaviour as it stands but would enable me to configure the window to a specific value (as the old JMS client uses) would be something like this: https://github.com/gemmellr/qpid-proton/commit/b2b15a617b42bd36eacf9b51fb146223ba516496 Robbie On 6 July 2015 at 19:28, Rafael Schloming wrote: > +1 > > The only issue that has me worried here is the sasl interop story between > proton-c and proton-j. I can cut a release later today just to give us > something to poke at, but there may still be sasl work needed. > > --Rafael > > > On Mon, Jul 6, 2015 at 9:57 AM, Flavio Percoco wrote: > >> On 06/07/15 12:35 -0400, Ken Giusti wrote: >> >>> Hi all, >>> >>> I remember recently that there was a rumor that we may be able to start >>> the release of 0.10 at the end of june. >>> >>> Now that my july 4th long weekend has worn off - what's left to be done >>> before we can cut an alpha? >>> >> >> Yup, I shot a request and there seemed to be consensus. I'd really >> love to see the 0.10 release out this week. >> >> Thanks, >> Flavio >> >> -- >> @flaper87 >> Flavio Percoco >>
0.10 alpha1
As promised, here is the first alpha for 0.10. It's posted in the usual places: Source code is here: http://people.apache.org/~rhs/qpid-proton-0.10-alpha1/ Java binaries are here: https://repository.apache.org/content/repositories/orgapacheqpid-1036 Please check it out and follow up with any issues. --Rafael
Re: AMQP 1.0 session outgoing-window usage / meaning
IIRC, the definition of the outgoing-window was largely motivated by the need to express to receivers certain conditions under which they may be required to settle deliveries in order to receive more. For example if an implementation uses a fixed sized array to store deliveries, and this array is keyed by the offset of the delivery-id from the smallest unsettled delivery, then although the sender may have sufficient credit to send more transfers, it may not actually be capable of doing this because the next delivery-id would land outside the range of deliveries that are currently represented within its fixed size array. This could happen for example if the receiver issues N credits (where N is the size of the sender's fixed array) and settles deliveries 2 through N. The sender is then stuck with an unsettled delivery in the first slot of its fixed sized array and cannot send another delivery until that first delivery is settled. Given this, it's certainly true an outgoing-window of 0 is kind of strange and useless. It's probably also true that it is never super useful for the incoming window of the receiver to be larger than the outgoing window of the sender (or vice versa) since one can't ever exceed the other, so I'd say your largest-possible-int default and max-frame-like treatment are fairly appropriate. --Rafael On Fri, Jul 3, 2015 at 8:57 AM, Robbie Gemmell wrote: > Rob, Rafi, as authors of the spec and the related code in proton, do > you have any thoughts to add here? > > Barring any discussion otherwise I will be looking to change proton to > at least optionally allow controlling the outgoing window along the > lines I mentioned near the end of my original mail. > > Robbie > > On 2 July 2015 at 00:15, Robbie Gemmell wrote: > > Thanks James. Some expansion which may be useful to add. > > > > When comparing the older JMS client, proton-c via the Messenger API, > > and the new JMS client using proton-j, its important to note that they > > aren't all doing the same thing even where their underlying > > implementations do seem to share the same behaviour in the cases of > > proton-c and proton-j. > > > > The older JMS client initializes its outgoing window to a fixed number > > in the session Begin frame and then doesnt seem to ever change it for > > subsequent Flow frames, and simply manages whether its session can > > later send transfer frames based on the current value of the remote > > incoming window. Proton-J and Proton-C similarly only base their > > session level decision to send transfers on the remote incoming window > > and not their own outgoing window (which as noted below means they > > violate their advertised outgoing window, which is often going to be > > 0). > > > > Proton-C and Proton-J both currently look to set the outgoing window > > at any given time to a calculated value based on either the number of > > buffered messages or the buffered bytes divided by frame size. If > > there are no buffered messages at the point the Begin and Flow frames > > are generated, then the outgoing-window will be set to 0. This appears > > to function the same for both proton-c and proton-j. A key point > > though is that I think much of the historic usage of proton-c against > > Service Bus has been via the Messenger API, which works somewhat > > differently than many others in that it looks to create a session and > > a sender and sends the messages in one pipelined sequence of transport > > output, which means that by the point the Begin frame actually gets > > generated there are indeed buffered messages to send which means the > > outgoing-window is initialised to a value greater than zero. Other > > APIs which create the session as a distinct step thus wont ever have > > buffered messages when the Begin frame gets created and so the > > outgoing-window is initialised to 0, which is the behaviour observed > > with the new JMS client using proton-j and also what I saw when trying > > proton-c via the Qpid Messaging C++ client (against qpidd). > > > > Robbie > > > > > > On 1 July 2015 at 20:54, James Birdsall wrote: > >> FYI, I have forwarded this and important bits of the preceding > discussion to our AMQP stack dev within the ServiceBus team. > >> > >> Both the Qpid JMS AMQP 1.0 "legacy" client and Proton-C have been > working fine with Azure SB for years now. Proton-J, however, is not > something we have explored previously, and obviously there is something > different about its behavior compared to the other clients. > >> > >> The Qpid JMS client is our recommended JMS client for interop with > ServiceBus, and we would like to keep up with the times and not have to > direct customers to the legacy client, so we are very interested in > figuring out the correct resolution to this issue. > >> > >> -Original Message- > >> From: Robbie Gemmell [mailto:robbie.gemm...@gmail.com] > >> Sent: Wednesday, July 1, 2015 7:48 AM > >> To: us...@qpid.apache.org; proton@qpid.apache.org > >> S
