Re: Proton-J Reactor sending delay
On 26. 02. 16 15.36, Bozo Dragojevic wrote: > Iiuc, calling anything except wakeup from outside the reactor thread is > forbidden. > > Reactor (whole proton-c and proton-j) has no locks to protect against > such use. > > The general pattern is to trigger your application from a handler for > 'on reactor init' and > add that handler to the reactor before starting it. > > To send messages from another thread, one needs to build that > functionality around proton. -- maybe it would make sense to add one > lock to quark to allow reactor.schedule() to be called from another s/quark/proton/ Context switch error :) > thread, that would simplify things a lot. > > Right now, I /think/ you cannot do it without a threadsafe queue, > producer needs to call reactor.wakeup() and consumer of the queue can be > a handler on the quiesced event and polls the queue (must not block). > > Bozzo > > > > On 25. 02. 16 20.03, Ken Giusti wrote: >> Andrew - are you calling send() from within a reactor callback? Or from >> another thread? >> >> I'm not very familiar with the Java reactor, but the C reactor has a method >> called pn_reactor_wakeup() which causes it to immediately return from the >> blocking select() call. >> >> -K >> >> - Original Message - >>> From: "Andrew Buckley" >>> To: d...@qpid.apache.org, proton@qpid.apache.org >>> Sent: Thursday, February 25, 2016 12:45:04 PM >>> Subject: RE: Proton-J Reactor sending delay >>> >>> Ah, thanks Robbie. Yes I do now notice the 3141ms timeout inside run(). Are >>> there plans to make that timeout configurable? At least from my point of >>> view, 3 seconds is quite a long time to wait between calling send and the >>> action actually being performed, and applications using the Reactor do >>> suffer a bit of a blow in performance because of this. >>> >>> Thanks, >>> -Andrew >>> >>> -Original Message- >>> From: Robbie Gemmell [mailto:robbie.gemm...@gmail.com] >>> Sent: Monday, February 15, 2016 3:47 AM >>> To: d...@qpid.apache.org; proton@qpid.apache.org >>> Subject: Re: Proton-J Reactor sending delay >>> >>> On 13 February 2016 at 00:28, Andrew Buckley wrote: >>>> I'm using the reactor with Proton-J and have noticed that there is a 2-4 >>>> second delay between when I call send() on a particular link and when that >>>> transfer frame actually goes out. Is this expected behavior? If so, are >>>> there plans to improve on this? And if not, have you seen this in any >>>> other scenario and might you have any ideas what could be causing it? >>>> >>>> Thanks, >>>> -Andrew Buckley >>>> >>> Hi Andrew, >>> >>> While im no expert on the reactor, I'd be surprised if that was expected, >>> and >>> I can't say I'm aware of it being mentioned before. >>> >>> One thing that springs to mind from previous discussion [about proton-c >>> reactor] is that when the reactor has a particular thread dedicated to >>> running it, it sets a 3141ms timeout on its selector meaning it wakes up at >>> that period when it is 'quiesced' (has nothing to do). Seems like perhaps >>> that could be related given your note of 2-4sec. >>> >>> Do you have an example showing the behaviour? >>> >>> Robbie >>> >>> (added proton@ as well, in case anyone only paying attention there has >>> thoughts) >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional >>> commands, e-mail: dev-h...@qpid.apache.org >>> >>>
Re: Proton-J Reactor sending delay
Iiuc, calling anything except wakeup from outside the reactor thread is forbidden. Reactor (whole proton-c and proton-j) has no locks to protect against such use. The general pattern is to trigger your application from a handler for 'on reactor init' and add that handler to the reactor before starting it. To send messages from another thread, one needs to build that functionality around proton. -- maybe it would make sense to add one lock to quark to allow reactor.schedule() to be called from another thread, that would simplify things a lot. Right now, I /think/ you cannot do it without a threadsafe queue, producer needs to call reactor.wakeup() and consumer of the queue can be a handler on the quiesced event and polls the queue (must not block). Bozzo On 25. 02. 16 20.03, Ken Giusti wrote: > Andrew - are you calling send() from within a reactor callback? Or from > another thread? > > I'm not very familiar with the Java reactor, but the C reactor has a method > called pn_reactor_wakeup() which causes it to immediately return from the > blocking select() call. > > -K > > - Original Message - >> From: "Andrew Buckley" >> To: d...@qpid.apache.org, proton@qpid.apache.org >> Sent: Thursday, February 25, 2016 12:45:04 PM >> Subject: RE: Proton-J Reactor sending delay >> >> Ah, thanks Robbie. Yes I do now notice the 3141ms timeout inside run(). Are >> there plans to make that timeout configurable? At least from my point of >> view, 3 seconds is quite a long time to wait between calling send and the >> action actually being performed, and applications using the Reactor do >> suffer a bit of a blow in performance because of this. >> >> Thanks, >> -Andrew >> >> -Original Message- >> From: Robbie Gemmell [mailto:robbie.gemm...@gmail.com] >> Sent: Monday, February 15, 2016 3:47 AM >> To: d...@qpid.apache.org; proton@qpid.apache.org >> Subject: Re: Proton-J Reactor sending delay >> >> On 13 February 2016 at 00:28, Andrew Buckley wrote: >>> I'm using the reactor with Proton-J and have noticed that there is a 2-4 >>> second delay between when I call send() on a particular link and when that >>> transfer frame actually goes out. Is this expected behavior? If so, are >>> there plans to improve on this? And if not, have you seen this in any >>> other scenario and might you have any ideas what could be causing it? >>> >>> Thanks, >>> -Andrew Buckley >>> >> Hi Andrew, >> >> While im no expert on the reactor, I'd be surprised if that was expected, and >> I can't say I'm aware of it being mentioned before. >> >> One thing that springs to mind from previous discussion [about proton-c >> reactor] is that when the reactor has a particular thread dedicated to >> running it, it sets a 3141ms timeout on its selector meaning it wakes up at >> that period when it is 'quiesced' (has nothing to do). Seems like perhaps >> that could be related given your note of 2-4sec. >> >> Do you have an example showing the behaviour? >> >> Robbie >> >> (added proton@ as well, in case anyone only paying attention there has >> thoughts) >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional >> commands, e-mail: dev-h...@qpid.apache.org >> >>
Re: PN_REACTOR_QUIESCED
Hi Alan, Rafael,
On 9. 10. 15 21.25, aconway wrote:
> I'm fiddling with the C++ example broker, and when I install a debug
> handler, I see that when the broker is doing absolutely nothing there
> is a PN_REACTOR_QUIESCED event about every 3 seconds. Does anybody know
> what this is about? Why is the reactor waking up just to tell us that
> it is asleep?
>
>
On first sight seems like a debug thing accidentally committed.
I think something like this is in order:
$ git diff
diff --git a/proton-c/src/reactor/reactor.c b/proton-c/src/reactor/reactor.c
index 6b328bc..7542d4c 100644
--- a/proton-c/src/reactor/reactor.c
+++ b/proton-c/src/reactor/reactor.c
@@ -484,7 +484,6 @@ void pn_reactor_stop(pn_reactor_t *reactor) {
void pn_reactor_run(pn_reactor_t *reactor) {
assert(reactor);
- pn_reactor_set_timeout(reactor, 3141);
pn_reactor_start(reactor);
while (pn_reactor_process(reactor)) {}
pn_reactor_stop(reactor);
workaround is to pn_set_reactor_timeout(r, 0) in PN_REACTOR_INIT in your
broker.
Bozzo
[jira] [Created] (PROTON-1017) Engine does not handle UNINITALIZED/CLOSED sessions
Bozo Dragojevic created PROTON-1017:
---
Summary: Engine does not handle UNINITALIZED/CLOSED sessions
Key: PROTON-1017
URL: https://issues.apache.org/jira/browse/PROTON-1017
Project: Qpid Proton
Issue Type: Bug
Components: proton-j
Affects Versions: 0.10
Reporter: Bozo Dragojevic
Priority: Critical
If the initiator sends a BEGIN and END frame the receiving engine processed the
END frame before generating the outgoing BEGIN frame and it has no notion of
remoteChannel number anymore.
{noformat}
[2114881339:0] -> Open{ containerId='', hostname='', maxFrameSize=4294967295,
channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[1472159463:0] <- Open{ containerId='', hostname='', maxFrameSize=4294967295,
channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[1472159463:0] -> Open{ containerId='', hostname='', maxFrameSize=4294967295,
channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[2114881339:0] <- Open{ containerId='', hostname='', maxFrameSize=4294967295,
channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[2114881339:0] -> Begin{remoteChannel=null, nextOutgoingId=1,
incomingWindow=2147483647, outgoingWindow=2147483647, handleMax=65535,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[2114881339:0] -> End{error=null}
[1472159463:0] <- Begin{remoteChannel=null, nextOutgoingId=1,
incomingWindow=2147483647, outgoingWindow=2147483647, handleMax=65535,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[1472159463:0] <- End{error=null}
[1472159463:0] -> Begin{remoteChannel=65535, nextOutgoingId=1,
incomingWindow=2147483647, outgoingWindow=2147483647, handleMax=65535,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
[1472159463:0] -> End{error=null}
[2114881339:0] <- Begin{remoteChannel=65535, nextOutgoingId=1,
incomingWindow=2147483647, outgoingWindow=2147483647, handleMax=65535,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
{noformat}
test dies with
{noformat}
java.lang.NullPointerException: uncorrelated channel: 65535
at
org.apache.qpid.proton.engine.impl.TransportImpl.handleBegin(TransportImpl.java:1074)
at
org.apache.qpid.proton.engine.impl.TransportImpl.handleBegin(TransportImpl.java:1)
at org.apache.qpid.proton.amqp.transport.Begin.invoke(Begin.java:144)
at
org.apache.qpid.proton.engine.impl.TransportImpl.handleFrame(TransportImpl.java:1304)
at
org.apache.qpid.proton.engine.impl.FrameParser.input(FrameParser.java:419)
at
org.apache.qpid.proton.engine.impl.FrameParser.process(FrameParser.java:528)
at
org.apache.qpid.proton.engine.impl.TransportImpl.process(TransportImpl.java:1415)
at
org.apache.qpid.proton.engine.impl.TransportImpl.processInput(TransportImpl.java:1373)
at
org.apache.qpid.proton.systemtests.EngineTestBase.pumpServerToClient(EngineTestBase.java:73)
at
org.apache.qpid.proton.systemtests.ProtonEngineExampleTest.testPROTON_TBD(ProtonEngineExampleTest.java:350)
...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (PROTON-691) Allow proton to be built with add_subdirectory
[ https://issues.apache.org/jira/browse/PROTON-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic closed PROTON-691. -- Resolution: Won't Fix use ExternalProject_Add > Allow proton to be built with add_subdirectory > --- > > Key: PROTON-691 > URL: https://issues.apache.org/jira/browse/PROTON-691 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c > Reporter: Bozo Dragojevic > > When tracking proton as a git submodule it's very convenient to be able > to simply integrate it into the superproject cmake build system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-234) Add lots of logging to better understand inner working of proton
[ https://issues.apache.org/jira/browse/PROTON-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic closed PROTON-234. -- Resolution: Won't Fix proton has outgrown these > Add lots of logging to better understand inner working of proton > > > Key: PROTON-234 > URL: https://issues.apache.org/jira/browse/PROTON-234 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c > Reporter: Bozo Dragojevic > Labels: logging > > Add lots of logging to proton to ease understnding of it's inner workings > Introduce object naming facility as some layers are coupled really loosely > for now the logging route will just dump everything out to stderr, as it was > before > This is currently one giant commit off 0.3: > https://github.com/ttlj/qpid-proton/commits/upstream-logging > It doesn't merge quite cleanly to trunk. If this is interesting thing to have > I'd be willing to merge or rebase whichever is preferred. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-288) Make it possible to start the server later during the test
[ https://issues.apache.org/jira/browse/PROTON-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic closed PROTON-288. -- Resolution: Won't Fix proton has outgrown these > Make it possible to start the server later during the test > -- > > Key: PROTON-288 > URL: https://issues.apache.org/jira/browse/PROTON-288 > Project: Qpid Proton > Issue Type: Bug >Reporter: Bozo Dragojevic > Attachments: > 0001-Make-it-possible-to-start-the-server-later-during-th.patch, > 0001-PROTON-288-Make-it-possible-to-start-the-server-late.patch > > > I'm seeing weird behavior of messenger if the server is not around. > The attached patch adds such capability to the messenger test class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-289) outgoing tracker and outgoing queue disagree in case the server is not around
[ https://issues.apache.org/jira/browse/PROTON-289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic closed PROTON-289. -- Resolution: Won't Fix proton has outgrown these > outgoing tracker and outgoing queue disagree in case the server is not around > - > > Key: PROTON-289 > URL: https://issues.apache.org/jira/browse/PROTON-289 > Project: Qpid Proton > Issue Type: Bug >Reporter: Bozo Dragojevic > Attachments: > 0002-Add-test-for-sending-to-a-server-that-is-not-started.patch > > > send a message to localhost:12345 when there is no server > tracker will say it's PENDING > outgoing count will be 0 > Which of them is wrong? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-233) Nonblocking connect must be handled explicitly in pn_driver_wait
[ https://issues.apache.org/jira/browse/PROTON-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic closed PROTON-233. -- Resolution: Won't Fix proton has outgrown these > Nonblocking connect must be handled explicitly in pn_driver_wait > > > Key: PROTON-233 > URL: https://issues.apache.org/jira/browse/PROTON-233 > Project: Qpid Proton > Issue Type: Improvement >Reporter: Bozo Dragojevic > > Connecting to a non-existing address is not detected properly by the driver > fix: https://github.com/ttlj/qpid-proton/commits/upstream-connect -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-1014) Documentation: compilation instructions refer an invalid path
[ https://issues.apache.org/jira/browse/PROTON-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-1014. - Resolution: Not A Problem Assignee: Bozo Dragojevic Fix Version/s: (was: 0.11) Otavio, what you see is a cmake feature :) at some point before, you did a in-tree cmake. After that cmake does not honor out-of-tree builds anymore, it seems. in the qpid-proton checkout run: $ git clean -fd and proceed as per INSTALL.md > Documentation: compilation instructions refer an invalid path > - > > Key: PROTON-1014 > URL: https://issues.apache.org/jira/browse/PROTON-1014 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Reporter: Otavio Rodolfo Piske > Assignee: Bozo Dragojevic >Priority: Minor > Labels: documentation > > Documentation: compilation instructions refer an invalid path > The compilation instructions, in file INSTALL.md, instructs the user to > create a build directory and switch to that directory for compilation. > However, no files are generated in the build directory and the 'make install' > step fails with the following error: > make: *** No rule to make target 'install'. Stop. > The users should be advised to: a) either configure and generate the > configuration in the project root directory or b) switch to the project root > after configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1014) Documentation: compilation instructions refer an invalid path
[
https://issues.apache.org/jira/browse/PROTON-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948907#comment-14948907
]
Bozo Dragojevic commented on PROTON-1014:
-
bq. – Build files have been written to: /home/opiske/code/cpp/rh/qpid-proton
why is there /build missing at the end of this line in your output..
here is how it looks for me:
{code}
vagrant@builder:/vagrant/qpid-proton$ mkdir PROTON-1014 && cd PROTON-1014 &&
cmake ..
-- The C compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
[[[ lots of stuff removed ]]]
-- Configuring done
-- Generating done
-- Build files have been written to: /vagrant/qpid-proton/PROTON-1014
{code}
> Documentation: compilation instructions refer an invalid path
> -
>
> Key: PROTON-1014
> URL: https://issues.apache.org/jira/browse/PROTON-1014
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Reporter: Otavio Rodolfo Piske
>Priority: Minor
> Labels: documentation
> Fix For: 0.11
>
>
> Documentation: compilation instructions refer an invalid path
> The compilation instructions, in file INSTALL.md, instructs the user to
> create a build directory and switch to that directory for compilation.
> However, no files are generated in the build directory and the 'make install'
> step fails with the following error:
> make: *** No rule to make target 'install'. Stop.
> The users should be advised to: a) either configure and generate the
> configuration in the project root directory or b) switch to the project root
> after configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PROTON-997) Allow proton-j handler to be extended by a jython class
[ https://issues.apache.org/jira/browse/PROTON-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-997. Resolution: Fixed Fix Version/s: 0.11 > Allow proton-j handler to be extended by a jython class > --- > > Key: PROTON-997 > URL: https://issues.apache.org/jira/browse/PROTON-997 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j, python-binding > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > > Improve event propagation and exception propagation in the wrappers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-1016) Jython implements long with a BigInteger
[ https://issues.apache.org/jira/browse/PROTON-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-1016. - Resolution: Fixed Fix Version/s: 0.11 > Jython implements long with a BigInteger > > > Key: PROTON-1016 > URL: https://issues.apache.org/jira/browse/PROTON-1016 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > > If a java handler passes a HashMap to python which puts a long > into the map such map cannot be encoded in java since jython does not coerce > BigInteger to a Long in this case. > Add support for encoding of BigInteger in the range [Long.MIN_VALUE, > Long.MAX_VALUE] to a LONG amqp type, decoding is not needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1016) Jython implements long with a BigInteger
Bozo Dragojevic created PROTON-1016: --- Summary: Jython implements long with a BigInteger Key: PROTON-1016 URL: https://issues.apache.org/jira/browse/PROTON-1016 Project: Qpid Proton Issue Type: Improvement Components: proton-j Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic If a java handler passes a HashMap to python which puts a long into the map such map cannot be encoded in java since jython does not coerce BigInteger to a Long in this case. Add support for encoding of BigInteger in the range [Long.MIN_VALUE, Long.MAX_VALUE] to a LONG amqp type, decoding is not needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-982) Make proton python tests compatible with unittest
[ https://issues.apache.org/jira/browse/PROTON-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-982. Resolution: Fixed Fix Version/s: 0.11 > Make proton python tests compatible with unittest > - > > Key: PROTON-982 > URL: https://issues.apache.org/jira/browse/PROTON-982 > Project: Qpid Proton > Issue Type: Improvement > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-984) Document proton-j time units
[ https://issues.apache.org/jira/browse/PROTON-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-984. Resolution: Fixed Fix Version/s: 0.11 > Document proton-j time units > > > Key: PROTON-984 > URL: https://issues.apache.org/jira/browse/PROTON-984 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > > Proton-j uses undisclosed time units which seem to be milliseconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: pn_link_receive fails when previous delivery was not settled
Hi Tomas, On 6. 10. 15 14.04, Tomáš Šoltys wrote: > Hi, > > I would like to receive messages without settling them. And do settle > later. Is it possible? You need to tell the link that you are done receiving and to go to the next delivery by doing a pn_link_advance(). > I have modified tests/tools/apps/c/reactor-recv.c as such so the receiver > does not call pn_delivery_settle(). pn_delivery_settle() does it automatically if it's settling the link current delivery. > Result is that pn_link_recv() will fail. > > Is this expected behaviour? > > You see pn_link_recv() fail as there are no more bytes for the link current delivery. Bozzo
[jira] [Commented] (PROTON-1001) Python tests fail in environments without unittest2
[ https://issues.apache.org/jira/browse/PROTON-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903012#comment-14903012 ] Bozo Dragojevic commented on PROTON-1001: - you can also just pip install unittest2 > Python tests fail in environments without unittest2 > --- > > Key: PROTON-1001 > URL: https://issues.apache.org/jira/browse/PROTON-1001 > Project: Qpid Proton > Issue Type: Bug >Reporter: Justin Ross >Assignee: Ken Giusti > > Provide compatibility with python 2.6: > import unittest > try: > from unittest import SkipTest > except: >try: >from unittest2 import SkipTest >except: >class SkipTest(Exception): >pass -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-997) Allow proton-j handler to be extended by a jython class
Bozo Dragojevic created PROTON-997: -- Summary: Allow proton-j handler to be extended by a jython class Key: PROTON-997 URL: https://issues.apache.org/jira/browse/PROTON-997 Project: Qpid Proton Issue Type: Improvement Components: proton-j, python-binding Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic Improve event propagation and exception propagation in the wrappers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-964) Proton-J extensible event types
[ https://issues.apache.org/jira/browse/PROTON-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-964. Resolution: Fixed > Proton-J extensible event types > --- > > Key: PROTON-964 > URL: https://issues.apache.org/jira/browse/PROTON-964 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j >Affects Versions: 0.10 > Reporter: Bozo Dragojevic >Assignee: Bozo Dragojevic > Fix For: 0.11 > > > Event.Type is an enum which makes it impossible to extend. > Introduce a separate interface EventType and make Event.Type implement it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-981) Make JythonTest extendable
[ https://issues.apache.org/jira/browse/PROTON-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-981. Resolution: Fixed Fix Version/s: 0.11 > Make JythonTest extendable > -- > > Key: PROTON-981 > URL: https://issues.apache.org/jira/browse/PROTON-981 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-980) Enable handler processing the event after child handlers have processed it
[ https://issues.apache.org/jira/browse/PROTON-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-980. Resolution: Fixed > Enable handler processing the event after child handlers have processed it > -- > > Key: PROTON-980 > URL: https://issues.apache.org/jira/browse/PROTON-980 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j >Affects Versions: 0.10 > Reporter: Bozo Dragojevic >Assignee: Bozo Dragojevic > Fix For: 0.11 > > > A Handler may wish to do further processing after all child handlers have > handled the event. Allow the handler to delegate the event to it's children. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-937) LinkImpl.localOpen() does not initialize source and target
[ https://issues.apache.org/jira/browse/PROTON-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-937. Resolution: Fixed Fix Version/s: 0.11 > LinkImpl.localOpen() does not initialize source and target > -- > > Key: PROTON-937 > URL: https://issues.apache.org/jira/browse/PROTON-937 > Project: Qpid Proton > Issue Type: Bug >Affects Versions: 0.9.1 > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > Fix For: 0.11 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-984) Document proton-j time units
Bozo Dragojevic created PROTON-984: -- Summary: Document proton-j time units Key: PROTON-984 URL: https://issues.apache.org/jira/browse/PROTON-984 Project: Qpid Proton Issue Type: Improvement Components: proton-j Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic Proton-j uses undisclosed time units which seem to be milliseconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-983) Expose Event.copy() to bindings
Bozo Dragojevic created PROTON-983: -- Summary: Expose Event.copy() to bindings Key: PROTON-983 URL: https://issues.apache.org/jira/browse/PROTON-983 Project: Qpid Proton Issue Type: Improvement Reporter: Bozo Dragojevic A handler is not supposed to keep a reference to the event. In proton-j this is more obvious as collector actively clears the events before returning them to the pool. In proton-c this is not strictly necessary as referenced events are not re-pooled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-982) Make proton python tests compatible with unittest
Bozo Dragojevic created PROTON-982: -- Summary: Make proton python tests compatible with unittest Key: PROTON-982 URL: https://issues.apache.org/jira/browse/PROTON-982 Project: Qpid Proton Issue Type: Improvement Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-981) Make JythonTest extendable
Bozo Dragojevic created PROTON-981: -- Summary: Make JythonTest extendable Key: PROTON-981 URL: https://issues.apache.org/jira/browse/PROTON-981 Project: Qpid Proton Issue Type: Improvement Components: proton-j Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-980) Enable handler processing the event after child handlers have processed it
Bozo Dragojevic created PROTON-980: -- Summary: Enable handler processing the event after child handlers have processed it Key: PROTON-980 URL: https://issues.apache.org/jira/browse/PROTON-980 Project: Qpid Proton Issue Type: Improvement Components: proton-j Affects Versions: 0.10 Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic Fix For: 0.11 A Handler may wish to do further processing after all child handlers have handled the event. Allow the handler to delegate the event to it's children. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [GitHub] qpid-proton pull request: PROTON-964: Proton-J extensible event ty...
If there are no objections to this, I'll commit this tomorrow Bozzo On 29. 07. 15 15.07, boo wrote: > GitHub user boo opened a pull request: > > https://github.com/apache/qpid-proton/pull/48 > > PROTON-964: Proton-J extensible event types > > > > You can merge this pull request into a Git repository by running: > > $ git pull https://github.com/boo/qpid-proton proton-964 > > Alternatively you can review and apply these changes as the patch at: > > https://github.com/apache/qpid-proton/pull/48.patch > > To close this pull request, make a commit to your master/trunk branch > with (at least) the following in the commit message: > > This closes #48 > > > commit a72c23a136dddae7efd6385d3b32c406c861a016 > Author: Bozo Dragojevic > Date: 2015-07-13T10:57:13Z > > PROTON-964: Proton-J extensible event types > > > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at infrastruct...@apache.org or file a JIRA ticket > with INFRA. > ---
[jira] [Created] (PROTON-964) Proton-J extensible event types
Bozo Dragojevic created PROTON-964: -- Summary: Proton-J extensible event types Key: PROTON-964 URL: https://issues.apache.org/jira/browse/PROTON-964 Project: Qpid Proton Issue Type: Improvement Components: proton-j Affects Versions: 0.10 Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic Fix For: 0.11 Event.Type is an enum which makes it impossible to extend. Introduce a separate interface EventType and make Event.Type implement it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: proton-j messenger connection issues
On 24. 07. 15 08.44, Yanfeng Liu wrote: > Thanks. > > Our use case is Android app, so we can't provision queues beforehand. > Probably we need consider using proton-C via JNI from Java or look at other > ways. > I think the JNI bindings were dropped in recent proton. Cheers, Bozzo
Re: [1/2] qpid-proton git commit: PROTON-937: LinkImpl.localOpen() does not initialize source and target
On 8. 07. 15 18.16, Robbie Gemmell wrote: > Hi Bozzo, some comments and questions. > > I am seeing test failures due to NPE's from the new code: > https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-proton-j/1043/ > > There should be null checks on the remote Source and Target before > trying to copy them, since there are cases where they are allowed and > expected to be null (though in the failing tests its not clear the > they fall in to those, so much as perhaps not bothering to set them as > they werent of interest for the test). > > I see you listed the change as a bug, but I can't say I was ever under > the impression it should do this (though I can certainly see it makes > life easier in some cases). Does proton-c do this and proton-j did > not? > I reverted this, and will do it properly. Proton-c does do it in the handshaker and in the messenger. Just do a git grep pn_terminus_copy. And yes, it has a check so it doesn't try to copy non-existing stuff. In which case the copy returns an error, which handshaker ignores :) For some unexplicable reason I thought the LinkImpl.localOpen was a better place to do it while it's perfectly easy to do in in the Handshaker.onLinkRemoteOpen() and the messenger already does it in a way. //TODO: the following is not correct; should only copy those properties that we understand link.setSource(link.getRemoteSource()); link.setTarget(link.getRemoteTarget()); > If we are doing this for Source and Target objects, should it not also > work for the Coordinator? Seems a bit unfair to leave it out given the > method was added. I've yet to come up-to-speed with the Coordinator, so cannot comment here :) > Please don't use if statements without braces, someone will inevitably > screw it up when updating things later :) > Agree, too much switching between python and Java and C in one day :) Bozzo
Re: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks
Cooking a revert for the other one, in a few. Bozzo On 9. 07. 15 12.59, Robbie Gemmell wrote: > Thanks Bozzo, that sorts those failures (so long as the issue from > your other change is worked around, to let the build get that far). > > Robbie > > On 9 July 2015 at 11:22, Bozo Dragojevic wrote: >> Hi Robbie, >> >> Yeah, my bad. I was sitting on some local changes so I missed this. >> I'll test on a clean checkout next time. Sorry for all the mess. >> I've commited the missing methods to proton-j >> >> Bozzo >> >> On 9. 07. 15 12.14, Robbie Gemmell wrote: >>> Hi Bozzo, >>> >>> This change also seems to be causing test failures when using the >>> maven build (if you update things to get past the earlier failures, >>> caused by the commit mentioned in the other thread on proton@): >>> >>> proton_tests.reactor.ExceptionTest.test_schedule_cancel . >>> fail >>> Error during test: Traceback (most recent call last): >>> File "/home/gemmellr/workspace/proton/tests/python/proton-test", >>> line 360, in run >>> phase() >>> File >>> "/home/gemmellr/workspace/proton/tests/python/proton_tests/reactor.py", >>> line 181, in test_schedule_cancel >>> now = self.reactor.mark() >>> File >>> "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", >>> line 118, in mark >>> return pn_reactor_mark(self._impl) >>> NameError: global name 'pn_reactor_mark' is not defined >>> >>> >>> Robbie >>> >>> On 9 July 2015 at 10:55, Bozo Dragojevic wrote: >>>> Hi Ken, >>>> >>>> I've installed python3.4 and tox and friends and tried to reproduce it here >>>> and I can confirm that some completely unrelated test fail mysteriously >>>> with python3.4 and that reverting my change makes that failure go away :) >>>> >>>> I've added more task cancellation tests to force the error while still >>>> in the implicated code, as I suspected it could be some refcounting >>>> problem but the new tests do not show anything unusual. >>>> >>>> What is even weirder, with the new tests even the python3.4 suite passes >>>> without segfault! >>>> >>>> So, I consider this a false positive and have left the change in, >>>> including the new tests at ca47d72. >>>> >>>> Does such solution work for you? >>>> >>>> Bozzo >>>> >>>> On 8. 07. 15 16.32, Ken Giusti wrote: >>>>> Hi Bozzo, >>>>> >>>>> Can you please revert this change? >>>>> >>>>> It is causing a segfault in the python unit tests when they are run under >>>>> python3.4. >>>>> >>>>> I haven't hit the segfault on python2.7, only on python3.4 >>>>> >>>>> thanks, >>>>> >>>>> -K >>>>> >>>>> - Original Message - >>>>>> From: bo...@apache.org >>>>>> To: comm...@qpid.apache.org >>>>>> Sent: Tuesday, July 7, 2015 3:50:16 PM >>>>>> Subject: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks >>>>>> >>>>>> PROTON-928: cancellable tasks >>>>>> >>>>>> A scheduled task can be cancelled. >>>>>> A cancelled task does not prevent reactor from stopping running >>>>>> >>>>>> >>>>>> Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo >>>>>> Commit: >>>>>> http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/d4d22ee3 >>>>>> Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/d4d22ee3 >>>>>> Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/d4d22ee3 >>>>>> >>>>>> Branch: refs/heads/master >>>>>> Commit: d4d22ee396163babcac19c48845b1f10ca3b5a48 >>>>>> Parents: 09af375 >>>>>> Author: Bozo Dragojevic >>>>>> Authored: Tue Jul 7 10:17:40 2015 +0200 >>>>>> Committer: Bozo Dragojevic >>>>>> Committed: Tue Jul 7 21:49:44 2015 +0200 >>>>>> >>>>>>
Re: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks
Hi Robbie, Yeah, my bad. I was sitting on some local changes so I missed this. I'll test on a clean checkout next time. Sorry for all the mess. I've commited the missing methods to proton-j Bozzo On 9. 07. 15 12.14, Robbie Gemmell wrote: > Hi Bozzo, > > This change also seems to be causing test failures when using the > maven build (if you update things to get past the earlier failures, > caused by the commit mentioned in the other thread on proton@): > > proton_tests.reactor.ExceptionTest.test_schedule_cancel . fail > Error during test: Traceback (most recent call last): > File "/home/gemmellr/workspace/proton/tests/python/proton-test", > line 360, in run > phase() > File > "/home/gemmellr/workspace/proton/tests/python/proton_tests/reactor.py", > line 181, in test_schedule_cancel > now = self.reactor.mark() > File > "/home/gemmellr/workspace/proton/tests/../proton-c/bindings/python/proton/reactor.py", > line 118, in mark > return pn_reactor_mark(self._impl) > NameError: global name 'pn_reactor_mark' is not defined > > > Robbie > > On 9 July 2015 at 10:55, Bozo Dragojevic wrote: >> Hi Ken, >> >> I've installed python3.4 and tox and friends and tried to reproduce it here >> and I can confirm that some completely unrelated test fail mysteriously >> with python3.4 and that reverting my change makes that failure go away :) >> >> I've added more task cancellation tests to force the error while still >> in the implicated code, as I suspected it could be some refcounting >> problem but the new tests do not show anything unusual. >> >> What is even weirder, with the new tests even the python3.4 suite passes >> without segfault! >> >> So, I consider this a false positive and have left the change in, >> including the new tests at ca47d72. >> >> Does such solution work for you? >> >> Bozzo >> >> On 8. 07. 15 16.32, Ken Giusti wrote: >>> Hi Bozzo, >>> >>> Can you please revert this change? >>> >>> It is causing a segfault in the python unit tests when they are run under >>> python3.4. >>> >>> I haven't hit the segfault on python2.7, only on python3.4 >>> >>> thanks, >>> >>> -K >>> >>> - Original Message - >>>> From: bo...@apache.org >>>> To: comm...@qpid.apache.org >>>> Sent: Tuesday, July 7, 2015 3:50:16 PM >>>> Subject: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks >>>> >>>> PROTON-928: cancellable tasks >>>> >>>> A scheduled task can be cancelled. >>>> A cancelled task does not prevent reactor from stopping running >>>> >>>> >>>> Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo >>>> Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/d4d22ee3 >>>> Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/d4d22ee3 >>>> Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/d4d22ee3 >>>> >>>> Branch: refs/heads/master >>>> Commit: d4d22ee396163babcac19c48845b1f10ca3b5a48 >>>> Parents: 09af375 >>>> Author: Bozo Dragojevic >>>> Authored: Tue Jul 7 10:17:40 2015 +0200 >>>> Committer: Bozo Dragojevic >>>> Committed: Tue Jul 7 21:49:44 2015 +0200 >>>> >>>> -- >>>> proton-c/bindings/python/proton/reactor.py | 5 +++- >>>> proton-c/include/proton/reactor.h | 1 + >>>> proton-c/src/reactor/timer.c| 25 +++- >>>> proton-c/src/tests/reactor.c| 15 >>>> .../org/apache/qpid/proton/reactor/Task.java| 4 >>>> .../qpid/proton/reactor/impl/TaskImpl.java | 10 >>>> .../apache/qpid/proton/reactor/impl/Timer.java | 19 --- >>>> proton-j/src/main/resources/creactor.py | 3 +++ >>>> tests/python/proton_tests/reactor.py| 14 +++ >>>> 9 files changed, 91 insertions(+), 5 deletions(-) >>>> -- >>>> >>>> >>>> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/d4d22ee3/proton-c/bindings/python/proton/reactor.py >>>> -
Re: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks
Hi Ken, I've installed python3.4 and tox and friends and tried to reproduce it here and I can confirm that some completely unrelated test fail mysteriously with python3.4 and that reverting my change makes that failure go away :) I've added more task cancellation tests to force the error while still in the implicated code, as I suspected it could be some refcounting problem but the new tests do not show anything unusual. What is even weirder, with the new tests even the python3.4 suite passes without segfault! So, I consider this a false positive and have left the change in, including the new tests at ca47d72. Does such solution work for you? Bozzo On 8. 07. 15 16.32, Ken Giusti wrote: > Hi Bozzo, > > Can you please revert this change? > > It is causing a segfault in the python unit tests when they are run under > python3.4. > > I haven't hit the segfault on python2.7, only on python3.4 > > thanks, > > -K > > - Original Message - >> From: bo...@apache.org >> To: comm...@qpid.apache.org >> Sent: Tuesday, July 7, 2015 3:50:16 PM >> Subject: [2/2] qpid-proton git commit: PROTON-928: cancellable tasks >> >> PROTON-928: cancellable tasks >> >> A scheduled task can be cancelled. >> A cancelled task does not prevent reactor from stopping running >> >> >> Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo >> Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/d4d22ee3 >> Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/d4d22ee3 >> Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/d4d22ee3 >> >> Branch: refs/heads/master >> Commit: d4d22ee396163babcac19c48845b1f10ca3b5a48 >> Parents: 09af375 >> Author: Bozo Dragojevic >> Authored: Tue Jul 7 10:17:40 2015 +0200 >> Committer: Bozo Dragojevic >> Committed: Tue Jul 7 21:49:44 2015 +0200 >> >> -- >> proton-c/bindings/python/proton/reactor.py | 5 +++- >> proton-c/include/proton/reactor.h | 1 + >> proton-c/src/reactor/timer.c| 25 +++- >> proton-c/src/tests/reactor.c| 15 >> .../org/apache/qpid/proton/reactor/Task.java| 4 >> .../qpid/proton/reactor/impl/TaskImpl.java | 10 >> .../apache/qpid/proton/reactor/impl/Timer.java | 19 --- >> proton-j/src/main/resources/creactor.py | 3 +++ >> tests/python/proton_tests/reactor.py| 14 +++ >> 9 files changed, 91 insertions(+), 5 deletions(-) >> -- >> >> >> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/d4d22ee3/proton-c/bindings/python/proton/reactor.py >> -- >> diff --git a/proton-c/bindings/python/proton/reactor.py >> b/proton-c/bindings/python/proton/reactor.py >> index c66334b..d019554 100644 >> --- a/proton-c/bindings/python/proton/reactor.py >> +++ b/proton-c/bindings/python/proton/reactor.py >> @@ -53,6 +53,9 @@ class Task(Wrapper): >> def _init(self): >> pass >> >> +def cancel(self): >> +pn_task_cancel(self._impl) >> + >> class Acceptor(Wrapper): >> >> def __init__(self, impl): >> @@ -112,7 +115,7 @@ class Reactor(Wrapper): >> pn_reactor_yield(self._impl) >> >> def mark(self): >> -pn_reactor_mark(self._impl) >> +return pn_reactor_mark(self._impl) >> >> def _get_handler(self): >> return WrappedHandler.wrap(pn_reactor_get_handler(self._impl), >> self.on_error) >> >> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/d4d22ee3/proton-c/include/proton/reactor.h >> -- >> diff --git a/proton-c/include/proton/reactor.h >> b/proton-c/include/proton/reactor.h >> index 59b2282..6f52d22 100644 >> --- a/proton-c/include/proton/reactor.h >> +++ b/proton-c/include/proton/reactor.h >> @@ -96,6 +96,7 @@ PN_EXTERN pn_task_t *pn_timer_schedule(pn_timer_t *timer, >> pn_timestamp_t deadlin >> PN_EXTERN int pn_timer_tasks(pn_timer_t *timer); >> >> PN_EXTERN pn_record_t *pn_task_attachments(pn_task_t *task); >> +PN_EXTERN void pn_task_cancel(pn_task_t *task); >> >> PN_EXTERN pn_reactor_t *pn_class_reactor(const pn_class_t *clazz, void >> *object); >> PN_EXTERN pn_reactor_t *pn_obje
[jira] [Resolved] (PROTON-928) Cancellable tasks
[ https://issues.apache.org/jira/browse/PROTON-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic resolved PROTON-928. Resolution: Fixed > Cancellable tasks > - > > Key: PROTON-928 > URL: https://issues.apache.org/jira/browse/PROTON-928 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c, proton-j, python-binding > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > > Allow scheduled tasks to be cancelled. > A cancelled task does not keep reactor from stoping running -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-937) LinkImpl.localOpen() does not initialize source and target
[ https://issues.apache.org/jira/browse/PROTON-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617261#comment-14617261 ] Bozo Dragojevic commented on PROTON-937: I've commited the fix, but i still owe a proper testcase, will leave this issue open > LinkImpl.localOpen() does not initialize source and target > -- > > Key: PROTON-937 > URL: https://issues.apache.org/jira/browse/PROTON-937 > Project: Qpid Proton > Issue Type: Bug >Affects Versions: 0.9.1 > Reporter: Bozo Dragojevic >Assignee: Bozo Dragojevic > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-937) LinkImpl.localOpen() does not initialize source and target
Bozo Dragojevic created PROTON-937: -- Summary: LinkImpl.localOpen() does not initialize source and target Key: PROTON-937 URL: https://issues.apache.org/jira/browse/PROTON-937 Project: Qpid Proton Issue Type: Bug Affects Versions: 0.9.1 Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-928) Cancellable tasks
[ https://issues.apache.org/jira/browse/PROTON-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic reassigned PROTON-928: -- Assignee: Bozo Dragojevic > Cancellable tasks > - > > Key: PROTON-928 > URL: https://issues.apache.org/jira/browse/PROTON-928 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c, proton-j, python-binding > Reporter: Bozo Dragojevic > Assignee: Bozo Dragojevic > > Allow scheduled tasks to be cancelled. > A cancelled task does not keep reactor from stoping running -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-928) Cancellable tasks
Bozo Dragojevic created PROTON-928: -- Summary: Cancellable tasks Key: PROTON-928 URL: https://issues.apache.org/jira/browse/PROTON-928 Project: Qpid Proton Issue Type: Improvement Components: proton-c, proton-j, python-binding Reporter: Bozo Dragojevic Allow scheduled tasks to be cancelled. A cancelled task does not keep reactor from stoping running -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: codec changes
On 7. 04. 15 15.38, Alan Conway wrote:
> On Tue, 2015-03-31 at 19:17 +0200, Božo Dragojevič wrote:
>> Given the memory overhead of a pn_data_t before encoding, why not have it
>> own an encode buffer? it could get by with exactly that grow_buffer()
>> callback if ownership is the issue .
> I like this idea a lot. Ownership is an issue for me in Go so I would
> want an alloc/free callback but we could have a non-callback version as
> well (simple wrapper for the callback version with default callbacks)
> for users that don't care about ownership - what do others think?
>
[snip]
> Note:
> - alloc called 0 or 1 times, and is passed the exact size needed to encode.
I would hate to have the encoder know a-priori the exact size of allocation.
Especially if we want to, at some point be able to generate a
{LIST|MAP|ARRAY}8 encodings.
I think we can aim for upper bound, but leave out exact-ness.
calculating encoding size for non-composite atoms is easy and cheap so
it could be moved from
encoding to pn_data_put_*() mutators. as long as uuid is part of atom
union, adding another int will not hurt :P
Calculating conservative size for composites is also cheap.
Maybe pn_data_t could even be made to encode as it goes and would thus
not need the extra buffer *and* copying for
variable-sized atoms (strings and binary) and each atom could be reduced
to a offset into the encoded buffer.
> - user is free to allocate more (e.g. doubling strategies)
> - free called after alloc, if and only if alloc is called.
> - separate free instead of single realloc call: encode impl decides
> if/how much data to copy from old to new buffer.
Partial copying of encoded buffers is also problematic, because encoder
has to backtrack to fill out byte sizes for structured types.
So at best we could do a scatter-gather thing, which quickly becomes
another mess as the pn_data_t then needs
a much better arithmetic for offset and size calculations. I really
think we should just offer exactly realloc() for now.
Bozzo
Re: Items for the 0.9 Release notes/announcement
On 24. 03. 15 20.16, Rafael Schloming wrote:
> I'm trying to put together a relatively complete set of changes and release
> notes for 0.9. If there is any particular feature be it new or some
> existing behaviour change that is worthy of being mentioned in the release
> notes or release announcement, please follow up with a suitable blurb on
> this thread.
>
> --Rafael
>
Here is the list of API changes that I noticed going from 0.8 to 0.9 for
proton-c:
- pn_buffer_t is no longer public API
- pn_bytes_t.bytes is renamed to pn_bytes_t.start
- pn_sasl_{client|server}() is replaced by pn_transport_set_server() and
client being the default.
This is just engine, not using any reactor or selector stuff.
Cheers,
Bozzo
Re: working with 0.9 release branch
On 13. 03. 15 22.54, Bozo Dragojevic wrote: > Hi! > > I want to get the fix for PROTON-838 to the 0.9 release. > From the look of it the other fixes are just cherry-picked from master > and applied to 0.9 branch. > Is that the way to do it? Figured it out. cherry-pick -x Bozzo
working with 0.9 release branch
Hi! I want to get the fix for PROTON-838 to the 0.9 release. >From the look of it the other fixes are just cherry-picked from master and applied to 0.9 branch. Is that the way to do it? Cheers, Bozzo
[jira] [Created] (PROTON-838) proton-hawtdispatch cannot connect with SSL
Bozo Dragojevic created PROTON-838: -- Summary: proton-hawtdispatch cannot connect with SSL Key: PROTON-838 URL: https://issues.apache.org/jira/browse/PROTON-838 Project: Qpid Proton Issue Type: Bug Affects Versions: 0.5 Reporter: Bozo Dragojevic Assignee: Bozo Dragojevic Fix For: 0.9 AmqpTransport::createTransport() calls transport.connecting() twice. this is harmless for a TcpTransport but asserts for SslTransport -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-787) thread safety issue -- openssl initialization
[
https://issues.apache.org/jira/browse/PROTON-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256848#comment-14256848
]
Bozo Dragojevic commented on PROTON-787:
from :
{quote}
* The messenger interface is single-threaded. All methods except one
* (::pn_messenger_interrupt()) are intended to be used by one thread
* at a time.
{quote}
> thread safety issue -- openssl initialization
> -
>
> Key: PROTON-787
> URL: https://issues.apache.org/jira/browse/PROTON-787
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.8
> Environment: unix
>Reporter: Jim Spring
>Priority: Minor
>
> Doing multiple calls to pn_messenger_subscribe can potentially end up in
> simultaneous calls to pn_ssl_globals_init which calls into openssl which is
> not thread safe.
> One fix is to explicitly serialize calls (my current approach).
> One other fix - allow for an initial "proton_init" type call that includes
> openssl initialization. Which is called before usage of the library.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-789) Lots of Deprecation Warnings on OSX
[
https://issues.apache.org/jira/browse/PROTON-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256847#comment-14256847
]
Bozo Dragojevic commented on PROTON-789:
here are the rough steps for preparing the SSL related bit of the build
environment for OSX:
{code}
$ brew install pkg-config cmake
$ brew install openssl # this will install a keg-only openssl
\# (cmake finds openssl through pkg-config so link that in manually)
$ ln -s $(brew ls openssl | grep /lib/pkgconfig/) /usr/local/lib/pkgconfig/
{code}
> Lots of Deprecation Warnings on OSX
> ---
>
> Key: PROTON-789
> URL: https://issues.apache.org/jira/browse/PROTON-789
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.8
> Environment: Darwin gethsemane.local 14.0.0 Darwin Kernel Version
> 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64
> x86_64
>Reporter: Weston M. Price
>Priority: Minor
> Fix For: 0.9
>
> Attachments: proton-build-osx.txt
>
>
> Building proton-c on OS X 10.10.1 (Yosemite) gives quite a few deprecation
> warnings, primarily in the SSL libraries. I have attached a file showing a
> typical build run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-789) Lots of Deprecation Warnings on OSX
[ https://issues.apache.org/jira/browse/PROTON-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256845#comment-14256845 ] Bozo Dragojevic commented on PROTON-789: Fwiw, building against homebrew openssl works without these warnings. > Lots of Deprecation Warnings on OSX > --- > > Key: PROTON-789 > URL: https://issues.apache.org/jira/browse/PROTON-789 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Darwin gethsemane.local 14.0.0 Darwin Kernel Version > 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 > x86_64 >Reporter: Weston M. Price >Priority: Minor > Fix For: 0.9 > > Attachments: proton-build-osx.txt > > > Building proton-c on OS X 10.10.1 (Yosemite) gives quite a few deprecation > warnings, primarily in the SSL libraries. I have attached a file showing a > typical build run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: proton slowdown - major clue
On 28. 10. 14 20:18, Michael Goulish wrote: > I have gotten callgrind call-graph pictures of my proton/engine sender > and receiver when the test is running fast and when it slows down. > > The difference is in the sender -- when running fast, it is spending > most of its time in the subtree of pn_connector_process() .. like 71%. > > When it slows down it is instead spending 47% in pn_delivery_writeable(), > and only 17% in pn_connector_process(). > > > Since it is still not instantly obvious to me what has happened, > I thought I would share with you-all. > > > Please see cool pictures at: > > > http://people.apache.org/~mgoulish/protonics/performance/slowdown/2014_10_28/svg/psend_fast.svg > > > http://people.apache.org/~mgoulish/protonics/performance/slowdown/2014_10_28/svg/psend_slow.svg > > > > > To recap -- I can trigger this condition by getting the box busy while > my proton/engine test is running. I.e. by doing a build. > Even though I stop the build, and all 6 other processors on > the box go back to being idle -- the test never recovers. > > The receiver goes down to 50% CPU or worse -- but these pictures > show that the behavior change is in the sender. > > > look at call counts, for pn_connector_process() and pn_delivery_writable() fast : ratio 1 : 5 slow : ratio 1 : 244.5 (!) The iteration over connection work list gets really expensive, which means the connection thinks it has to work on other stuff than what psend.c wants to work on. I still think that the call to pn_delivery() in psend.c is in a really unfortunate spot. btw, why do you iterate over connection work list at all, you could just remember the delivery when calling pn_delivery()? Bozzo
Re: proton gradual slowdown -- I know how to cause it
On 27. 10. 14 13:16, Michael Goulish wrote: > You know, I thought of something along those lines, but I can't > see how it makes the receiver actually use less CPU permanently. > It seems like it ought to simply get a backlog, but go back > to normal CPU usage. My guess is that sender creates for itself some kind of internal backlog that it never recovers from. consequence of which is a bored receiver. Sender should be more or less CPU bound. so for example, if sender is for some reason forced to get another delivery on one of the linked lists, it will always from there on have to traverse two list items to send one delivery (wildly assuming that it takes more than one round thru driver_wait and all the rest to clean things up) so if sender rate-limits itself to create a new delivery only after previous one is sent, it can clean up it's backlog. > Can you think of any way that a backlog would cause receiver to > stay at low CPU? > Yeah, a sender that needs more CPU cycles per message. > Now that i can make this happen easily instead of waiting forever- > and-a-day, I will get callgrind snapshots of both programs when the > test is fast and slow. It seems like that just must show me > something. > > > If it's really CPU bound, it should. Bozzo > > > > - Original Message - >> On 27. 10. 14 09:10, Michael Goulish wrote: >>> Earlier I reported a very gradual slowdown in the performance >>> of my simple 1-sender 1-receiver test, on RHEL 7.0 and Fedora 20 >>> but not on RHEL 6.3 . >>> >>> The slowdown caused the test to end up running at half speed after >>> a billion or two billion messages. ( Which took hours to run. ) >>> >>> >>> I now know how to cause this slowdown to happen any time, and >>> it works just as well on RHEL 6.3 as it does on RHEL 7.0 . >>> >>> All I have to do is make the machine busy. Even though I do not >>> swamp all processors -- in fact, I leave a couple processors idle -- >>> my receiver program slows down when the machine becomes busy -- >>> ***and it never recovers***. >>> >> Michael, >> >> this is totally a wild guess, but looking at your psend.c the only thing >> that jumps out is that you couple 1:1 number of deliveries created and >> number of >> calls to pn_driver_wait(). So if anything (which I cannot explain >> /what/) happens where >> sender starts to lag in talking to receiver it may not be able to dig >> itself out. >> >> Maybe try to create first delivery before the loop and create the next >> delivery after >> pn_link_advance() >> >> Bozzo >>
Re: proton gradual slowdown -- I know how to cause it
On 27. 10. 14 09:10, Michael Goulish wrote: > Earlier I reported a very gradual slowdown in the performance > of my simple 1-sender 1-receiver test, on RHEL 7.0 and Fedora 20 > but not on RHEL 6.3 . > > The slowdown caused the test to end up running at half speed after > a billion or two billion messages. ( Which took hours to run. ) > > > I now know how to cause this slowdown to happen any time, and > it works just as well on RHEL 6.3 as it does on RHEL 7.0 . > > All I have to do is make the machine busy. Even though I do not > swamp all processors -- in fact, I leave a couple processors idle -- > my receiver program slows down when the machine becomes busy -- > ***and it never recovers***. > Michael, this is totally a wild guess, but looking at your psend.c the only thing that jumps out is that you couple 1:1 number of deliveries created and number of calls to pn_driver_wait(). So if anything (which I cannot explain /what/) happens where sender starts to lag in talking to receiver it may not be able to dig itself out. Maybe try to create first delivery before the loop and create the next delivery after pn_link_advance() Bozzo
Re: how can I get proton-api version 1.0-SNAPSHOT for api-reconciliation?
On 2. 10. 14 17:31, Ernest Allen wrote: > Since the api-reconciliation tool is no longer viable, are there any example > java apps that use the engine api? > contrib/proton-hawtdispatch is also using the engine. Bozzo
[jira] [Commented] (PROTON-660) Fix openssl.c build on windows
[ https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148148#comment-14148148 ] Bozo Dragojevic commented on PROTON-660: https://reviews.apache.org/r/26050/ > Fix openssl.c build on windows > -- > > Key: PROTON-660 > URL: https://issues.apache.org/jira/browse/PROTON-660 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows 7, VS2013 >Reporter: Bozo Dragojevic > Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch, > 25_openssl_fix_for_windows_CMakeLists.patch, > 25_openssl_fix_for_windows_data.h.patch, > 25_openssl_fix_for_windows_platform.h.patch > > > Compiled openssl-1.0.1i from source > Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Updated] (PROTON-660) Fix openssl.c build on windows
As a workaround, I've applied the patch to a svn checkout and uploaded svn diff. That works, but is a bit clumsy... Bozzo On 25. 09. 14 20:50, Bozo Dragojevic wrote: > Can somebody help me with the reviewboard, please? > > I wanted to stuff this patch to reviewboard, but it gets rejected with: > > The specified diff file could not be parsed. > Line 19: No valid separator after the filename was found in the diff header > > > I selected repository 'qpid' > I uploaded the patch below > I selected base directory /proton/trunk (I have also tried > /qpid/proton/trunk with same effect) > > fwiw, I'm doing this from a mac using chrome > > Bozzo > > On 25. 09. 14 20:44, Bozo Dragojevic (JIRA) wrote: >> [ >> https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel >> ] >> >> Bozo Dragojevic updated PROTON-660: >> --- >> Attachment: (was: >> 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch) >> >>> Fix openssl.c build on windows >>> -- >>> >>> Key: PROTON-660 >>> URL: https://issues.apache.org/jira/browse/PROTON-660 >>> Project: Qpid Proton >>> Issue Type: Bug >>> Components: proton-c >>>Affects Versions: 0.8 >>> Environment: Windows 7, VS2013 >>>Reporter: Bozo Dragojevic >>> Attachments: 25_openssl_fix_for_windows_CMakeLists.patch, >>> 25_openssl_fix_for_windows_data.h.patch, >>> 25_openssl_fix_for_windows_platform.h.patch >>> >>> >>> Compiled openssl-1.0.1i from source >>> Proton finds it, but openssl.c does not compile without small adjustments. >> >> -- >> This message was sent by Atlassian JIRA >> (v6.3.4#6332)
Re: [jira] [Updated] (PROTON-660) Fix openssl.c build on windows
Can somebody help me with the reviewboard, please? I wanted to stuff this patch to reviewboard, but it gets rejected with: The specified diff file could not be parsed. Line 19: No valid separator after the filename was found in the diff header I selected repository 'qpid' I uploaded the patch below I selected base directory /proton/trunk (I have also tried /qpid/proton/trunk with same effect) fwiw, I'm doing this from a mac using chrome Bozzo On 25. 09. 14 20:44, Bozo Dragojevic (JIRA) wrote: > [ > https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > Bozo Dragojevic updated PROTON-660: > --- > Attachment: (was: > 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch) > >> Fix openssl.c build on windows >> -- >> >> Key: PROTON-660 >> URL: https://issues.apache.org/jira/browse/PROTON-660 >> Project: Qpid Proton >> Issue Type: Bug >> Components: proton-c >> Affects Versions: 0.8 >> Environment: Windows 7, VS2013 >>Reporter: Bozo Dragojevic >> Attachments: 25_openssl_fix_for_windows_CMakeLists.patch, >> 25_openssl_fix_for_windows_data.h.patch, >> 25_openssl_fix_for_windows_platform.h.patch >> >> >> Compiled openssl-1.0.1i from source >> Proton finds it, but openssl.c does not compile without small adjustments. > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332)
[jira] [Updated] (PROTON-660) Fix openssl.c build on windows
[ https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-660: --- Attachment: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch The proposed patch that fixes openssl build by mostly just reshuffling include file order > Fix openssl.c build on windows > -- > > Key: PROTON-660 > URL: https://issues.apache.org/jira/browse/PROTON-660 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows 7, VS2013 >Reporter: Bozo Dragojevic > Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch, > 25_openssl_fix_for_windows_CMakeLists.patch, > 25_openssl_fix_for_windows_data.h.patch, > 25_openssl_fix_for_windows_platform.h.patch > > > Compiled openssl-1.0.1i from source > Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-660) Fix openssl.c build on windows
[ https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-660: --- Attachment: (was: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch) > Fix openssl.c build on windows > -- > > Key: PROTON-660 > URL: https://issues.apache.org/jira/browse/PROTON-660 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows 7, VS2013 >Reporter: Bozo Dragojevic > Attachments: 25_openssl_fix_for_windows_CMakeLists.patch, > 25_openssl_fix_for_windows_data.h.patch, > 25_openssl_fix_for_windows_platform.h.patch > > > Compiled openssl-1.0.1i from source > Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-678) proton-c: (Win 32-bit) pn_post_frame misinterprets variable argument data when ERROR is used
[ https://issues.apache.org/jira/browse/PROTON-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146004#comment-14146004 ] Bozo Dragojevic commented on PROTON-678: Dominic, did you see this error, with you openssl build on windows fix, aka PROTON-660. See my last comment there, I locally also saw a redefinition of ERROR, and transport.c includes platform.h at an awkward place after the protocol.h. > proton-c: (Win 32-bit) pn_post_frame misinterprets variable argument data > when ERROR is used > > > Key: PROTON-678 > URL: https://issues.apache.org/jira/browse/PROTON-678 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 > Environment: Windows x86-32 >Reporter: Dominic Evans >Assignee: Andrew Stitcher > Attachments: 27_32bit_windows_fix_transport.c.patch > > > For Windows 32bit ERROR is not a uint64_t type, which causes > pn_post_frame to misinterpret the passed variable argument data when ERROR is > used > patch attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-660) Fix openssl.c build on windows
[
https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14144717#comment-14144717
]
Bozo Dragojevic commented on PROTON-660:
{quote}
So the real solution is a oneliner in ../platform.h to include winsock.h on
windows. wdyt?
{quote}
.. which is then suspiciously close to what Dominic did. :) Dominic, I saw you
only included wincrypt.h ahead of time
so you only have issues with {{#define small}}. If you include the whole
winsock.h (as windows/io.c does it)
then one gets an additional issue with {{#define ERROR}}. In any case I'd
propose we move the needed #undefs to the platform.h.
I'm thinking it'd make sense to do an include cleanup to make all of this less
fragile and always have the same order of different includes
# system includes
# platform.h
# proton public includes
# proton private includes
sometimes now they are mixed up a bit and it's hard to make sure that
platform.h (un)does all platform-specific fixups
> Fix openssl.c build on windows
> --
>
> Key: PROTON-660
> URL: https://issues.apache.org/jira/browse/PROTON-660
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.8
> Environment: Windows 7, VS2013
>Reporter: Bozo Dragojevic
> Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch,
> 25_openssl_fix_for_windows_CMakeLists.patch,
> 25_openssl_fix_for_windows_data.h.patch,
> 25_openssl_fix_for_windows_platform.h.patch
>
>
> Compiled openssl-1.0.1i from source
> Proton finds it, but openssl.c does not compile without small adjustments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-660) Fix openssl.c build on windows
[
https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14144563#comment-14144563
]
Bozo Dragojevic commented on PROTON-660:
{quote}
I don't understand the need to undefine symbols - the windows header file that
defines them shouldn't be included in any "include chain", and if it is then
that is a problem. The only files in our code base that should include windows
(or more broadly platform specific headers) is platform.c and c files that are
only compiled on specific platforms. None of the .h files should (need to)
include windows headers.
{quote}
I've got the the bottom of this, and have a cleaner solution, still. The root
cause is that openssl/ssl.h includes
openssl/x509.h that already has the exact same undefine. But that includes
happens before it includes openssl/dtls1.h which pulls in winsock.h that
re-defines the symbols. End effect is, openssl on windows works only if
winsock.h is explicitly included before any openssl header is included -- which
I guess is why only some people see this, some not.
So the real solution is a oneliner in ../platform.h to include winsock.h on
windows. wdyt?
{quote}
The use of strncasecmp() should be abstracted the same way we've abstracted
snprintf() - by adding a definition to platform.h using some implementation
probably in platform.c or util.c
{quote}
I've noticed pn_i_eq_nocase() and just followed suit with pn_i_eq_n_nocase().
So, iiuc, that part is OK, I should just leave openssl.c call strncasecmp() and
add to platform.h a
{code}
#define strncasecmp(a,b) (!pn_i_eq_nocase(a,b))
{code}
> Fix openssl.c build on windows
> --
>
> Key: PROTON-660
> URL: https://issues.apache.org/jira/browse/PROTON-660
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.8
> Environment: Windows 7, VS2013
>Reporter: Bozo Dragojevic
> Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch,
> 25_openssl_fix_for_windows_CMakeLists.patch,
> 25_openssl_fix_for_windows_data.h.patch,
> 25_openssl_fix_for_windows_platform.h.patch
>
>
> Compiled openssl-1.0.1i from source
> Proton finds it, but openssl.c does not compile without small adjustments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: 0.8 release prep
On 22. 09. 14 19:57, Andrew Stitcher wrote: > On Sun, 2014-09-21 at 23:43 +0200, Bozo Dragojevic wrote: >> PROTON-660: Fix openssl.c build on windows > Are you planning to do this work yourself? If so please take into > account my last comments on this bug. I'd work on it myself except that > I don't currently have the correct set up. If you can detail the set up > (and it's not hard to set up) I'd do it. > Yep, learning by doing :) What's the most convenient way for the review process? Patches to the mailing list? links to github commits? apache's reviewboard? My guess is the later, except it's the most unfamiliar process to me :) If you want to set up, just build openssl from source (which requires ActivePerl iirc) and just make sure to install in one of the locations that cmake looks at. >> PROTON-691: allow proton to be built with add_subdirectory > How important is this? The current way to integrate a proton build with > another build is via the install directory, this seems to work well. > It's not super important, just very convenient. > Is there a specific scenario when you can't run 2 builds with a common > install prefix? (This would actually probably make more sense added to > the jira itself) > The idea behind this is to get one build system (esp handy for VS and Xcode) We started with two build systems with the intermediate install step. Since there always seem to be at least one proton bugfix waiting to go upstream, we keep track of a patched proton tree as a git submodule. For development cycle when one is working on said fixes it's really convenient to be able to run our tests in-tree quickly, skipping the install step. When proton stabilizes, all this becomes irrelevant. I know such approach does not really scale because cmake's symbols are not namespaced. So, if there is a more proper cmake-istic approach how to achieve a similar workflow I'd be happy to adapt in that direction. Bozzo
Re: 0.8 release prep
On 18. 09. 14 16:55, Rafael Schloming wrote: > Hi Everyone, > > I'd like to put out a 0.8 RC soon. I will be going through JIRA shortly in > order to triage any remaining issues. If there are any particular issues > that you feel strongly should be included in 0.8, please follow up on this > thread and bring them to my attention. > > --Rafael > I'd like to get these two of mine in PROTON-660: Fix openssl.c build on windows PROTON-691: allow proton to be built with add_subdirectory Plus the other half of the Windows IOCP work by Cliff PROTON-684: Restrict IOCP enlistment to pn_selector_t usage scenarios in Windows Bozzo
[jira] [Created] (PROTON-691) Allow proton to be built with add_subdirectory
Bozo Dragojevic created PROTON-691: -- Summary: Allow proton to be built with add_subdirectory Key: PROTON-691 URL: https://issues.apache.org/jira/browse/PROTON-691 Project: Qpid Proton Issue Type: Improvement Components: proton-c Reporter: Bozo Dragojevic When tracking proton as a git submodule it's very convenient to be able to simply integrate it into the superproject cmake build system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Please welcome Bozo Dragojevic as a Qpid committer
On 16. 09. 14 17:59, Rafael Schloming wrote: > The Qpid PMC has voted to grant Bozo Dragojevic commit rights to Qpid for > his long standing interest in and involvement in Proton. > > Welcome Bozo, and thanks for all the contributions! > > --Rafael > Hello everybody, glad to be on board! Bozzo
Re: [jira] [Commented] (PROTON-668) Document Proton-c IO restrictions for 0.8 release
Hi Cliff,
I hope you don't mind if I reply on list, and not in Jira...
On 12. 09. 14 09:18, Cliff Jansen (JIRA) wrote:
> [
> https://issues.apache.org/jira/browse/PROTON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131213#comment-14131213
> ]
>
> Cliff Jansen commented on PROTON-668:
> -
>
> Additional notes based on comments to the Proton mailing list:
>
>>> a socket may only ever be used with a single pn_io_t in its lifetime
>>> exception: a socket from pn_accept() is not yet associated with any
>>> pn_io_t
>>> and its first use can be with any pn_io_t (or never with a pn_io_t at
>>> all)
>> I'm not sure if it's implied, but pn_connect() and pn_listen() also need
>> to support 'third party event loop'.
>> Specifically, pn_connect() has to remain non-blocking (we get to know
>> about the connect error later in the external event loop)
> Yes third party loop support is implied for pn_listen and pn_connect.
> The exception comment is about "moving" a socket between two pn_io_t
> instances (or determining "first use"). The third party loop is merely
> not allowed to pass any given socket to more that one pn_io_t.
>
OK.
>>> threads may do concurrent pn_io_xx) calls as long as no two are
>>> simultaneous
>>> on the same socket (where xxx is send/recv/read/write)
>> This will break pn_io_error() and pn_io_wouldblock() as they are defined now.
> There is certainly no error or blocking state kept per socket, or per
> thread. I am describing the existing multi-threaded use in driver.c
> (heavily used by Dispatch) and assuming others may wish to do the same
> on Windows. pn_io_error and pn_wouldblock are conspicuously absent in
> driver.c (old and new).
>
How the new driver propagates errors? I saw it's looking at errno, but,
does errno interact with iocp?
(didn't look at that, sorry)
Here is one example why sharing an pn_io_t across threads seems unsafe,
even if those threads do not
use pn_io_error() call:
ssize_t pn_send(pn_io_t *io, pn_socket_t sockfd, const void *buf,
size_t len) {
ssize_t count;
iocpdesc_t *iocpd = pni_iocpdesc_map_get(io->iocp, sockfd);
if (iocpd) {
count = pni_iocp_begin_write(iocpd, buf, len, &io->wouldblock,
io->error);
So if two threads do pn_send() on same pn_io_t but on two different
pn_socket_t that are both in
some error state (that is captured in their respective iocpdesc_t->error),
both threads will access the same io->error, and that cannot end well.
>> I have a feeling you don't really want/need to expose the pn_pipe(),
>> but add a pn_selector_interrupt() and a mechanism of querying that for
>> the caller of pn_selector_select() especially as you want to implement
>> it completely differently on windows.
> Yes, the pipe is an awkward difference between Windows and Posix
> thinking, where the latter sees pipes and sockets as close kin but
> Windows sees them as very distant relatives.
>
> Ditching the pn_pipe() call altogether in favour of a
> pn_selector_interrupt() is cleanest and most performant from the
> Windows IOCP perspective. However, as long as a pn_pipe is only
> accessed via pn_read and pn_write the scope for optimization reduces
> any performance penalty to a minimum. So I am OK keeping things as
> they are but would not resist change.
>
>
agree that starting with the smallest API change is better.
Bozzo
[jira] [Commented] (PROTON-684) Restrict IOCP enlistment to pn_selector_t usage scenarios in Windows
[ https://issues.apache.org/jira/browse/PROTON-684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131305#comment-14131305 ] Bozo Dragojevic commented on PROTON-684: The attached patch fixes things for me. pn_io_t without use of selectors behaves as before iocp was introduced. > Restrict IOCP enlistment to pn_selector_t usage scenarios in Windows > > > Key: PROTON-684 > URL: https://issues.apache.org/jira/browse/PROTON-684 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows >Reporter: Cliff Jansen >Assignee: Cliff Jansen >Priority: Blocker > Fix For: 0.8 > > Attachments: PN-684-0.patch > > > Prevent IOCP enlistemnt interfering with external event loops. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-660) Fix openssl.c build on windows
[ https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131207#comment-14131207 ] Bozo Dragojevic commented on PROTON-660: The cmake fix was done as part of PROTON-657 > Fix openssl.c build on windows > -- > > Key: PROTON-660 > URL: https://issues.apache.org/jira/browse/PROTON-660 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows 7, VS2013 >Reporter: Bozo Dragojevic > Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch, > 25_openssl_fix_for_windows_CMakeLists.patch, > 25_openssl_fix_for_windows_data.h.patch, > 25_openssl_fix_for_windows_platform.h.patch > > > Compiled openssl-1.0.1i from source > Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Created] (PROTON-668) Document Proton-c IO restrictions for 0.8 release
Hi Cliff, comments inline On 11. 09. 14 09:32, Cliff Jansen (JIRA) wrote: > > Proton is designed to provide an efficient IO layer that functions without > imposing a threading model on the application. Applications may (1) roll > their own IO and just use the Proton engine, (2) use all Proton primitives, > (3) use some Proton primitives augmented by an external event loop. > > Case (1) is unrelated to this JIRA. The others may be restated: > > Scenario 2: Proton event loop: a proton selector manages socket events for > all sockets placed in the selector, all associated sockets use pn_io_xxx() > calls. Sockets outside the selector are "unmanaged" and passed through to > the OS socket function unchanged. > > Scenario 3: Third party event loop (no proton selector involved), all sockets > are treated as for "unmanaged" in scenario 2. yay, looks like my use-case :) > Scenario 4, 5...: Others to support? > > > The problem: > > The Proton Posix pattern for efficient IO is: > > "tell me when your (OS) buffer is ready for io transfer (in or out)" > > Whereas the normal Windows pattern is somewhat reversed (IO completion ports): > > "tell me when you are done transferring data (to or from) my (user space) > buffer" > > The current Windows IOCP implementation (PROTON-640) tries to make the latter > look like the former with some constraints. There should be documentation > specifying reasonable limits on Proton usage that may be falsely implied by > the API but do not translate efficiently to Windows. Assuming that future > Windows implementations may adopt more aggressive performance strategies > (especially on the read side), I would propose something along the lines of: > > > a socket may only ever be used with a single pn_io_t in its lifetime > exception: a socket from pn_accept() is not yet associated with any > pn_io_t and its first use can be with any pn_io_t (or never with a pn_io_t at > all) I'm not sure if it's implied, but pn_connect() and pn_listen() also need to support 'third party event loop'. Specifically, pn_connect() has to remain non-blocking (we get to know about the connect error later in the external event loop) > send/recv/close may not be intermixed with similar non-Proton OS calls > (otherwise: out of order or lost data) > > a socket can move once from an external loop to a proton loop, but never > the other way > > pn_pipe() values can only be used with pn_read and pn_write and > pn_selector_select, they cannot participate in an external event loop. > External event loops should have their own mechanism how to signal non-socket events into the loop. > Furthermore, there is no thread safety except: > > threads may do concurrent pn_io_xxx() calls as long as no two are > simultaneous on the same socket (where xxx is send/recv/read/write) > This will break pn_io_error() and pn_io_wouldblock() as they are defined now. > pn_selector_select() is thread safe against > pn_read/pn_write/pn_send/pn_recv, but the outcome of the select is > indeterminate. pn_selector_select() must be interrupted and restarted at any > time when other simultaneous IO may affect the outcome. > When you say 'interrupted' is there a simpler way than a pn_write() to writeable pn_socket_t of pn_pipe() that has it's readable pn_socket_t associated with a pn_selectable_t that is added to said pn_selector_t ? ;) I have a feeling you don't really want/need to expose the pn_pipe(), but add a pn_selector_interrupt() and a mechanism of querying that for the caller of pn_selector_select() especially as you want to implement it completely differently on windows. > calls on different pn_io_t objects do not interact and are thread safe. > > > If it is desirable for a socket to be used in an external loop after being > used in a Proton loop, we would need some sort of blocking calls along the > lines of: > > pn_io_flush() > pn_io_drain() > > which would be no-ops on Posix but would unwind outstanding completions on > Windows. > > this part sounds ok, if it is needed Thanks, Bozzo
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Hi Cliff, I agree that the current extra API call is kludgy if not downright ugly. I noticed today you already added a new API call pn_io_selector(), which has separate windows and posix implementations. For short term 'fix', I'd propose to hide my hack inthere -- make calling pn_io_selector() turn on the iocp on windows. Bozzo This would not impact the API and would restore the On 9. 09. 14 07:02, Cliff Jansen wrote: > Ho Bozo, > > Thank you for comments and the suggested patch. I would prefer a > solution that did not have a special Windows-only-sometimes call > "pn_io_no_iocp()". It seems to me anyway that there is another class > of sockets that are pulled into an IOCP context too early, so that a > separate solution is required that should fix your problem too. > Basically, a more lazy enlistment strategy should leave you outside > IOCP and fix the other issue too. > > Based on your problem and how Dispatch strives for multithreaded > performance, I think I have a better handle on what Proton should be > providing for small to medium-large scalability. > > I am going to try to define what is a sensible intersection of Windows > and Posix capabilities to be supported by the proton > io/selector/selectable classes in a separate documentation JIRA and > try to get a fix for you ASAP, probably in yet another JIRA. > > Cliff
Messenger does not verify SSL peers in server mode?
Hi, spotted this while figuring out how to use the SSL module. Messenger pn_listener_ctx()is not configuring trusted CA list for passive connections. Or, at least pn_messenger_set_trusted_certificates() documentation needs to specify it's for client connections ;) Cheers, Bozzo
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
On 5. 09. 14 11:53, Bozo Dragojevic wrote:
The patch is a bit rough
- not sure I really like the name pn_io_no_iocp()
- missing non-windows stub for pn_io_no_iocp()
- it most likely handles the selector access wrong (see XXX comment)
I missed this bit (blush)
diff --git a/proton-c/src/windows/io.c b/proton-c/src/windows/io.c
index f9c85b4..b5fc7b2 100644
--- a/proton-c/src/windows/io.c
+++ b/proton-c/src/windows/io.c
@@ -263,6 +263,14 @@ pn_socket_t pn_connect(pn_io_t *io, const char
*hostarg, const char *port)
if (io->iocp != NULL) {
sock = pni_iocp_begin_connect(io->iocp, sock, addr, io->error);
} else {
+if (connect(sock, addr->ai_addr, addr->ai_addrlen) != 0) {
+ if (WSAGetLastError() != WSAEWOULDBLOCK) {
+pni_win32_error(io->error, "connect", WSAGetLastError());
+freeaddrinfo(addr);
+closesocket(sock);
+return INVALID_SOCKET;
+ }
+}
freeaddrinfo(addr);
}
return sock;
Bozzo
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Hi Cliff,
On 4. 09. 14 20:03, Cliff Jansen wrote:
Hi,
I am not sure what you are trying to do with xdispatch.
Exactly the same idea as contrib/proton-hawtdispatch, but in C.
If you are
using and external loop and managing your own IO, you can bypass the
low level Proton IO primitives and access the proton engine directly.
This is how Qpid uses Proton.
I am using the engine directly.
I am also using io.[ch] as a *very* convenient multi platform socket
abstraction.
If you were using the driver.c mechanism, please take a look at
PROTON-658. This and the main messenger loop show how the
selectors/selectables can be used within Proton. pn_selector_select()
is pretty central to the completion port model on windows.
I'm not using the proton driver, I have my own driver and thread model.
All serialization of one engine instance is one serial libdispatch task
queue.
I'm also not using proton selectors but two libdispatch sources (read
and write)
that are bound to the same task queue.
All bits of my application that need to be talk to the engine then just
queue
tasks to the same libdispatch queue, so we have practically no locks in
the app.
You may find that if you use Windows accept and connect instead of
pn_accept and pn_connect you can keep your descriptors completely
outside the completion port model and use things as you did. If not,
perhaps we can find another way to isolate your sockets from the
completion port implementation.
I like io's eror thing, and the wouldblock thing, I'd hate to duplicate
all that
across the platforms.
Today I had a read through the iocp change.
I see it kept support for 'user supplied' sockets, where it behaves
synchronously, as before.
I can make iocp support be controlled by an opt-out parameter.
Then for example, pn_listen does not enroll the socket to iocp and
pn_accept
does not find it in the map and treat it as user supplied.
Patch attached, with it my code works again.
Would such a change be acceptable?
The patch is a bit rough
- not sure I really like the name pn_io_no_iocp()
- missing non-windows stub for pn_io_no_iocp()
- it most likely handles the selector access wrong (see XXX comment)
The Windows completion port implementation is designed to provide a
highly scalable throughput in a constrained (mostly) single thread
model, but remain lightweight for small clients. Central to that is
the integration with selectors and selectables. But it can't be
everything to everybody.
I did not want to say this is a bad thing! :) I just wanted to let you
guys know
there is a user of the library with slightly different use-case and to
figure out if
such use can be considered to be within scope of the things that proton
is trying to accomplish.
Cheers,
Bozzo
From 357bf8fa698e9a49170b5e53b9abb2a74007dbb2 Mon Sep 17 00:00:00 2001
From: Bozo Dragojevic
Date: Fri, 5 Sep 2014 11:40:57 +0200
Subject: [PATCH 1/1] Make iocp optional on windows
---
proton-c/include/proton/io.h | 1 +
proton-c/src/windows/io.c| 47 ++--
proton-c/src/windows/iocp.c | 8
proton-c/src/windows/iocp.h | 1 +
4 files changed, 47 insertions(+), 10 deletions(-)
diff --git a/proton-c/include/proton/io.h b/proton-c/include/proton/io.h
index 2d56736..3aac524 100644
--- a/proton-c/include/proton/io.h
+++ b/proton-c/include/proton/io.h
@@ -47,6 +47,7 @@ typedef struct pn_io_t pn_io_t;
typedef struct pn_selector_t pn_selector_t;
PN_EXTERN pn_io_t *pn_io(void);
+PN_EXTERN bool pn_io_no_iocp(pn_io_t *io);
PN_EXTERN void pn_io_free(pn_io_t *io);
PN_EXTERN pn_error_t *pn_io_error(pn_io_t *io);
PN_EXTERN pn_socket_t pn_connect(pn_io_t *io, const char *host, const char
*port);
diff --git a/proton-c/src/windows/io.c b/proton-c/src/windows/io.c
index b5660be..f9c85b4 100644
--- a/proton-c/src/windows/io.c
+++ b/proton-c/src/windows/io.c
@@ -92,7 +92,8 @@ void pn_io_finalize(void *obj)
{
pn_io_t *io = (pn_io_t *) obj;
pn_error_free(io->error);
- pn_free(io->iocp);
+ if (io->iocp != NULL)
+pn_free(io->iocp);
WSACleanup();
}
@@ -112,6 +113,20 @@ void pn_io_free(pn_io_t *io)
pn_free(io);
}
+bool pn_io_no_iocp(pn_io_t *io)
+{
+ if (io->iocp != NULL) {
+if (pni_iocpdesc_map_empty(io->iocp)) {
+ pn_free(io->iocp);
+ io->iocp = NULL;
+} else {
+ pn_error_format(io->error, PN_ERR, "Cannot turn off iocp, already in
use");
+ return false;
+}
+ }
+ return true;
+}
+
pn_error_t *pn_io_error(pn_io_t *io)
{
assert(io);
@@ -210,14 +225,17 @@ pn_socket_t pn_listen(pn_io_t *io, const char *host,
const char *port)
return INVALID_SOCKET;
}
- iocpdesc_t *iocpd = pni_iocpdesc_create(io->iocp, sock, false);
- if (!iocpd) {
-pn_i_error_from_errno(io->error, "register");
-closesocket(sock);
-return INVALID_SOCKET;
+ if (io->iocp != N
[jira] [Updated] (PROTON-660) Fix openssl.c build on windows
[ https://issues.apache.org/jira/browse/PROTON-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-660: --- Attachment: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch > Fix openssl.c build on windows > -- > > Key: PROTON-660 > URL: https://issues.apache.org/jira/browse/PROTON-660 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.8 > Environment: Windows 7, VS2013 >Reporter: Bozo Dragojevic > Attachments: 0001-PROTON-660-Fix-openssl.c-build-on-windows.patch > > > Compiled openssl-1.0.1i from source > Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-660) Fix openssl.c build on windows
Bozo Dragojevic created PROTON-660: -- Summary: Fix openssl.c build on windows Key: PROTON-660 URL: https://issues.apache.org/jira/browse/PROTON-660 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows 7, VS2013 Reporter: Bozo Dragojevic Compiled openssl-1.0.1i from source Proton finds it, but openssl.c does not compile without small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Hi! Just a headsup. This commit breaks pn_accept() for me, it fails all the time. I'm not using selectors but drive the socket with libdispatch (from xdispatch) with a custom driver. If I revert this commit the trunk is useable. I'll dig deeper what is going on, I was focusing on getting SSL to work on windows. Cheers, Bozzo On 4. 09. 14 04:12, ASF subversion and git services (JIRA) wrote: [ https://issues.apache.org/jira/browse/PROTON-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120849#comment-14120849 ] ASF subversion and git services commented on PROTON-640: Commit 1622394 from cliffjan...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1622394 ] PROTON-640 Windows IO completion port implementation for pn_io and pn_selectable classes IO completion port Windows IO for Proton Key: PROTON-640 URL: https://issues.apache.org/jira/browse/PROTON-640 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.7 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 Provide a native IO completion port layer similar to the C++ QPID version for Proton. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-657) OSX: build proton with homebrew openssl
[ https://issues.apache.org/jira/browse/PROTON-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14121184#comment-14121184 ] Bozo Dragojevic commented on PROTON-657: Trunk builds on OSX fine. I like your fix better, btw > OSX: build proton with homebrew openssl > --- > > Key: PROTON-657 > URL: https://issues.apache.org/jira/browse/PROTON-657 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.7 > Environment: OSX 10.9.4 >Reporter: Bozo Dragojevic >Assignee: Andrew Stitcher > Attachments: > 0001-PROTON-657-OSX-build-proton-with-homebrew-openssl.patch > > > System supplied openssl deprecates practically all methods used by proton. > Compile against openssl from homebrew which requires additional include > and link directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-657) OSX: build proton with homebrew openssl
[ https://issues.apache.org/jira/browse/PROTON-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-657: --- Attachment: 0001-PROTON-657-OSX-build-proton-with-homebrew-openssl.patch homebrew openssl is keg-only and to make it work with pkg-config one has to do something like: $ ln -s /usr/local/Cellar/openssl/1.0.1h/lib/pkgconfig/* /usr/local/lib/pkgconfig/ > OSX: build proton with homebrew openssl > --- > > Key: PROTON-657 > URL: https://issues.apache.org/jira/browse/PROTON-657 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.7 > Environment: OSX 10.9.4 >Reporter: Bozo Dragojevic > Attachments: > 0001-PROTON-657-OSX-build-proton-with-homebrew-openssl.patch > > > System supplied openssl deprecates practically all methods used by proton. > Compile against openssl from homebrew which requires additional include > and link directory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (PROTON-657) OSX: build proton with homebrew openssl
Bozo Dragojevic created PROTON-657: -- Summary: OSX: build proton with homebrew openssl Key: PROTON-657 URL: https://issues.apache.org/jira/browse/PROTON-657 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.7 Environment: OSX 10.9.4 Reporter: Bozo Dragojevic System supplied openssl deprecates practically all methods used by proton. Compile against openssl from homebrew which requires additional include and link directory. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: events work and demo (How to implement your own broker in < 100 lines of code)
On 2. 07. 14 21:24, Rafael Schloming wrote: Hi Everyone, Apologies for being a bit quiet recently. I've been heads down working on some improvements to the events API. I've put together a little demo that showcases what I've been doing. I like it! One question on opening up the events API a bit more: Our project has a libdispatch based driver for proton (C equivalent of proton-hawtdispatch) and the collector has greatly simplified tying together top and bottom half. One area where I feel my interface between my top and bottom abstraction could be futher simplified is if the driver could inject socket related events (connection down, connect completed, ...) into the same collector. Even just one event type would suffice, I can then interrogate the driver what happened. Currently I have two callbacks in the driver, one for 'there are events' and one for 'there are socket issues' Do you feel such a thing would be of use in general? (or maybe there already is such a thing, but I missed it)? Another big theme is providing the ability to encapsulate different aspects of protocol behaviour into distinct but composable chunks. This is achieved because the Driver dispatches each event to a configurable set of handlers. This allows the Collector to serve as a central "event bus" and allows the various handlers to all tie into this bus and work together to provide useful combinations of behaviour. I can say that writing the top half on top of engine consisted of copying quite large chunks of messenger connection, session and link setup logic with only minor tweaks, so having reusable pieces here sure sounds interesting. Ultimately I hope this work will address the gap that we've had between the simple/easy to use Messenger API and the experts-only Engine API. I believe with a relatively small amount of work on a toolkit like this, the internals of Messenger can be fairly trivially expressed as a composition of different protocol behaviours, thereby allowing a simple and easy starting point that can smoothly transition to any level of sophistication desired. Sorry for the long spiel. Please have a look and let me know what you think. IMHO it'd be good if there was more than one user of said components than just Messenger or their interfaces can quickly get too Messenger specific. One concrete area are threading assumptions between independent engine instances in connection oriented architectures which are in contrast with Messenger single-threaded approach over several connections. Regards, Bozzo
[jira] [Updated] (PROTON-623) Add missing error check to pn_string_inspect
[ https://issues.apache.org/jira/browse/PROTON-623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-623: --- Attachment: 0001-PROTON-623-Add-missing-error-check-to-pn_string_insp.patch Trivial patch appended > Add missing error check to pn_string_inspect > > > Key: PROTON-623 > URL: https://issues.apache.org/jira/browse/PROTON-623 > Project: Qpid Proton > Issue Type: Bug > Reporter: Bozo Dragojevic > Attachments: > 0001-PROTON-623-Add-missing-error-check-to-pn_string_insp.patch > > > Found by XCode Analyze -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (PROTON-622) Add missing error check to pn_event_inspect
[ https://issues.apache.org/jira/browse/PROTON-622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-622: --- Attachment: 0002-PROTON-622-Add-missing-error-check-to-pn_event_inspe.patch > Add missing error check to pn_event_inspect > --- > > Key: PROTON-622 > URL: https://issues.apache.org/jira/browse/PROTON-622 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c > Reporter: Bozo Dragojevic >Priority: Trivial > Attachments: > 0002-PROTON-622-Add-missing-error-check-to-pn_event_inspe.patch > > > Found by XCode Analyze -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (PROTON-623) Add missing error check to pn_string_inspect
Bozo Dragojevic created PROTON-623: -- Summary: Add missing error check to pn_string_inspect Key: PROTON-623 URL: https://issues.apache.org/jira/browse/PROTON-623 Project: Qpid Proton Issue Type: Bug Reporter: Bozo Dragojevic Found by XCode Analyze -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (PROTON-622) Add missing error check to pn_event_inspect
Bozo Dragojevic created PROTON-622: -- Summary: Add missing error check to pn_event_inspect Key: PROTON-622 URL: https://issues.apache.org/jira/browse/PROTON-622 Project: Qpid Proton Issue Type: Improvement Components: proton-c Reporter: Bozo Dragojevic Priority: Trivial Found by XCode Analyze -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE]: Release Proton 0.7 RC4 as 0.7 final
fwiw, [X] Yes, release 0.7 RC4 as 0.7 final [ ] No, 0.7 RC4 as the following issues... Bozzo On 22. 04. 14 13:12, Rafael Schloming wrote: Hi Everyone, I haven't heard of any issues in RC4, so I'm going to put this to a formal vote now: Source artifacts are here: - http://people.apache.org/~rhs/qpid-proton-0.7rc4/ Java binaries are here: - https://repository.apache.org/content/repositories/orgapacheqpid-1004/ Please review and register your vote: [ ] Yes, release 0.7 RC4 as 0.7 final [ ] No, 0.7 RC4 as the following issues... Thanks, --Rafael
Re: Proton 0.7 RC3
On 10. 04. 14 17:26, Rafael Schloming wrote: Hi Everyone, I've put out RC3 due to the compilation issue found on OSX (PROTON-559). The source is posted here: http://people.apache.org/~rhs/qpid-proton-0.7rc3/ Java binaries are here: https://repository.apache.org/content/repositories/orgapacheqpid-1003/ The only delta from RC2 is the fix contributed by Bozo (thanks!). Please continue to test and post any issues you find here. Given the isolated nature of the delta, any issues you find in RC2 should also be relevant to RC3, so feel free to post those here as well. --Rafael trunk now works for me, didn't test with RC as such. right now using proton-c (no proton-j) and running on OSX 10.7, Windows 7 and Linux ubuntu. Bozzo
Re: Proton 0.7 RC2
On 8. 04. 14 21:34, Rafael Schloming wrote: Hi Everyone, I've posted 0.7 RC2 in the usual places. Source can be found here: http://people.apache.org/~rhs/qpid-proton-0.7rc2/ Java binaries here: https://repository.apache.org/content/repositories/orgapacheqpid-1002/ Note that the install procedure has changed from prior versions and the README has had some significant updates, so any build/install testing and README proof reading are very welcome. Please follow up on this thread with any issues you encounter. --Rafael PROTON-559 typo prevents compilation of posix/io.c on OSX Bozzo
[jira] [Created] (PROTON-559) typo prevents compilation of posix/io.c on OSX
Bozo Dragojevic created PROTON-559:
--
Summary: typo prevents compilation of posix/io.c on OSX
Key: PROTON-559
URL: https://issues.apache.org/jira/browse/PROTON-559
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: 0.7
Reporter: Bozo Dragojevic
diff --git a/proton-c/src/posix/io.c b/proton-c/src/posix/io.c
index c6de09e..11379ff 100644
--- a/proton-c/src/posix/io.c
+++ b/proton-c/src/posix/io.c
@@ -229,7 +229,7 @@ static inline int pn_create_socket(int af) {
}
#elif defined(SO_NOSIGPIPE)
ssize_t pn_send(pn_io_t *io, pn_socket_t socket, const void *buf, size_t size)
{
- ssize_t count = return send(socket, buf, len, 0);
+ ssize_t count = send(socket, buf, size, 0);
io->wouldblock = count < 0 && (errno == EAGAIN || errno == EWOULDBLOCK);
return count;
}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Proton 0.7 RC1
On 14. 03. 14 21:25, Rafael Schloming wrote: Hi Everyone, There's been a bunch of key improvements/fixes since Proton 0.6, so it's probably about time for a new release. I've just posted the first RC in the usual places. Please check it out and give a shout if you run into any issues. Source artifacts: http://people.apache.org/~rhs/qpid-proton-0.7rc1/ Java binaries: https://repository.apache.org/content/repositories/orgapacheqpid-1001/ --Rafael Building on VS2013 uncovered a typo and while there I fixed a warning that apparently wasn't there before. https://github.com/boo/qpid-proton/commits/trunk 25a5701 Do not redefine va_copy if not needed (for example, VS2013) 3ec1c14 remove extraenous return There is also a third commit 88176dc Do not assume there is just one element in CMAKE_MODULE_PATH The last one is not strictly a fix of a bug, but allows adding proton with add_subdirectory to the outer project Bozzo
Re: call pn_messenger_accept from another thread
On 18. 01. 14 04:10, serega wrote: Thanks for the answer. Yes, I do have thread safe queue, so I might just follow your suggestion and set up a reverse queue for trackers. If you care about latency this may not be enough because the thread owning the messenger will only be able to observe the queue when it comes out of the (most probably) pn_messenger_recv() which will happen after the next message is received (and if this depends on previous message being accepted, it's deadlock not latency :). The way out is to drive messenger in async mode and when you enqueue the tracker you call pn_messenger_interrupt(), which is safe to call from another thread context. Bozzo
[jira] [Updated] (PROTON-467) Stop initializing encoder and decoder for each message sent/received
[ https://issues.apache.org/jira/browse/PROTON-467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-467: --- Attachment: 0001-PROTON-467-Stop-initializing-encoder-and-decoder-for.patch > Stop initializing encoder and decoder for each message sent/received > > > Key: PROTON-467 > URL: https://issues.apache.org/jira/browse/PROTON-467 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-j >Affects Versions: 0.5 > Reporter: Bozo Dragojevic > Attachments: > 0001-PROTON-467-Stop-initializing-encoder-and-decoder-for.patch > > > the really expensive part is the call to registerAllTypes() > create a thread-local reusable instance of both -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (PROTON-467) Stop initializing encoder and decoder for each message sent/received
Bozo Dragojevic created PROTON-467: -- Summary: Stop initializing encoder and decoder for each message sent/received Key: PROTON-467 URL: https://issues.apache.org/jira/browse/PROTON-467 Project: Qpid Proton Issue Type: Improvement Components: proton-j Affects Versions: 0.5 Reporter: Bozo Dragojevic the really expensive part is the call to registerAllTypes() create a thread-local reusable instance of both -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: The implicit reply_to address
On 5. 11. 13 18:58, Rafael Schloming wrote:
Hi,
I'm scratching my head a bit to understand your scenario so I'll answer the
easy question first. ;-) The address format currently used by messenger
isn't part of any official spec, and the official spec will likely look a
bit different. When the spec is ratified, we will have to change and
probably do some work to maintain backwards compatibility.
in that case it's better to have just one backward compatibility problem
instead of adding
another one on top.
Now as for your scenario, I applied your patch and ran through the scenario
you describe and what I observed was the first reply going from window 1 to
window 3 getting dropped due to the (intentionally) aborted connection, and
the second reply ending up going to window 2 because the window 1 messenger
saw that there was no connection and was able to initiate its own
connection based on the address in the reply_to. What is perhaps a little
odd about this configuration is the fact that the messenger in window 3 has
chosen a dns registered name that collides with the ip/port used in window
2. If you imagine instead that the messenger in window 3 is actually
subscribed to amqp://~127.0.0.1:23456/, then the behaviour could be quite
useful. The server could in fact get a reply back to the client even if the
client's original connection dies.
I fully admit the example is a bit contrieved. the ip-address name was
to ease scenario setup
using the available bits and pieces.
So assuming I understand your scenario/suggestion correctly, I'd say the
ability to choose a dns registered name for your messenger is actually a
feature, not a bug.
The bug part is this: only really simplistic applications will be able
to respond to each request
immediately. As soon as there is any long-running processing (maybe
sending and receiving
another amqp message) required to compute the reply it's much much
easier to come to the
point where reply address will get intermittently interpreted as dns
addresses.
I'll also point out that because the messenger name is
placed in the reply_to address with simple text substitution, you can
always choose your messenger name to be '/foo' instead of foo (or indeed
you could use any illegal domain character) in order to avoid the
possibility of accidental collision.
I like the "/foo" approach but '/' is not a good character as the
resulting reply_to address gets mis-parsed.
try to get an actual reply:
$ ./recv -r & ./send -r -n /foo
...
recv.c:162: unable to connect to amqp:///foo: connect: Address family
not supported by protocol family
Btw, starting the messenger name with "~" and a bit of bad luck will
make the respondent start listening on a new port :)
"Don't do that then" is a good answer :)
Any other illegal domain character seems to fit the bill
here's a proposal for documentation change
---
diff --git a/proton-c/include/proton/message.h
b/proton-c/include/proton/message.h
index 757b076..541cd46 100644
--- a/proton-c/include/proton/message.h
+++ b/proton-c/include/proton/message.h
@@ -86,6 +86,18 @@ PN_EXTERN const char *
pn_message_get_subject (pn_message_t *msg);
PN_EXTERN intpn_message_set_subject
(pn_message_t *msg, const char *subject);
PN_EXTERN const char * pn_message_get_reply_to
(pn_message_t *msg);
+/** Set the reply-to message property
+ *
+ * If the reply_to equals "~" or starts with "~/" then the reply_to
+ * will be rewritten when message is sent with pn_messenger_put(). The
+ * '~' is replaced with "amqp://" where is the name of
+ * the messenger used for sending.
+ *
+ * @param[in] msg the message to modify
+ * @param[in] reply_to the reply-to address
+ *
+ * @return 0 on success or a messenger error
+ */
PN_EXTERN intpn_message_set_reply_to
(pn_message_t *msg, const char *reply_to);
PN_EXTERN pn_data_t *pn_message_correlation_id
(pn_message_t *msg);
diff --git a/proton-c/include/proton/messenger.h
b/proton-c/include/proton/messenger.h
index 5b318bf..f8f7e46 100644
--- a/proton-c/include/proton/messenger.h
+++ b/proton-c/include/proton/messenger.h
@@ -49,6 +49,13 @@ typedef enum {
/** Construct a new Messenger with the given name. The name is global.
* If a NULL name is supplied, a UUID based name will be chosen.
*
+ * Note: See documentation for pn_message_set_reply_to for details how
+ * the name is used. Since the reply address will be re-interpreted by
+ * the responder make sure that the name does not accidentally resolve
+ * as a dns name or even starts with a "~" as that may lead to
+ * messages getting delivered to unexpected destination in certain
+ * edge cases. Name containing "/" should also be avoided.
+ *
* @param[in] name the name of the messenger or NULL
*
* @return pointer to a new Messenger
---
Bozzo
Re: The implicit reply_to address
On 3. 11. 13 17:41, Bozo Dragojevic wrote: [snip] The attached patch 0002-... implements the former solution. I just noticed it's possible to say reply_to = "~/foo/bar". If the principle is sound I can supply the rest of needed changes. Bozzo
The implicit reply_to address
Hi,
when I set message reply_to="~" the messenger changes that to
"amqp://" in outward_munge.
is such address format for addressing the container by name required by
some AMQP spec or is this a messenger/proton specific feature?
for something like this:
m = Messenger("www.google.com:80")
msg = Message()
msg.address = "amqp://127.0.0.1:12345"
msg.reply_to = "~"
m.send(1)
sys.exit(0)
the reply_to will be amqp://www.google.com:80
The server from the testcase will *appear* to work, because the
messenger is not given a chance
to notice connection teardown before it needs to resolve the reply_to
address.
Apply the attached patch 0001-... and run:
# window 1 'server' that sends two replies to each request
$ PN_TRACE_FRM=1 ./recv -rr
# window 2 'google'
$ PN_TRACE_FRM=1 ./recv amqp://~127.0.0.1:23456/
# window 3 'client' with an unfortunate name that dies before it can
receive the response
$ PN_TRACE_FRM=1 ./send -n127.0.0.1:23456 -rw
which forces server to send two replies, one by one and give the client
the chance to go away.
I think it might be better for that address to read "amqp:///" so
that the 'magic' is signaled by empty domain which is otherwise not valid.
or maybe even "amqp:///~/" so that it doesn't pollute the the
'normal' part of the address namespace?
The attached patch 0002-... implements the former solution.
Now the server dies with:
recv.c:162: unable to connect to amqp:///127.0.0.1:23456: connect:
Address family not supported by protocol family
The other beneficial effect of this change is that 'server' has no more
sender links with null address which also solves the problem from my
previous question.
Bozzo
From a296f585c95b3d2f9001e9c60e4970769581fb04 Mon Sep 17 00:00:00 2001
From: Bozo Dragojevic
Date: Sun, 3 Nov 2013 17:07:18 +0100
Subject: [PATCH 1/2] allow sending and expecting more than one reply allow
unclean exit after sending a message with expected reply
---
examples/messenger/c/recv.c | 30 +-
examples/messenger/c/send.c | 45 +++--
2 files changed, 64 insertions(+), 11 deletions(-)
diff --git a/examples/messenger/c/recv.c b/examples/messenger/c/recv.c
index fe7b153..5fac9df 100644
--- a/examples/messenger/c/recv.c
+++ b/examples/messenger/c/recv.c
@@ -40,13 +40,14 @@ void die(const char *file, int line, const char *message)
exit(1);
}
-void usage()
+void usage(const char *address)
{
printf("Usage: recv [options] \n");
printf("-c\tPath to the certificate file.\n");
printf("-k\tPath to the private key file.\n");
printf("-p\tPassword for the private key.\n");
- printf("\tAn address.\n");
+ printf("-r\treply if reply_to is set. (can be specified more than
once)\n");
+ printf("\tAn address. (%s)\n", address);
exit(0);
}
@@ -55,18 +56,21 @@ int main(int argc, char** argv)
char* certificate = NULL;
char* privatekey = NULL;
char* password = NULL;
+ int reply = 0;
char* address = (char *) "amqp://~0.0.0.0";
int c;
opterr = 0;
- while((c = getopt(argc, argv, "hc:k:p:")) != -1)
+ while((c = getopt(argc, argv, "hrc:k:p:")) != -1)
{
switch(c)
{
case 'h':
- usage();
+ usage(address);
break;
+case 'r': reply++; break;
+
case 'c': certificate = optarg; break;
case 'k': privatekey = optarg; break;
case 'p': password = optarg; break;
@@ -142,8 +146,24 @@ int main(int argc, char** argv)
printf("Address: %s\n", pn_message_get_address(message));
const char* subject = pn_message_get_subject(message);
- printf("Subject: %s\n", subject ? subject : "(no subject)");
+ subject = subject ? subject : "(no subject)";
+ printf("Subject: %s\n", subject);
printf("Content: %s\n", buffer);
+ for (int i = 0; i < reply; i++) {
+const char *reply_to = pn_message_get_reply_to(message);
+if (reply_to) {
+ printf("Reply %d/%d to %s\n", i, reply, reply_to);
+ pn_message_set_address(message, reply_to);
+ char subject_buffer[1024];
+ strcpy(subject_buffer, "Re: ");
+ strcat(subject_buffer, subject);
+ pn_message_set_subject(message, subject_buffer);
+ pn_messenger_put(messenger, message);
+ check(messenger);
+ pn_messenger_send(messenger, -1);
+ check(messenger);
+}
+ }
}
}
diff --git a/examples/messenger/c/send.c b/examples/messenger/c/send.c
index 9b969d2..e482635 100644
--- a/examples/messenger/c/send.c
+++ b/examples/messenger/c/send.c
@@ -41,10 +41
[jira] [Updated] (PROTON-290) Allow querying of unsent messages by tracker
[ https://issues.apache.org/jira/browse/PROTON-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic updated PROTON-290: --- Attachment: 0001-PROTON-290-add-a-test-and-more-explicit-logic.patch Here is a test for buffered() plus python binding update and a fixed implementation You'll need to remove all the prints I left in the patch. reason is this: I find it odd that all 100 messages immediately get a delivery! Is this really intended behavior? if this is going to remain the case, either old or new logic had the same result > Allow querying of unsent messages by tracker > > > Key: PROTON-290 > URL: https://issues.apache.org/jira/browse/PROTON-290 > Project: Qpid Proton > Issue Type: New Feature >Reporter: Bozo Dragojevic >Assignee: Rafael H. Schloming > Fix For: 0.6 > > Attachments: > 0001-PROTON-290-add-a-test-and-more-explicit-logic.patch, > 0001-PROTON-393-check-if-delivery-is-still-buffered-by-pr.patch > > > messenger.status(tracker) returns PENDING until the message is either > accepted or rejected by the recipient. > Allow distinguishing between buffered locally and transmitted messages. -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: messenger store and links
(17) [next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=0]
[0x7fbbc9840c00:0] <- @attach(18) [name="receiver-xxx", handle=0,
role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40)
[durable=0, timeout=0, dynamic=false], target=@target(41) [durable=0,
timeout=0, dynamic=false], initial-delivery-count=0]
[0x7fbbc9840c00:0] <- @flow(19) [incoming-window=2147483647,
next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0,
link-credit=1024, drain=false]
links with null address: 459 links with ok address 0
[0x7fbbc9840c00:0] -> @begin(17) [remote-channel=0, next-outgoing-id=0,
incoming-window=2147483647, outgoing-window=1]
[0x7fbbc9840c00:0] -> @attach(18) [name="receiver-xxx", handle=0,
role=false, snd-settle-mode=2, rcv-settle-mode=0, initial-delivery-count=0]
[0x7fbbc9840c00:0] -> @transfer(20) [handle=0, delivery-id=0,
delivery-tag=b"w\x00\x00\x00\x00\x00\x00\x00", message-format=0,
settled=true, more=false] (551)
"\x00Sp\xd0\x00\x00\x00\x0b\x00\x00\x00\x05BP\x04@BR\x00\x00Ss\xd0\x00\x00\x00z\x00\x00\x00\x0d@@\xa1+amqp://d80a10f2-5e82-4fa4-a3d1-03ff188bbffa@\xa1+amqp://8FC1DAF7-1783-41D7-A892-9CDC45BCAEA7@@@\x83\x00\x00\x00\x00\x00\x00\x00\x00\x83\x00\x00\x00\x00\x00\x00\x00\x00@R\x00@\x00Sw\xd0\x00\x00\x01\x8a\x00\x00\x00\x06q\x00\x00\x00\x04q\x00\x00\x00\x08`\x00\x01\xa1\x0d"...
(truncated)
links with null address: 460 links with ok address 0
[0x7fbbc9840c00:0] -> @transfer(20) [handle=0, delivery-id=1,
delivery-tag=b"x\x00\x00\x00\x00\x00\x00\x00", message-format=0,
settled=true, more=false] (551)
"\x00Sp\xd0\x00\x00\x00\x0b\x00\x00\x00\x05BP\x04@BR\x00\x00Ss\xd0\x00\x00\x00z\x00\x00\x00\x0d@@\xa1+amqp://d80a10f2-5e82-4fa4-a3d1-03ff188bbffa@\xa1+amqp://8FC1DAF7-1783-41D7-A892-9CDC45BCAEA7@@@\x83\x00\x00\x00\x00\x00\x00\x00\x00\x83\x00\x00\x00\x00\x00\x00\x00\x00@R\x00@\x00Sw\xd0\x00\x00\x01\x8a\x00\x00\x00\x06q\x00\x00\x00\x04q\x00\x00\x00\x08`\x00\x01\xa1\x0d"...
(truncated)
links with null address: 461 links with ok address 0
[0x7fbbc9840c00:0] -> @transfer(20) [handle=0, delivery-id=2,
delivery-tag=b"y\x00\x00\x00\x00\x00\x00\x00", message-format=0,
settled=true, more=false] (551)
"\x00Sp\xd0\x00\x00\x00\x0b\x00\x00\x00\x05BP\x04@BR\x00\x00Ss\xd0\x00\x00\x00z\x00\x00\x00\x0d@@\xa1+amqp://d80a10f2-5e82-4fa4-a3d1-03ff188bbffa@\xa1+amqp://8FC1DAF7-1783-41D7-A892-9CDC45BCAEA7@@@\x83\x00\x00\x00\x00\x00\x00\x00\x00\x83\x00\x00\x00\x00\x00\x00\x00\x00@R\x00@\x00Sw\xd0\x00\x00\x01\x8a\x00\x00\x00\x06q\x00\x00\x00\x04q\x00\x00\x00\x08`\x00\x01\xa1\x0d"...
(truncated)
Bozzo
p.s. I had to break pn_messenger_buffered() again or broker would not
flood the messenger with messages :)
On 24. 10. 13 22:07, Rafael Schloming wrote:
If this is with trunk I'm guessing you might be noticing a change in
reply-to behaviour from a recent fix:
https://issues.apache.org/jira/browse/PROTON-278
As you mention, previously an unset reply-to would get automatically filled
in with the messenger's name. This is no longer the case. That behaviour
was unintentional as there are times when you legitimately want the
reply-to to be left unset. The intended behaviour was to expand "~" at the
beginning of an address to the messenger's name. That is now how trunk
behaves, so if you set your reply-to's to "~" then your problem might go
away, although your question is still an interesting one as I believe if
you wished you could intentionally set up competing receivers using
explicit non-uuid addresses that collide.
--Rafael
On Thu, Oct 24, 2013 at 2:49 PM, Bozo Dragojevic wrote:
All messengers are created with default constructor (uuid-based names).
The 'broker' messenger does a pn_messenger_subscribe("amqp:/**/~
0.0.0.0:8194")
All messages are constructed with address amqp://127.0.0.1:8194 and leave
reply_to unset (so it's set to amqp://$uuid)
Broker does application-level routing of messages
a publisher sends a special 'register' message
replies are constructed using stored 'reply_to' address from incoming
message
forwarded messages are constructed using stored 'reply_to' address from
incoming 'registration' messages
Messenger routing facility is not used in any way.
All Messengers are running in async mode (broker and client library share
the same 'event loop code').
We're using outgoing trackers, mostly for the 'buffered' check
All incoming messages are accepted as soon as they are processed.
All outgoing messages are settled as soon as they are not buffered anymore
maybe it'd be possible to simulate the situation by commenting out the
pni_pump_out() in pn_messenger_put(), that, or at least checking if sender
link address really has anything to do with
the address calculated in pn_messenger_put()
Bozzo
On 24. 10.
Re: messenger store and links
All messengers are created with default constructor (uuid-based names).
The 'broker' messenger does a pn_messenger_subscribe("amqp://~0.0.0.0:8194")
All messages are constructed with address amqp://127.0.0.1:8194 and
leave reply_to unset (so it's set to amqp://$uuid)
Broker does application-level routing of messages
a publisher sends a special 'register' message
replies are constructed using stored 'reply_to' address from incoming
message
forwarded messages are constructed using stored 'reply_to' address
from incoming 'registration' messages
Messenger routing facility is not used in any way.
All Messengers are running in async mode (broker and client library
share the same 'event loop code').
We're using outgoing trackers, mostly for the 'buffered' check
All incoming messages are accepted as soon as they are processed.
All outgoing messages are settled as soon as they are not buffered anymore
maybe it'd be possible to simulate the situation by commenting out the
pni_pump_out() in pn_messenger_put(), that, or at least checking if
sender link address really has anything to do with
the address calculated in pn_messenger_put()
Bozzo
On 24. 10. 13 20:25, Rafael Schloming wrote:
Can you post the exact addresses and routing configuration you're using and
which direction messages are flowing? I'd like to try to simulate this
using the example send/recv scripts.
My guess is that the issue may not be so much related to whether the
addresses are NULL or not but whether there are multiple receivers
competing for the same messages.
--Rafael
On Thu, Oct 24, 2013 at 11:52 AM, Bozo Dragojevicwrote:
Hi!
Chasing down a weird behavior...
looking at messengers pni_pump_out() and how it's used from
pn_messenger_endpoints()
link = pn_link_head(conn, PN_LOCAL_ACTIVE | PN_REMOTE_ACTIVE);
while (link) {
if (pn_link_is_sender(link)) {
pni_pump_out(messenger, pn_terminus_get_address(pn_**link_target(link)),
link);
is it really fair to assume that target address is always expected to be
non NULL?
I've added a bit of debug code to pn_messenger_endpoints() so it reads:
link = pn_link_head(conn, PN_LOCAL_ACTIVE | PN_REMOTE_ACTIVE);
while (link) {
if (pn_link_is_sender(link)) {
static int addrnull, addrok;
const char *address = pn_terminus_get_address(pn_**
link_target(link));
if (!address) {
addrnull++;
} else {
addrok++;
}
fprintf(stderr, "links with null address: %d links with ok address
%d\n",
addrnull, addrok);
pni_pump_out(messenger, address, link);
and I never see 'addrok' change from 0
when pni_pump_out is called with address==NULL:
int pni_pump_out(pn_messenger_t *messenger, const char *address, pn_link_t
*sender)
{
pni_entry_t *entry = pni_store_get(messenger->**outgoing, address);
if (!entry) return 0;
pni_store_get cheerfuly returns first message on the list
end effect is that random clients start receiving messages not directed at
them.
For some inexplicable reason is mostly works out while there are just two
clients
connected to the messenger and we're not pushing it really hard. Still
trying to come
up with a simple test-case.
Can anyone shed some light how the addressing on the link level is
supposed to work in mesenger?
Bozzo
[jira] [Commented] (PROTON-290) Allow querying of unsent messages by tracker
[ https://issues.apache.org/jira/browse/PROTON-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804507#comment-13804507 ] Bozo Dragojevic commented on PROTON-290: Sounds fair :) > Allow querying of unsent messages by tracker > > > Key: PROTON-290 > URL: https://issues.apache.org/jira/browse/PROTON-290 > Project: Qpid Proton > Issue Type: New Feature > Reporter: Bozo Dragojevic >Assignee: Rafael H. Schloming > Fix For: 0.6 > > Attachments: > 0001-PROTON-393-check-if-delivery-is-still-buffered-by-pr.patch > > > messenger.status(tracker) returns PENDING until the message is either > accepted or rejected by the recipient. > Allow distinguishing between buffered locally and transmitted messages. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Reopened] (PROTON-290) Allow querying of unsent messages by tracker
[ https://issues.apache.org/jira/browse/PROTON-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bozo Dragojevic reopened PROTON-290: > Allow querying of unsent messages by tracker > > > Key: PROTON-290 > URL: https://issues.apache.org/jira/browse/PROTON-290 > Project: Qpid Proton > Issue Type: New Feature > Reporter: Bozo Dragojevic >Assignee: Rafael H. Schloming > Fix For: 0.6 > > Attachments: > 0001-PROTON-393-check-if-delivery-is-still-buffered-by-pr.patch > > > messenger.status(tracker) returns PENDING until the message is either > accepted or rejected by the recipient. > Allow distinguishing between buffered locally and transmitted messages. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (PROTON-290) Allow querying of unsent messages by tracker
[
https://issues.apache.org/jira/browse/PROTON-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804488#comment-13804488
]
Bozo Dragojevic commented on PROTON-290:
I think I just spotted a thinko in my logic. Buffered should return true if we
do not have a delivery yet or if delivery is still buffered:
{code}
-return d && pn_delivery_buffered(d);
+return !d || pn_delivery_buffered(d);
{code}
> Allow querying of unsent messages by tracker
>
>
> Key: PROTON-290
> URL: https://issues.apache.org/jira/browse/PROTON-290
> Project: Qpid Proton
> Issue Type: New Feature
> Reporter: Bozo Dragojevic
>Assignee: Rafael H. Schloming
> Fix For: 0.6
>
> Attachments:
> 0001-PROTON-393-check-if-delivery-is-still-buffered-by-pr.patch
>
>
> messenger.status(tracker) returns PENDING until the message is either
> accepted or rejected by the recipient.
> Allow distinguishing between buffered locally and transmitted messages.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
