[jira] [Assigned] (PROTON-546) C++ Windows warnings: conversions and possible loss of data
[ https://issues.apache.org/jira/browse/PROTON-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming reassigned PROTON-546: -- Assignee: Rafael H. Schloming > C++ Windows warnings: conversions and possible loss of data > --- > > Key: PROTON-546 > URL: https://issues.apache.org/jira/browse/PROTON-546 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.6 >Reporter: Chuck Rolke >Assignee: Rafael H. Schloming > Attachments: PROTON-546-size-warnings.txt > > > Windows has other warnings similar to PROTON-545, especially during 64-bit > builds. > See attached file for details. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (PROTON-579) Typo in proton.php use of $self should be $this
[ https://issues.apache.org/jira/browse/PROTON-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming reassigned PROTON-579: -- Assignee: Rafael H. Schloming > Typo in proton.php use of $self should be $this > --- > > Key: PROTON-579 > URL: https://issues.apache.org/jira/browse/PROTON-579 > Project: Qpid Proton > Issue Type: Bug > Components: php-binding >Affects Versions: 0.7 > Environment: n/a >Reporter: Robert Nicholson >Assignee: Rafael H. Schloming >Priority: Trivial > > On line 289 of /qpid/proton/trunk/proton-c/bindings/php/proton.php there is a > reference to $self that should be $this > This results in the following warning: > PHP Warning: Creating default object from empty value in > /home/nicholsr/bm2/worker.php/proton.php on line 289 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (PROTON-546) C++ Windows warnings: conversions and possible loss of data
[ https://issues.apache.org/jira/browse/PROTON-546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-546: --- Assignee: Andrew Stitcher (was: Rafael H. Schloming) > C++ Windows warnings: conversions and possible loss of data > --- > > Key: PROTON-546 > URL: https://issues.apache.org/jira/browse/PROTON-546 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.6 >Reporter: Chuck Rolke >Assignee: Andrew Stitcher > Attachments: PROTON-546-size-warnings.txt > > > Windows has other warnings similar to PROTON-545, especially during 64-bit > builds. > See attached file for details. -- This message was sent by Atlassian JIRA (v6.2#6252)
big improvement in memory usage for proton address scale-up
In my original testing for address scale-up with a Proton Messenger based client, I was measuring a memory cost of 115 KB per subscribed address in the client. Now, after Rafi's recent changes, I am seeing a better than 7x improvement, to just under 16 KB per subscribed address. The downside, of course, is that this will make it about 7x harder for me to persuade my boss to buy me a Really Big Box. ( But I'll think of something... ) Thanks for the memory!
Re: delivery.setPayload
On May 13, 2014, at 1:46 PM, Rafael Schloming wrote: > I'm not sure this will work as an API. One problem I see with this is that > it is circumventing the engine's flow control logic. If you notice there is > logic inside send() to update counters on the session. Unless I've missed > something your patch doesn't seem to have equivalent logic. This might just > be an oversight, but I don't see how you could easily add the same logic > since you don't know how many bytes the payload is until much much later on > in the control flow of the engine. > as I told you this was just a prototyped idea... it's not in fact updating the window yet.. If this idea is a good idea, we could pursue the idea here. > Can you supply some more detail as to why it got 5% faster? If it was > merely avoiding the copy, then I can think of some ways we could avoid that > copy without changing the API quite so drastically, e.g. just overload > send() to take some sort of releasable buffer reference. The encoding is done directly the the FrameWriter::__outputBuffer. I've made a framework where I'm testing the send and it made it somewhat fast than copying the encoding over 1 million messages. On this case it could be a bit more if you encoded a MesasgeImpl on a new buffer every time > > FWIW, I think that a good buffer abstraction that we could use everywhere > would help a lot. I suspect having distinct abstractions for payload > buffers vs encodable buffers vs decodable buffers is just going to result > in lots of unnecessary conversions. probably.. I was just trying to improve the idea of the payloads. I don't like the send API right now.. I think it would make more sense to set the payload on the delivery than send bytes through sender.
[jira] [Updated] (PROTON-562) More gracefully handle connections to non-1.0 parties
[
https://issues.apache.org/jira/browse/PROTON-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rafael H. Schloming updated PROTON-562:
---
Assignee: Andrew Stitcher
> More gracefully handle connections to non-1.0 parties
> -
>
> Key: PROTON-562
> URL: https://issues.apache.org/jira/browse/PROTON-562
> Project: Qpid Proton
> Issue Type: Improvement
>Affects Versions: 0.7
>Reporter: Justin Ross
>Assignee: Andrew Stitcher
>Priority: Minor
>
> Right now, attempting a connection from messenger to a 0-10 only qpidd
> results in a "SASL header mismatch". That's correct but less helpful than it
> might be.
> {noformat}
> jross@localhost bailey$ LD_LIBRARY_PATH=/tmp/tmp.mXaVrrHF8a/lib64
> PYTHONPATH=/tmp/tmp.mXaVrrHF8a/lib64/proton/bindings/python/ PN_TRACE_FRM=1
> python ~/test.py
> [0x2418e30]: -> SASL
> [0x2418e30]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS, initial-response=b""]
> [0x2418e30]:ERROR[-2] SASL header mismatch: 'AMQP\x01\x01\x00\x0a'
>
> [0x2418e30]: <- EOS
> [0x2418e30]:ERROR[-2] SASL header mismatch: ''
>
> [0x2418e30]: <- EOS
> CONNECTION ERROR connection aborted (remote)
> {noformat}
> See
> https://issues.apache.org/jira/browse/PROTON-561?focusedCommentId=13966905&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13966905
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (PROTON-558) Make friendly protocol field logging optional for low-memory devices
[ https://issues.apache.org/jira/browse/PROTON-558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-558: --- Assignee: Andrew Stitcher > Make friendly protocol field logging optional for low-memory devices > > > Key: PROTON-558 > URL: https://issues.apache.org/jira/browse/PROTON-558 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.6, 0.7 >Reporter: Markus Horstmann >Assignee: Andrew Stitcher > > Embedded devices tend to be memory constrained (i.e. 128KB RAM). The FIELDS > structure used for protocol logging consumes significant memory. Making it > optional would help reduce the memory footprint. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PROTON-562) More gracefully handle connections to non-1.0 parties
[
https://issues.apache.org/jira/browse/PROTON-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996779#comment-13996779
]
Rafael H. Schloming commented on PROTON-562:
See the comments in PROTON-561 for more detailed thoughts on how to improve
things.
> More gracefully handle connections to non-1.0 parties
> -
>
> Key: PROTON-562
> URL: https://issues.apache.org/jira/browse/PROTON-562
> Project: Qpid Proton
> Issue Type: Improvement
>Affects Versions: 0.7
>Reporter: Justin Ross
>Assignee: Andrew Stitcher
>Priority: Minor
>
> Right now, attempting a connection from messenger to a 0-10 only qpidd
> results in a "SASL header mismatch". That's correct but less helpful than it
> might be.
> {noformat}
> jross@localhost bailey$ LD_LIBRARY_PATH=/tmp/tmp.mXaVrrHF8a/lib64
> PYTHONPATH=/tmp/tmp.mXaVrrHF8a/lib64/proton/bindings/python/ PN_TRACE_FRM=1
> python ~/test.py
> [0x2418e30]: -> SASL
> [0x2418e30]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS, initial-response=b""]
> [0x2418e30]:ERROR[-2] SASL header mismatch: 'AMQP\x01\x01\x00\x0a'
>
> [0x2418e30]: <- EOS
> [0x2418e30]:ERROR[-2] SASL header mismatch: ''
>
> [0x2418e30]: <- EOS
> CONNECTION ERROR connection aborted (remote)
> {noformat}
> See
> https://issues.apache.org/jira/browse/PROTON-561?focusedCommentId=13966905&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13966905
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (PROTON-541) Problems with multi-frame deliveries
[ https://issues.apache.org/jira/browse/PROTON-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafael H. Schloming updated PROTON-541: --- Assignee: Andrew Stitcher > Problems with multi-frame deliveries > > > Key: PROTON-541 > URL: https://issues.apache.org/jira/browse/PROTON-541 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.6 >Reporter: Ted Ross >Assignee: Andrew Stitcher > > I made a modification to messenger.c to configure the maximum frame size on > transports (there is no API access to this configuration) and then ran a > point-to-point test between a sender and a passive receiver using messages > larger than the frame max. > The receiver crashes with the following assertion after several dozen > messages are transferred successfully. > python: /home/ross/svn/proton/proton-c/src/message/message.c:653: > pn_message_decode: Assertion `msg && bytes && size' failed. > Another observation: > When the max-frame-size is specified only by one side of the connection, > there is no negotiation. Frames larger than the specified max are > transferred. It is not clear from the specification (2.7.1) whether > max-frame-size is intended to be symmetric and apply to transfers in both > directions on the connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: delivery.setPayload
I'm not sure this will work as an API. One problem I see with this is that it is circumventing the engine's flow control logic. If you notice there is logic inside send() to update counters on the session. Unless I've missed something your patch doesn't seem to have equivalent logic. This might just be an oversight, but I don't see how you could easily add the same logic since you don't know how many bytes the payload is until much much later on in the control flow of the engine. Can you supply some more detail as to why it got 5% faster? If it was merely avoiding the copy, then I can think of some ways we could avoid that copy without changing the API quite so drastically, e.g. just overload send() to take some sort of releasable buffer reference. FWIW, I think that a good buffer abstraction that we could use everywhere would help a lot. I suspect having distinct abstractions for payload buffers vs encodable buffers vs decodable buffers is just going to result in lots of unnecessary conversions. --Rafael On Tue, May 13, 2014 at 11:19 AM, Clebert Suconic wrote: > I have been playing with the API, and there is one change that would make > the API clearer IMO. > > > Right now you have to send a delivery, and then call send(bytes) to add > Bytes to the delivery what will make it copy data to the Delivery and self > expand the buffer. > > > > I have played with a change that made it 5% faster than the most optimal > way to expand the payload on the Delivery (using the same buffer over and > over) > > > And 15% on a brief calculation against creating the buffer every time... > but there are cases where this could be a bit worse. > > > > Basically I have created an interface called Payload, and added a method > setPayload on Delivery. > > > > I'm not sure yet how I would implement framing into multiple packages.. > but I think it could be done.. this is just a prototyped idea: > > > > https://github.com/clebertsuconic/qpid-proton/commit/02abe61fc54911955ddcce77b792a153c5476aef > > > > in case you want to fetch the buffer from my git, it's this branch: > https://github.com/clebertsuconic/qpid-proton/tree/payload > > > > In any case I liked the idea of the setPayload better than > sender.send(bytes) to set the payload of a message. > > > > Ideas?
[jira] [Commented] (PROTON-579) Typo in proton.php use of $self should be $this
[ https://issues.apache.org/jira/browse/PROTON-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996702#comment-13996702 ] ASF subversion and git services commented on PROTON-579: Commit 1594309 from r...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1594309 ] PROTON-579: fixed typo in proton.php > Typo in proton.php use of $self should be $this > --- > > Key: PROTON-579 > URL: https://issues.apache.org/jira/browse/PROTON-579 > Project: Qpid Proton > Issue Type: Bug > Components: php-binding >Affects Versions: 0.7 > Environment: n/a >Reporter: Robert Nicholson >Assignee: Rafael H. Schloming >Priority: Trivial > > On line 289 of /qpid/proton/trunk/proton-c/bindings/php/proton.php there is a > reference to $self that should be $this > This results in the following warning: > PHP Warning: Creating default object from empty value in > /home/nicholsr/bm2/worker.php/proton.php on line 289 -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: codec strategies
On Thu, May 8, 2014 at 9:42 AM, Alan Conway wrote: > I vote for DispatchingDecode: it's the simplest, the fastest and is > based on a well established parsing pattern with a good track record for > performance (SAX). Its not so hard to ignore data in a handler. > > Writing a handler state machine is a bit more complex than writing a > sequence of calls to a stream API, but I think you could encapsulate > most of a standard state machine that given a sequence of type codes > will fill a sequence of variables. Not sure about the right way to do > that in Java performance-wise. > > Hmm. That might be worth another performance test though - if you did > have such tools for making it easy to build handlers, would those tools > introduce a penalty that would make the StreamingDecode look more > attractive... > The biggest difference from an API perspective has to do with data conversions/coersion. Say you're writing a piece of Java code that wants to operate on Java integer or a Java float and doesn't care what the underlying wire type is so long as it can be reasonable converted to that type. In a stream style API you would simple write: int i = decoder.getInt(); float f = decoder.getFloat(); The decoder implementation itself can the be smart enough to convert whatever underlying wire type there might be into the appropriate java type. The SAX API on the other hand will have a distinct callback for byte vs ubyte vs short vs ushort, etc, and it could be quite cumbersome to convert all the different possibilities into the type you actually want to operate on. Put another way, the stream style API is capable of incorporating the desired output type of the user, whereas the SAX style API is not. It might be possible to provide some kind of coercing handler that would help the SAX situation. As you say though it's probably worth checking that something like that would be usable and not introduce its own penalties. --Rafael
delivery.setPayload
I have been playing with the API, and there is one change that would make the API clearer IMO. Right now you have to send a delivery, and then call send(bytes) to add Bytes to the delivery what will make it copy data to the Delivery and self expand the buffer. I have played with a change that made it 5% faster than the most optimal way to expand the payload on the Delivery (using the same buffer over and over) And 15% on a brief calculation against creating the buffer every time... but there are cases where this could be a bit worse. Basically I have created an interface called Payload, and added a method setPayload on Delivery. I'm not sure yet how I would implement framing into multiple packages.. but I think it could be done.. this is just a prototyped idea: https://github.com/clebertsuconic/qpid-proton/commit/02abe61fc54911955ddcce77b792a153c5476aef in case you want to fetch the buffer from my git, it's this branch: https://github.com/clebertsuconic/qpid-proton/tree/payload In any case I liked the idea of the setPayload better than sender.send(bytes) to set the payload of a message. Ideas?
[jira] [Commented] (PROTON-573) proton-c: Messenger appears to have hard-coded address limits of 1024
[ https://issues.apache.org/jira/browse/PROTON-573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992836#comment-13992836 ] ASF subversion and git services commented on PROTON-573: Commit 1593307 from r...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1593307 ] PROTON-573: removed hardcoded address limits in messenger > proton-c: Messenger appears to have hard-coded address limits of 1024 > - > > Key: PROTON-573 > URL: https://issues.apache.org/jira/browse/PROTON-573 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.7 >Reporter: Dominic Evans > Attachments: 06_arbitrary_length_addresses_in_store.patch, > 07_arbitrary_length_addresses_in_messenger.patch > > > The AMQP 1.0 spec permits pretty much arbitrary length link names, but > Messenger hard-codes some 1K limits in various places. -- This message was sent by Atlassian JIRA (v6.2#6252)
