[jira] [Created] (PROTON-803) Message codec improvements

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-803:
---

 Summary: Message codec improvements
 Key: PROTON-803
 URL: https://issues.apache.org/jira/browse/PROTON-803
 Project: Qpid Proton
  Issue Type: Improvement
  Components: proton-j
Affects Versions: 0.8
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu
 Fix For: 0.9


This JIRA covers improvements made to the codec, especially lists and maps.

The work is based on 
https://github.com/rhs/qpid-proton-old/tree/codec/proton-j/src/main/java/org/apache/qpid/proton/codec2

The above referenced code, is quite an improvement with respect to writing 
lists, maps and strings over the existing codec.

Simply put the writeList and writeMap methods in the old encorder is about ~10 
times slower than the new encorder.
If I run with a sufficiently large set of strings, the old encorder is about ~2 
times slower than the new encorder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PROTON-594) In tree builds with cmake causes issues when running python based tests

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018873#comment-14018873
 ] 

Rajith Attapattu commented on PROTON-594:
-

Rafi could you please create a new component for the cmake build system.
I just temporarily assigned this JIRA to proton-c to create the JIRA as it 
doesn't allow to leave it blank.

> In tree builds with cmake causes issues when running python based tests
> ---
>
> Key: PROTON-594
> URL: https://issues.apache.org/jira/browse/PROTON-594
> Project: Qpid Proton
>  Issue Type: Bug
>  Components: proton-c
>    Reporter: Rajith Attapattu
>Assignee: Rafael H. Schloming
> Fix For: 0.8
>
>
> If we do an in-tree build it causes path issues when trying to run the python 
> based tests against the java code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (PROTON-594) In tree builds with cmake causes issues when running python based tests

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-594:
---

 Summary: In tree builds with cmake causes issues when running 
python based tests
 Key: PROTON-594
 URL: https://issues.apache.org/jira/browse/PROTON-594
 Project: Qpid Proton
  Issue Type: Bug
  Components: proton-c
Reporter: Rajith Attapattu
Assignee: Rafael H. Schloming
 Fix For: 0.8


If we do an in-tree build it causes path issues when trying to run the python 
based tests against the java code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (PROTON-589) Implement passive mode for proton-j messenger

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-589:
---

 Summary: Implement passive mode for proton-j messenger
 Key: PROTON-589
 URL: https://issues.apache.org/jira/browse/PROTON-589
 Project: Qpid Proton
  Issue Type: Improvement
  Components: proton-j
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu
 Fix For: 0.8


Passive mode allows the file descriptors for messenger to be serviced by an 
external loop.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Using Proton 0.7 to work with Qpid 0.26

[Rajith Attapattu]

Mark,

Nothing is wrong with your code.
The issue is down to a difference in how SASL is handled in the Proton java
side and the c++ broker.
If I comment out the SASL code in the messenger impl your example works
properly.
I have seen this issue before and will investigate it further.

Rajith


On Thu, May 22, 2014 at 1:29 PM, Mark Soderquist  wrote:

> I am struggling to get Proton 0.7 to send any messages to a Qpid 0.26
> broker. I have been able to send and receive message using the Qpid
> Messaging API for AMQP 1.0 but would rather use Proton if I can. I'm sure
> I'm "just" missing something but I haven't found it yet. Here is my code:
>
> Messenger messenger = Proton.messenger( "proton" );
> try {
>   messenger.start();
>
>   Message message = Proton.message();
>   message.setAddress( "amqp://testuser@qpid/pocqueue" );
>   message.setBody( new AmqpValue( "test message" ) );
>
>   messenger.put( message );
>   messenger.send();
> }
> catch( Throwable throwable ) {
>   throwable.printStackTrace( System.err );
> }
> finally {
>   messenger.stop();
> }
>
> Any suggestions?
>
>
>  NOTICE: This email message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information. Any
> unauthorized review, use, disclosure or distribution is prohibited. If you
> are not the intended recipient, please contact the sender by reply email
> and destroy all copies of the original message.
>
>

Test (eom)

[Rajith Attapattu]

[jira] [Updated] (PROTON-565) Modify the Messenger to use the Collector API.

[Rajith Attapattu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PROTON-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajith Attapattu updated PROTON-565:


Attachment: PROTON-565.part1.patch

> Modify the Messenger to use the Collector API.
> --
>
> Key: PROTON-565
> URL: https://issues.apache.org/jira/browse/PROTON-565
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: proton-j
>    Reporter: Rajith Attapattu
>Assignee: Rajith Attapattu
> Fix For: 0.7
>
> Attachments: PROTON-565.part1.patch
>
>
> Modify the Messenger to use the Collector API instead of querying the for 
> state changes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (PROTON-565) Modify the Messenger to use the Collector API.

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13971420#comment-13971420
 ] 

Rajith Attapattu commented on PROTON-565:
-

I had to mark the fix version as 0.7 as there is no 0.8 tag yet.
We should create a "future" tag to get around this issue.

> Modify the Messenger to use the Collector API.
> --
>
> Key: PROTON-565
> URL: https://issues.apache.org/jira/browse/PROTON-565
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: proton-j
>Reporter: Rajith Attapattu
>Assignee: Rajith Attapattu
> Fix For: 0.7
>
>
> Modify the Messenger to use the Collector API instead of querying the for 
> state changes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (PROTON-565) Modify the Messenger to use the Collector API.

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-565:
---

 Summary: Modify the Messenger to use the Collector API.
 Key: PROTON-565
 URL: https://issues.apache.org/jira/browse/PROTON-565
 Project: Qpid Proton
  Issue Type: Improvement
  Components: proton-j
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu
 Fix For: 0.7


Modify the Messenger to use the Collector API instead of querying the for state 
changes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (PROTON-543) Frame Parser error if input stream is read before SASL is initialized in the transport

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-543:
---

 Summary: Frame Parser error if input stream is read before SASL is 
initialized in the transport
 Key: PROTON-543
 URL: https://issues.apache.org/jira/browse/PROTON-543
 Project: Qpid Proton
  Issue Type: Bug
  Components: proton-j
Affects Versions: 0.6, 0.5
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu


If the input stream is read and passed into the transport, before the sasl() 
method of TransportImpl.java is called then the _inputProcessor defaults to 
FrameParser instead of being wrapped by the SASL frame parser.
This causes a Frame Parsing error as it expects '0' as per the regular header 
but instead finds '3' which is the correct format if the process is the SASL 
frame parser.

As per the discussion on @proton we decide to throw an exception if sasl is set 
after the input stream has been processed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Race condition in the TransportImpl in Proton-J

[Rajith Attapattu]

I agree with you.
I will create a JIRA and attach a patch for you to review.

It also appears the Frame Parser errors was swallowed before it made to the
application layer.
I will take a closer look and create a JIRA and patch if necessary.

Rajith


On Tue, Mar 25, 2014 at 6:47 AM, Rafael Schloming  wrote:

> On Mon, Mar 24, 2014 at 9:37 PM, Rajith Attapattu  >wrote:
>
> > I encountered an issue in Proton J which I believe is a race condition.
> > If the input stream is read and passed into the transport, before the
> > sasl() method of TransportImpl.java is called then the _inputProcessor
> > defaults to FrameParser instead of being wrapped by the SASL frame
> parser.
> > This causes a Frame Parsing error as it expects '0' as per the regular
> > header but instead finds '3' which is the correct format if the process
> is
> > the SASL frame parser.
> >
> > We should either test the incoming header and determine the right
> > inputProcessor
> > OR
> > clearly document that the transport needs to be ready (i.e the sasl()
> > method has to be called) before any incoming data is fed to the
> transport.
> >
>
> I'd say we should do the latter. In fact we should probably throw an
> exception if you attempt to configure a sasl layer after input has occurred
> since there is never anything sensible we can do if you try to configure
> the sasl layer "midstream". It all needs to be set up prior to actually
> processing any input bytes.
>
> --Rafael
>

Race condition in the TransportImpl in Proton-J

[Rajith Attapattu]

I encountered an issue in Proton J which I believe is a race condition.
If the input stream is read and passed into the transport, before the
sasl() method of TransportImpl.java is called then the _inputProcessor
defaults to FrameParser instead of being wrapped by the SASL frame parser.
This causes a Frame Parsing error as it expects '0' as per the regular
header but instead finds '3' which is the correct format if the process is
the SASL frame parser.

We should either test the incoming header and determine the right
inputProcessor
OR
clearly document that the transport needs to be ready (i.e the sasl()
method has to be called) before any incoming data is fed to the transport.

Rajith

Logging situation with proton-j

[Rajith Attapattu]

I would like to get logging working with the protocol engine.
Does anybody know the current status and how to get trace level logging
going on ?

Regards,

Rajith

Re: proton-j API factory simplification.

[Rajith Attapattu]

>
> So, I'd be in favour of Hiram's proposal if ProtonJ and ProtonC reside in
> proton-api.jar.  This would be very easy to do, e.g.
>
>
I don't think ProtonJ and ProtonC should reside in the proton-api.jar
And I don't think thats what Hiram suggested either (pls correct me if I
have misunderstood).

Only the Proton interface and supported interfaces needs to be there in
proton-api.jar.
ProtonJ, ProtonC, or any other implementation will be in the respective
implementation jars.

Apart from testing, I doubt most people will want to switch between
implementations the way we have them now.
For most it would be an informed decision of choosing a particular
implementation after weighing the pros and cons of it.

Rajith

Re: RFC: new routing functionality for messenger

[Rajith Attapattu]

On Mon, Mar 25, 2013 at 11:42 AM, Rafael Schloming  wrote:
> On Mon, Mar 25, 2013 at 10:34 AM, Rajith Attapattu wrote:
>
>> For starters I would copy this email to the user list.
>> (In the future we should post things like this to a more wider
>> audience, especially if we are looking for feedback based on real
>> world use cases.)
>>
>
> I posted it where I intended it to go. I don't mind a broader audience
> looking at it, but at this stage I'm really looking for feedback from
> proton developers and people more keyed into messenger's distributed
> messaging model. This is for two reasons: (1) I don't want to push out
> stuff and have people start using it before it is sufficiently reviewed by
> other proton developers, and (2) a lot of the people on the users list
> (perhaps most of them) are quite firmly entrenched in the traditional hub
> and spoke broker model, and from their perspective the primary use for this
> functionality would simply be to route("*", "my-giant-hub-broker/$1") and
> thereby make messenger simulate your traditional hub and spoke client,
> however this functionality is about way more than that.
>
> I think after a round of discussion and documentation (not just API docs)
> it would probably be more digestable to a broader audience. Feel free to
> kick off wider discussion if you wish though, I've no objection, I'm just
> explaining my thinking at this stage.

Thanks for your explanation, and you do have a point about people
starting to use the feature a bit prematurely and perhaps in a
narrower scope than what we would have liked.
Unfortunately the same holds true if we publish it even after we deem
it's ready for general consumption.
Changing the mindset especially after years of being conditioned to a
particular approach is going to be difficult.
The challenge for Proton is to educate our users and the general
development community that we can do much more than the hub-spoke
model.

My desire to bring the discussion to a more wider audience was
motivated by recent comments/observations by Fraser and Gordon.
Within our user base, we do seem to have a few early adopters who
seems keen to be a bit more involved in the development process than a
regular user would.
Perhaps a more reasonable approach would be to copy dev@ instead of
user@ hoping the above crowd is on @dev.

>
>> I actually like the minimalistic approach you've taken here. It works
>> well in an embedded context.
>> Support for regex will be a welcome addition.
>> I don't think we should bother with a config file for proton. To me
>> it's neither here nor there. Let the application developer or the
>> deployment folks who use the proton based application worry about it.
>> As both you and Ted pointed out, a more sophisticated routing
>> capability is best achieved outside the scope of Proton.
>>
>
> I agree it's not the job of a library to provide system level
> integration/configuration, however we should do what we can to make it easy
> for system integrators to do so themselves and, to the extent that we can,
> encourage them to provide as consistent an experience as possible across
> systems, e.g. I considered plugging into system provided regex libraries
> for providing regular expression support, but that could result in slightly
> different styles of regex depending on which system you're on, so I decided
> against that option. (Not saying I don't want to provide regex support,
> just that if we do we should find a way that works the same regardless of
> where you are.)

Thats a fair point, I agree with you.

> Regarding your statement on sophisticated routing capabilities being kept
> outside of proton, I think it's worth pointing out that unless you're
> considering just the hub and spoke topology, routing isn't something that
> is contained to a single component or even a single class of component in
> your network. It's something that everything, including the endpoints in
> your network need to be aware of and cooperatively participate in so that
> you can achieve a given overall effect.

This is indeed what I meant, perhaps I should have elaborated a bit more.
As you said, for complicated networks, the routing aspect will be
de-centralized and achieved by the collaboration/cooperation of all
kinds of nodes/services within the network that may span
organizations.
Therefore it's impossible for us to provide a comprehensive solution
through proton, or other components (routers, bridges, brokers ..etc)
we offer through the larger QPid umbrella.
What we can provide is some simple basic building blocks/helper
utilities that will help the end users to compose more larger complex
routing strategies.

In that regard what you've offered seems quite good. Anything more
will be neither here nor there.

Rajith

> --Rafael

Re: RFC: new routing functionality for messenger

[Rajith Attapattu]

For starters I would copy this email to the user list.
(In the future we should post things like this to a more wider
audience, especially if we are looking for feedback based on real
world use cases.)

I actually like the minimalistic approach you've taken here. It works
well in an embedded context.
Support for regex will be a welcome addition.
I don't think we should bother with a config file for proton. To me
it's neither here nor there. Let the application developer or the
deployment folks who use the proton based application worry about it.
As both you and Ted pointed out, a more sophisticated routing
capability is best achieved outside the scope of Proton.

Regards,

Rajith

On Mon, Mar 25, 2013 at 9:30 AM, Rafael Schloming  wrote:
> On Mon, Mar 25, 2013 at 8:36 AM, Ted Ross  wrote:
>
>> I generally like this translation facility, though I'll need to think
>> through some use cases to make sure there's sufficient capability in the
>> matching and substitution.
>>
>
> I would definitely appreciate some more feedback on this area. I think some
> real world usage will help us out. I struggled with various options here. I
> didn't want to add a dependency onto a regular expression library without
> at least some kind of fallback as I think this functionality is going to
> end up being too useful to be optional.
>
> There are a number of public domain regular expression libraries out there
> (e.g. t-rex) that are basically designed to be small enough and simple
> enough to be sucked into a codebase wholesale for when you can't afford an
> external dependency. I considered grabbing one of those, but I also wasn't
> 100% sure that full on regex was the best/only option here. There is
> something nice and simple about being able to say route("*", "my-uplink")
> as opposed to writing a fullon regex.
>
> I can see possibly even wanting to support multiple syntax options
> eventually, but for starters I decided to keep it simple. I wrote a very
> small glob style matcher with substitution. It's roughly 100 lines of code
> and should work on any platform. I'm entirely open to replacing this with
> something a real regex library if we want or something else entirely, I
> just wanted something simple/portable to start with so we could actually
> play with the functionality and learn a bit more about what is appropriate.
>
>
>> I question the value of having this as part of the API.  It seems to me
>> that the Messenger routing table should be part of the configuration of a
>> system (i.e. in a place like /etc/qpid/messenger/, or the Windows Registry,
>> etc.).  Eventually, it might be desirable to have an address translation
>> service that can be accessed remotely.  I guess I see this translation as
>> being not the concern of the application, but more of the deployment of the
>> application.
>>
>>
> Initially I was actually going to add the functionality via some kind of
> configuration file as you say. I have a similar conceptual view of the
> functionality as you describe, i.e. it really should only be used as part
> of the deployment/configuration of the messenger, not part of the
> "application work" portion of the API. I ended up going this route for two
> reasons (no pun intended). For starters it was a bit more expedient as I
> didn't have to write some kind of file format parser, but the second, and
> perhaps more important reason, is that I wasn't sure of the implications of
> adding dependencies onto files/file systems. I want messenger to be usable
> from embedded devices that might not even have a file system, so I figured
> it would be safer to offer as a pure API. I do think it would be
> appropriate to layer some kind of system integration on top of this API
> just as you describe.
>
> --Rafael

Re: Yet Another communication improvement suggestion

[Rajith Attapattu]

Phil,

I don't think what you suggested is against the spirit of open source.
As a project we certainly need to think about how to better
communicate among us and also with our user base.
A number of users have voiced their concerns about not knowing major
changes and plans in a timely manner.

We could ask the developers to send an email with,
1. A brief summary of what they are planing to do in the short term.
2. Expected start and end dates (a rough estimation)
3. Any JIRA's associated with the work .. we can update as we go along

And then one person (Sir, looks like you volunteered) can collect
these summaries into a wiki page of some sort.
(Ideally each contributor should do this themselves, but chances are
that it will not happen, so we need a few folks to drive this).

We should also email a summary of this page to our user list every two
weeks or so.
This can also be used as a basis for our quarterly reports.

Regards,

Rajith

On Tue, Mar 12, 2013 at 1:22 PM, Phil Harvey  wrote:
> There is a lot of really exciting development being done on Proton at the
> moment.  However, I often wish that I had better visibility of ongoing
> work, so that I could better complement the work others are doing.
>
> Currently, the ways I find out about this work are:
>
> - Jira updates
> - The mailing list
> - IRC
>
> There are two problems with this: (1) I only get a partial view of what's
> going on, and (2) stuff usually gets put on Jira and the mailing list too
> late, i.e. when it's already in progress or is actually finished.
>
> Also, we do have a roadmap on the wiki [1], but I don't think this is used
> by many people at the moment.
>
> Maybe my desire for more visibility and coordination could be viewed as
> rather "command and control", and therefore not in the spirit of open
> source.  I'd be interested to hear what others think about this.
>
> For the record, what I think we should introduce is:
>
> 1. A regular round-up email that gets sent to the list.  Someone would be
> responsible for collating brief emails from developers describing what
> they're planning to work on, and would condense this into something useful
> to the general Proton community.  I would be happy to perform this role.
> This round-up would necessarily be descriptive, not prescriptive.
>
> 2. We would commit to keeping the roadmap more up to date so that it
> becomes a useful resource for people wishing to work in a complementary way.
>
>
> I believe that most of the above points could apply to the Qpid project as
> a whole.  But, to avoid trying to boil the ocean, I thought it would be
> worth testing these ideas in the narrower Proton domain first.
>
> Phil
>
> [1] https://cwiki.apache.org/confluence/display/qpid/AMQP+1.0+Roadmap

Re: put vs. send

[Rajith Attapattu]

On Wed, Mar 6, 2013 at 11:37 AM, Rajith Attapattu  wrote:
> On Wed, Mar 6, 2013 at 10:09 AM, Rafael Schloming  wrote:
>> On Wed, Mar 6, 2013 at 6:52 AM, Ted Ross  wrote:
>>
>>>
>>> On 03/06/2013 08:30 AM, Rafael Schloming wrote:
>>>
>>>> On Wed, Mar 6, 2013 at 5:15 AM, Ted Ross  wrote:
>>>>
>>>>  This is exactly right.  The API behaves in a surprising way and causes
>>>>> reasonable programmers to write programs that don't work. For the sake of
>>>>> adoption, we should fix this, not merely document it.
>>>>>
>>>>
>>>> This seems like a bit of a leap to me. Have we actually seen anyone
>>>> misusing or abusing the API due to this? Mick didn't come across it till I
>>>> pointed it out and even then he had to construct an experiment where he's
>>>> basically observing the over-the-wire behaviour in order to detect it.
>>>>
>>>> --Rafael
>>>>
>>>>
>>> The following code doesn't work:
>>>
>>> while (True) {
>>>   wait_for_and_get_next_event(&**event);
>>>   pn_messenger_put(event);
>>> }
>>>
>>> If I add a send after every put, I'm going to limit my maximum message
>>> rate.  If I amortize my sends over every N puts, I may have
>>> arbitrarily/infinitely high latency on messages if the source of events
>>> goes quiet.

Having a background thread in the Messenger will only push this
problem from your application to the Messenger implementation.
Furthermore you will be at the mercy of the particulars of the client
library implementation as to how this background thread will take care
of the "outstanding work".
We could provide all kinds of knobs to tweak and tune this behaviour,
but I'd be far more comfortable if I as the application developer can
be in control of when the "flush" happens.

Either way you will have arbitrarily/infinitely high latency due to
complications at the TCP stack or the OS level. But you can at least
help your case a bit by having the application issue the flush than
letting the messenger doing it, bcos the application is in a better
position to determine what are the optimal conditions for doing so and
those conditions could be other than time, msg or byte count.

> You can employ a timer along with your event count (or based on a byte
> count) to get around that problem.
> The timer will ensure you flush events when there isn't enough activity.
> Isn't that acceptable ?
>
>>> I guess I'm questioning the mission of the Messenger API.  Which is the
>>> more important design goal:  general-purpose ease of use, or strict
>>> single-threaded asynchrony?
>>
>>
>> I wouldn't say it's a goal to avoid background threads, more of a really
>> nice thing to avoid if we can, and quite possibly a necessary mode of
>> operation in certain environments.
>>
>> I don't think your example code will work though even if there is a
>> background thread.
>
> This is a key point I missed when I thought about the problem along
> the same lines as Ted.
> Having a background thread cannot guarantee that your messages will be
> written on to the wire as that thread can be blocked due to TCP
> buffers being full or the thread being suppressed in favour of another
> more higher priority thread (for longer than you desire) thus
> increasing your latency beyond acceptable limits.
> You will invariably have outliers in your latency graph.
>
> On the other hand the library code will be much more simpler without
> the background thread.
>
>>  What do you want to happen when things start backing up?
>> Do you want messages to be dropped? Do you want put to start blocking? Do
>> you just want memory to grow indefinitely?
>>
>> --Rafael

Re: put vs. send

[Rajith Attapattu]

On Wed, Mar 6, 2013 at 10:09 AM, Rafael Schloming  wrote:
> On Wed, Mar 6, 2013 at 6:52 AM, Ted Ross  wrote:
>
>>
>> On 03/06/2013 08:30 AM, Rafael Schloming wrote:
>>
>>> On Wed, Mar 6, 2013 at 5:15 AM, Ted Ross  wrote:
>>>
>>>  This is exactly right.  The API behaves in a surprising way and causes
 reasonable programmers to write programs that don't work. For the sake of
 adoption, we should fix this, not merely document it.

>>>
>>> This seems like a bit of a leap to me. Have we actually seen anyone
>>> misusing or abusing the API due to this? Mick didn't come across it till I
>>> pointed it out and even then he had to construct an experiment where he's
>>> basically observing the over-the-wire behaviour in order to detect it.
>>>
>>> --Rafael
>>>
>>>
>> The following code doesn't work:
>>
>> while (True) {
>>   wait_for_and_get_next_event(&**event);
>>   pn_messenger_put(event);
>> }
>>
>> If I add a send after every put, I'm going to limit my maximum message
>> rate.  If I amortize my sends over every N puts, I may have
>> arbitrarily/infinitely high latency on messages if the source of events
>> goes quiet.

You can employ a timer along with your event count (or based on a byte
count) to get around that problem.
The timer will ensure you flush events when there isn't enough activity.
Isn't that acceptable ?

>> I guess I'm questioning the mission of the Messenger API.  Which is the
>> more important design goal:  general-purpose ease of use, or strict
>> single-threaded asynchrony?
>
>
> I wouldn't say it's a goal to avoid background threads, more of a really
> nice thing to avoid if we can, and quite possibly a necessary mode of
> operation in certain environments.
>
> I don't think your example code will work though even if there is a
> background thread.

This is a key point I missed when I thought about the problem along
the same lines as Ted.
Having a background thread cannot guarantee that your messages will be
written on to the wire as that thread can be blocked due to TCP
buffers being full or the thread being suppressed in favour of another
more higher priority thread (for longer than you desire) thus
increasing your latency beyond acceptable limits.
You will invariably have outliers in your latency graph.

On the other hand the library code will be much more simpler without
the background thread.

>  What do you want to happen when things start backing up?
> Do you want messages to be dropped? Do you want put to start blocking? Do
> you just want memory to grow indefinitely?
>
> --Rafael

Re: put vs. send

[Rajith Attapattu]

On Tue, Mar 5, 2013 at 3:20 PM, Rafael Schloming  wrote:
> On Tue, Mar 5, 2013 at 11:33 AM, Rajith Attapattu wrote:
>
>> On Tue, Mar 5, 2013 at 2:24 PM, Ted Ross  wrote:
>> >
>> > On 03/05/2013 02:14 PM, Rajith Attapattu wrote:
>> >>
>> >>
>> >> This is a good explanation that we need to put in the docs, as
>> >> Application developers certainly need to know how it behaves.
>> >> If one were to use the current C impl, it certainly gives the
>> >> impression that put() is meant to write messages into your internal
>> >> buffer and send() will actually write it to the wire.
>> >> Unfortunately some applications will depend on this behaviour, even
>> >> though it's not advisable
>> >>
>> >> If we are to change from say #2 to #1 or even #3 we need to release
>> >> note it prominently.
>> >>
>> >> I think the best solution is to make this behaviour configurable, and
>> >> advertise the default very prominently.
>> >> This way application developers will know exactly what they are
>> >> getting instead of us making changes underneath.
>> >>
>> >> Rajith
>> >>
>> >
>> > Making this configurable multiplies the size of the test matrix.  Can't
>> we
>> > make this simpler?
>>
>> I do understand your concern here, but the Java impl already does both
>> #1 and #2 and Rafi wants to do #3 in the future.
>> The old JMS client does something similar.
>>
>> I agree that if we just do option #2 (as you suggest below), then the
>> application can easily do #1 and #3 on top of that.
>> But I'm sure they will like if the library implements those strategies
>> for them and they have the ability to pick a strategy.
>>
>
> I don't  see why we'd make this configurable. All three options actually
> fit the same general semantics. Even if you're optimistically trying to
> transmit every single time put is called it's entirely possible for the
> socket to be blocked every single time you try. If this were to happen the
> implementation of #1 would appear to behave precisely the same as #2
> behaves.
>
> In other words if you're coding correctly against the API you can't assume
> that put will or won't have transmitted anything regardless of which
> strategy is used internally.
>

I agree with you. You make a very good point.
Perhaps we should explicitly make that clear in our docs to avoid
applications written against wrong assumptions.

Rajith

> --Rafael

Re: semantics vs. behavior

[Rajith Attapattu]

Mick, great question!
As I mentioned in the other thread we owe it to application developers
to describe the behaviour.
And if we change the behaviour btw releases we need to document it
prominently in the release notes as is often the case applications
will be written taking advantage of certain behavioural patterns.

A somewhat related example is, DUPS_OK and AUTO_ACK had the same
behaviour (all though wrong) and most applications that used AUTO_ACK
saw a performance drop when we eventually fixed it. Some applications
used the wrong ack mode, just bcos it behaved in a certain way.
Chances are that applications will be written to rely on the current
behaviour. So we need text to explain that all though that this is the
current behaviour things may change in the future. If it does change
then we need to tell them how to fall back to the previous behaviour.
This is my rationale behind making the behaviour configurable.

Rajith

On Tue, Mar 5, 2013 at 2:27 PM, Michael Goulish  wrote:
>
> Wow, I never thought I would have a philosophy question about documentation.
>
> Should I be documenting semantics that will be stable across all[1] releases?
> Or observable behavior in the release you, dear developer, have in your hands 
> right now?
> Or, perhaps, both ?
>
> I'll tell you, as a developer, I would certainly want to know exactly what's
> going to happen when I call put() and send() with this release.
>
> --
>
> [1] for sufficiently small values of "all".
>
>
>
>
>
>> What I said was put is *allowed* to send optimistically, not that it is
>> required to. It actually did send optimistically in a previous version of
>> the code, however I commented that line out.
>>
>> I would say the documented semantics of put and send should allow the
>> implementation the flexibility to do any of the following:
>>
>>   1) optimistically transmit whatever it can everytime so long as it
>> doesn't block
>>   2) never bother transmitting anything until you force it to by calling
>> send
>>   3) anything in between the first two, e.g. magically transmit once you've
>> put enough messages to reach the optimal batch size
>>
>> The reason for the behaviour you are observing is that we currently do
>> option 2 in the C impl, however we've done option 1 in the past (and I
>> think we do option 1 still in the Java impl), and we will probably do
>> option 3 in the future.

Re: put vs. send

[Rajith Attapattu]

On Tue, Mar 5, 2013 at 2:24 PM, Ted Ross  wrote:
>
> On 03/05/2013 02:14 PM, Rajith Attapattu wrote:
>>
>>
>> This is a good explanation that we need to put in the docs, as
>> Application developers certainly need to know how it behaves.
>> If one were to use the current C impl, it certainly gives the
>> impression that put() is meant to write messages into your internal
>> buffer and send() will actually write it to the wire.
>> Unfortunately some applications will depend on this behaviour, even
>> though it's not advisable
>>
>> If we are to change from say #2 to #1 or even #3 we need to release
>> note it prominently.
>>
>> I think the best solution is to make this behaviour configurable, and
>> advertise the default very prominently.
>> This way application developers will know exactly what they are
>> getting instead of us making changes underneath.
>>
>> Rajith
>>
>
> Making this configurable multiplies the size of the test matrix.  Can't we
> make this simpler?

I do understand your concern here, but the Java impl already does both
#1 and #2 and Rafi wants to do #3 in the future.
The old JMS client does something similar.

I agree that if we just do option #2 (as you suggest below), then the
application can easily do #1 and #3 on top of that.
But I'm sure they will like if the library implements those strategies
for them and they have the ability to pick a strategy.

> To me, this sounds like an I/O facility in which your output lines may never
> get sent if you don't call fflush().  This will be a surprise to most
> programmers, who rarely use fflush().  I think most programmers would be
> happier if "put" caused the messages to be eventually sent and "send" was
> used only for blocking until messages were flushed out.
>
> -Ted
>

Re: put vs. send

[Rajith Attapattu]

On Tue, Mar 5, 2013 at 2:01 PM, Rafael Schloming  wrote:
> On Tue, Mar 5, 2013 at 10:42 AM, Michael Goulish wrote:
>
>>
>> quoth Rafi:
>>
>> > The semantics of pn_messenger_put allow it to send if it can do so
>> without
>> > blocking.
>>
>>
>> So, am I understanding correctly? -- I should be able to get messages
>> from my sender to my receiver just by calling put() -- if the receiver
>> is ready to receive?
>>
>
> Not necessarily, the receiver being ready just means you are unblocked on
> AMQP level flow control. You could also potentially block on the socket
> write (i.e. TCP level flow control). You need to be unblocked on both for
> put to succeed.
>
>
>>
>> The only transmission difference between put() and send() is that send()
>> will actually block until they're all sent (or timeout hits).  put() should
>> get rid of all the messages that aren't blocked, and leave all that are.
>>
>> . . .
>>
>> Because what I'm seeing is -- with my receiver hanging in recv(),
>> I put 5 messages.  Sender sits there for a while.  No messages arrive at
>> receiver.  Then sender calls send() -- and all 5 messages arrive at
>> the receiver.
>>
>> This is true whether on the receiver side, I use
>>
>>pn_messenger_recv ( messenger, 100 );
>>pn_messenger_recv ( messenger, 5 );
>>pn_messenger_recv ( messenger, 1 );
>>or
>>pn_messenger_recv ( messenger, -1 );
>>
>>
>> That's why it seemed two-stage to me.  put() seems to gets them staged,
>> send() seems to shove them out the door.
>>
>> No?
>> Or is this a bug?
>>
>
> What I said was put is *allowed* to send optimistically, not that it is
> required to. It actually did send optimistically in a previous version of
> the code, however I commented that line out.
>
> I would say the documented semantics of put and send should allow the
> implementation the flexibility to do any of the following:
>
>   1) optimistically transmit whatever it can everytime so long as it
> doesn't block
>   2) never bother transmitting anything until you force it to by calling
> send
>   3) anything in between the first two, e.g. magically transmit once you've
> put enough messages to reach the optimal batch size
>
> The reason for the behaviour you are observing is that we currently do
> option 2 in the C impl, however we've done option 1 in the past (and I
> think we do option 1 still in the Java impl), and we will probably do
> option 3 in the future.

This is a good explanation that we need to put in the docs, as
Application developers certainly need to know how it behaves.
If one were to use the current C impl, it certainly gives the
impression that put() is meant to write messages into your internal
buffer and send() will actually write it to the wire.
Unfortunately some applications will depend on this behaviour, even
though it's not advisable

If we are to change from say #2 to #1 or even #3 we need to release
note it prominently.

I think the best solution is to make this behaviour configurable, and
advertise the default very prominently.
This way application developers will know exactly what they are
getting instead of us making changes underneath.

Rajith

> --Rafael

Re: How about docs at top level?

[Rajith Attapattu]

+1

Rajith

On Tue, Mar 5, 2013 at 1:48 PM, Rafael Schloming  wrote:
> I'm +1 on docs. It would be consistent with examples, tests, and tools.
>
> --Rafael
>
> On Tue, Mar 5, 2013 at 1:01 AM, Phil Harvey wrote:
>
>> I'm happy with the location although to increase consistency with other
>> projects I have a mild preference for either "docs" or "doc".  The former
>> seems to be the most common in other open source projects, and the latter
>> is the name used by Qpid.
>>
>> Phil
>>
>>
>> On 4 March 2013 19:53, Michael Goulish  wrote:
>>
>> > I'm planning to start checking my docs into the proton tree soon.
>> > I was assuming I would just put them at top level, i.e.
>> >
>> > qpid-proton/documentation
>> >
>> > Anybody care to agree, object, counter-offer, praise, complain, argue,
>> > question, or muse ?
>> >
>> >
>>

Re: [RESULT] [VOTE] 0.4 RC3

[Rajith Attapattu]

Rafi,

I don't want to sound pedantic, but we should tag our releases as per
the guidelines provided by Apache.
The previous releases don't have tags either (at least they do have a
branch, but the current release doesn't have a branch either).

Regards,

Rajith

On Mon, Feb 25, 2013 at 4:05 PM, Rafael Schloming  wrote:
> I've uploaded the artifacts and updated the download page[1]. It will be
> about 24 hours or so until the mirrors are fully synced.
>
> [1] http://qpid.apache.org/proton/download.html
>
> --Rafael
>
> On Mon, Feb 25, 2013 at 12:44 PM, Rafael Schloming  wrote:
>
>> The vote carries with 5 +1's and 0 -1's. I'll upload the artifacts shortly.
>>
>> --Rafael
>>
>> On Thu, Feb 21, 2013 at 1:54 PM, Rafael Schloming wrote:
>>
>>> Hi everyone, I've spun an RC3 with fixes for the two immediately soluble
>>> issues that turned up with RC2. Please check it out and cast your vote.
>>>
>>> Source is here:
>>>   - http://people.apache.org/~rhs/qpid-proton-0.4rc3/
>>>
>>> Java binaries are here:
>>>  - https://repository.apache.org/content/repositories/orgapacheqpid-292/
>>>
>>> Changes since RC2:
>>>
>>> PROTON-230, PROTON-246: Copying nested data fails in some cases
>>> PROTON-245: make swig skip pn_dtag for compatibility with older versions
>>> of swig
>>>
>>> [ ] Yes, I believe we should make 0.4 RC3 into 0.4 final
>>> [ ] No, because ...
>>>
>>
>>

Re: [documentation] -- Intro to Proton

[Rajith Attapattu]

Phil,

I think it's worth experimenting with an approach that has a low
barrier-to-entry.
Markdown seems very very appealing. Anytime you can get away with
plain text, it's great.

And as Alan mentioned, the source doc itself can be a finished product.
So my +1 for trying this.

Regards,

Rajith

On Mon, Feb 25, 2013 at 8:56 PM, Phil Harvey  wrote:
> I do agree with you that having documentation committed alongside code is
> the right approach.
>
> I propose that we write this documentation in Markdown syntax. That gives
> us (or our users) the option of easily generating HTML whilst keeping the
> barrier to entry low for authors.
>
> I recognise that Markdown lacks the semantic richness of Docbook (used for
> the Qpid Broker), but I believe that's ok in this case since our
> documentation should be quite short (or we're doing something wrong).
>
> Phil
> On Feb 25, 2013 7:07 PM, "Rajith Attapattu"  wrote:
>
>> I'm strong believer in maintaining our docs in the source tree, as it
>> makes it easy to release docs along side the code.
>> Also it helps keep the docs current.
>> The wiki based documentation in the past had many issues, the chief
>> complaint being stale most of the time.
>>
>> We could look at doing something similar to the qpid docs, or we could
>> also use this opportunity to experiment with a different approach/tool
>> set.
>>
>> Rajith
>>
>> On Mon, Feb 25, 2013 at 1:50 PM, Michael Goulish 
>> wrote:
>> >
>> > I think I will be landing it in the code tree first, and
>> > from there, I don't know.   Any suggestions?
>> >
>> > In the code -- I assume it should be at the top level?
>> > i.e. a sibling of the README file?   i.e.
>> >
>> >
>> >  qpid-proton-0.4/pulitzer_prize_winning_documentation
>> >
>> >
>> > or something along those lines?
>> >
>> > Agree?  Disagree?
>> >
>> >
>> >
>> >
>> >
>> >
>> > - Original Message -
>> > From: "Phil Harvey" 
>> > To: proton@qpid.apache.org
>> > Sent: Monday, February 25, 2013 12:14:00 PM
>> > Subject: Re: [documentation] -- Intro to Proton
>> >
>> > Hi Michael,
>> >
>> > Maybe you didn't see my previous question (or maybe I didn't see your
>> > answer).
>> >
>> > Where are you intending to store this documentation?  Similarly, where
>> are
>> > you intending to publish it, e.g. as HTML and/or PDF on our web site, as
>> a
>> > wiki page etc?
>> >
>> > Thanks
>> > Phil
>> >
>> >
>> > On 25 February 2013 16:15, Michael Goulish  wrote:
>> >
>> >>
>> >> Here's the introduction I'm planning on.
>> >>
>> >> If anyone has any opinions, I'd be happy to get them --
>> >> is there too much detail for a quick intro?  Too
>> >> little?  A crucial bit I left out?  Something I got wrong?
>> >>
>> >> ##
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Introduction to Proton
>> >> ===
>> >>
>> >>
>> >>
>> >> The Messenger interface is a simple, high-level API that lets
>> >> you create point-to-point messaging applications quickly and easily.
>> >>
>> >> The interface offers four main categories of functionality.
>> >>
>> >>
>> >>
>> >>
>> >> Messenger Operation
>> >> ---
>> >>
>> >>   There are only a few operations that are not directly concerning
>> >>   with message transmission.
>> >>
>> >>   A messenger can be created, named, and freed.  It can be started
>> >>   and stopped, and it can be checked for errors after any operation.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Sending and Receiving
>> >> ---
>> >>
>> >>   Both sending and receiving happen in two stages, the inner stage
>> >>   moving the message between your application and a queue, the
>> >>   outer stage transmitting messages between your queues and
>> >>   remote messaging nodes.
>> >>
>&g

Re: [documentation] -- Intro to Proton

[Rajith Attapattu]

I'm strong believer in maintaining our docs in the source tree, as it
makes it easy to release docs along side the code.
Also it helps keep the docs current.
The wiki based documentation in the past had many issues, the chief
complaint being stale most of the time.

We could look at doing something similar to the qpid docs, or we could
also use this opportunity to experiment with a different approach/tool
set.

Rajith

On Mon, Feb 25, 2013 at 1:50 PM, Michael Goulish  wrote:
>
> I think I will be landing it in the code tree first, and
> from there, I don't know.   Any suggestions?
>
> In the code -- I assume it should be at the top level?
> i.e. a sibling of the README file?   i.e.
>
>
>  qpid-proton-0.4/pulitzer_prize_winning_documentation
>
>
> or something along those lines?
>
> Agree?  Disagree?
>
>
>
>
>
>
> - Original Message -
> From: "Phil Harvey" 
> To: proton@qpid.apache.org
> Sent: Monday, February 25, 2013 12:14:00 PM
> Subject: Re: [documentation] -- Intro to Proton
>
> Hi Michael,
>
> Maybe you didn't see my previous question (or maybe I didn't see your
> answer).
>
> Where are you intending to store this documentation?  Similarly, where are
> you intending to publish it, e.g. as HTML and/or PDF on our web site, as a
> wiki page etc?
>
> Thanks
> Phil
>
>
> On 25 February 2013 16:15, Michael Goulish  wrote:
>
>>
>> Here's the introduction I'm planning on.
>>
>> If anyone has any opinions, I'd be happy to get them --
>> is there too much detail for a quick intro?  Too
>> little?  A crucial bit I left out?  Something I got wrong?
>>
>> ##
>>
>>
>>
>>
>>
>> Introduction to Proton
>> ===
>>
>>
>>
>> The Messenger interface is a simple, high-level API that lets
>> you create point-to-point messaging applications quickly and easily.
>>
>> The interface offers four main categories of functionality.
>>
>>
>>
>>
>> Messenger Operation
>> ---
>>
>>   There are only a few operations that are not directly concerning
>>   with message transmission.
>>
>>   A messenger can be created, named, and freed.  It can be started
>>   and stopped, and it can be checked for errors after any operation.
>>
>>
>>
>>
>>
>> Sending and Receiving
>> ---
>>
>>   Both sending and receiving happen in two stages, the inner stage
>>   moving the message between your application and a queue, the
>>   outer stage transmitting messages between your queues and
>>   remote messaging nodes.
>>
>>   By changing the ratio of transmissions to queue transfers, you
>>   can optimize your messaging application for message latency or
>>   for overall throughput.
>>
>>   Subscriptions control what sources your messenger can receive
>>   from, and what sources it can send to.  Your messenger
>>   subscribes to the sources you want to receive from, while your
>>   outgoing messages will be received by messengers that have
>>   subscribed to your outgoing address.
>>
>>
>>
>>
>>
>> Message Disposition
>> ---
>>
>>   When you receive messages, you must either accept or reject them.
>>
>>   You can either configure your messenger to automatically accept
>>   all messages that you get, or you can exercise finer control over
>>   message acceptance and rejection, individually or in groups.
>>
>>   Trackers and Windows let you set or check the disposition of
>>   messages in groups.  Applying the disposition operations to
>>   groups of messages can improve your system's throughput.
>>
>>   When receiving messages, you can create a tracker
>>   for the most recently received message, and later use that tracker
>>   to accept or reject all messages up to (and including) that one.
>>
>>   When sending messages, you can create a tracker for your most
>>   recently sent message, and later use it to inquire about the
>>   remote disposition of all sent messages up to that point.
>>   If you don't want to let a receiver make you wait forever
>>   to see what he's going to do, you can set a timeout that will
>>   control how long he can take making up his mind.
>>
>>   By using incoming and outgoing Windows, you can limit the
>>   number of messages that these operations affect.
>>
>>
>>
>>
>>
>>
>>
>> Security
>> ---
>>
>>   The messenger interface allows you to use the Secure Sockets Layer
>>   by exposing an interface to the OpenSSL library.
>>
>>
>>
>>

[proton-j] MessengerFactory

[Rajith Attapattu]

I was wondering what is the mechanism recommended for obtaining a
MessengerFactory instance (other than directly instantiating it).

IIRC people are planning to use the pure java and swig based impl side
by side especially for testing.
So this rules out the way we used for the old jms client (Where the
class name was specified as jvm arg and used reflecting to load it).

Rajith

Re: Additional components for language bindings?

[Rajith Attapattu]

Just to clarify further. We could have a component called proton-bindings.
And we use a label to denote which binding it is.
If the bug turns out to be in the engine, we change the component but
retain the labels.
If other bindings show the same bug we can add labels.

This way we know which bindings were associated with the bug.
Having a component per binding would not allow the above flexibility.

Rajith

On Mon, Feb 4, 2013 at 2:56 PM, Rajith Attapattu  wrote:
> We could use lables to denote which binding(s).
> The advantage here is that if multiple bindings expose the same bug,
> all we need to do is to add an additional label to the same JIRA.
> We currently use labels in the JMS client to denote sub categories (Ex
> addressing, exception-handling).
> The same strategy could be used here.
>
> Rajith
>
> On Mon, Feb 4, 2013 at 2:15 PM, Rafael Schloming  wrote:
>> This is in interesting question. The implementations do share a lot, so
>> it's actually quite likely that bugs will occur in the engine rather than
>> in the binding itself. My initial impression is that it will be difficult
>> to categorize correctly against the proper software component on first
>> blush, e.g. say Messenger.get() is not returning a message or blowing up
>> under some circumstance. This could be due to an interop issue with another
>> implementation, a bug in the C engine, or a bug in the binding itself, and
>> it might not be obvious which until we do some investigation/debugging.
>>
>> It strikes me that we really have two independent pieces of information to
>> capture here, (1) the bindings in which the bug manifests, and (2) the
>> software component that is ultimately determined to be at fault. The
>> potential drawback of using components to represent both of these is that
>> for every engine bug we're likely to get a duplicate report from every
>> binding.
>>
>> I'm just thinking out loud here, and I'm not a JIRA expert, so I don't know
>> exactly what's possible, but it might be worth considering adding a custom
>> field for the binding(s) under which the issue occurs and reserving
>> component for whatever is actually at fault. This way it's possible we
>> might get people to augment the first occurence of the bug with additional
>> bindings and actually provide a valuable clue that it's a problem with the
>> engine because it occurs in multiple bindings.
>>
>> --Rafael
>>
>> On Thu, Jan 31, 2013 at 9:16 AM, Darryl L. Pierce wrote:
>>
>>> Should we have separate components in Jira to represent each of the
>>> language bindings? If someone were to report a bug against any of them,
>>> it seems a little incorrect to report it against proton-c unless it were
>>> specifically something under the language covers.
>>>
>>> I'd like to have the following components if everybody agrees:
>>>  * proton-ruby
>>>  * proton-perl
>>>  * proton-php
>>>  * proton-c++
>>>  * proton-python
>>>
>>> --
>>> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
>>> Delivering value year after year.
>>> Red Hat ranks #1 in value among software vendors.
>>> http://www.redhat.com/promo/vendor/
>>>
>>>

Re: Thoughts on proton performance and interoperability testing

[Rajith Attapattu]

Ken,

I want to draw your attention to a type of performance testing that we
haven't paid much attention in the past.
(The goals for #1 may include this, but just want to clarify if it's the case).

In the past we have focused a lot on performance tests that are
somewhat artificial and meaningless in real world use cases. These
type of testing would give us a false sense of security, until things
blow up in production.
Ex creating a message and then sending/receiving it repeatedly. These
gloss over message creation overheads, memory issues ..etc .Also
caching and other optimizations etc may makes the numbers look more
impressive than they really are.
Same goes for stability testing.

We should look at some select *real world use cases* and see how
proton could handle them. We could try to setup identical deployments
(or at least something that closely resembles) and see if we could
meet the requirements associated with those use cases.

This type of tests could give us good feedback about whether the
product is "production ready".
These tests are not easy to automate and I don't' think they need to
be. They are the kind of tests that would need to be run before every
release.

--Rajith

P.S Artificial tests also have a place. They are good at identifying
regressions and could be easily automated.
   Both types of tests are important.

On Mon, Feb 4, 2013 at 11:36 AM, Ken Giusti  wrote:
> I like what I'm hearing - here are some objectives based on what has been 
> proposed so far:
>
>
> 1) Messenger-based scale and soak tests.
>
>These would be 'black-box' type tests that would mimic simple deployment 
> scenarios, using clients based on messenger.  The goal would be to stress 
> messenger's features under scale over time, e.g: # of connections, links, 
> window sizes, header count, etc.
>
> I think we should leverage Hiram's amqp benchmarking project here - it 
> appears to offer all the high-level test setup and control that we'd need for 
> these types of tests.  I'm assuming we'd need to develop the messenger-based 
> clients for sending/receiving traffic.  In the near term we could run the 
> benchmarks against the existing QPID test clients and QPID broker, leveraging 
> Gordon's 1.0 integration work.  But that would forgo messenger test coverage.
>
> 2) Static component performance tests.
>
> These would be the self-contained performance tests as described previously 
> by Rafi.  Each test would exercise one specific aspect of 
> proton/messenger/driver, limiting the impact of any non-relevant factors.  
> Each test would provide a "operations-per-time-unit" metric that we could 
> track.
>
> 3) Client language inter-opt tests.
>
> These tests would guarantee that the type encodings work across 
> implementation languages.  We'd need to develop a simple message creator and 
> consumer in each supported language binding.  The test would run all 
> combinations of creator vs consumer and verify that types encoded in one 
> language can be decoded in another (as best as can be done given a target 
> language's idiosyncrasies).
>
> Opinions?
>
>
> I think #3 is probably the 'low-hanging' fruit of the three - at least it's 
> bounded by the number of available bindings and supported types.
>
> #2 is a bit more open-ended, and would require some duplication of effort 
> assuming separate tests for each implementation language.
>
> I'd need to spend some time getting familiar with the benchmarking project, 
> but it seems like it would make writing tests for #1 a lot easier.
>
> -K
>
> - Original Message -
>> On Thu, Jan 31, 2013 at 9:41 AM, Ken Giusti 
>> wrote:
>>
>> > Hi Folks,
>> >
>> > I'd like to solicit some ideas regarding $SUBJECT.
>> >
>> > I'm thinking we could take an approach similar to what is done on
>> > the C++
>> > broker tests now.  That is we should develop a set of "native" send
>> > and
>> > receive programs that can be used to profile various performance
>> > characteristics (msgs/sec with varying size, header content
>> > encode/decode
>> > etc).  By "native" I mean implementations in Java and C.
>> >
>> > I've hacked our C "send" and "recv" examples to provide a rough
>> > swag a
>> > measuring msgs/sec performance.  I use these to double check that
>> > any
>> > changes I make to the proton C codebase do not have an unexpected
>> > impact on
>> > performance.  This really belongs somewhere in our source tree, but
>> > for now
>> > you can grab the source here:
>> >  https://github.com/kgiusti/proton-tools.git
>> >
>> > We do something similar for the QPID broker - simple native clients
>> > (qpid-send, qpid-receive) that do the performance sensitive message
>> > generation/consumption.  We've written python scripts that drive
>> > these
>> > clients for various test cases.
>> >
>> > If we follow that approach, not only could we create a canned set
>> > of basic
>> > benchmarks that we could distribute, but we could also build
>> > inter-opt
>> > tests by running one

Re: Additional components for language bindings?

[Rajith Attapattu]

We could use lables to denote which binding(s).
The advantage here is that if multiple bindings expose the same bug,
all we need to do is to add an additional label to the same JIRA.
We currently use labels in the JMS client to denote sub categories (Ex
addressing, exception-handling).
The same strategy could be used here.

Rajith

On Mon, Feb 4, 2013 at 2:15 PM, Rafael Schloming  wrote:
> This is in interesting question. The implementations do share a lot, so
> it's actually quite likely that bugs will occur in the engine rather than
> in the binding itself. My initial impression is that it will be difficult
> to categorize correctly against the proper software component on first
> blush, e.g. say Messenger.get() is not returning a message or blowing up
> under some circumstance. This could be due to an interop issue with another
> implementation, a bug in the C engine, or a bug in the binding itself, and
> it might not be obvious which until we do some investigation/debugging.
>
> It strikes me that we really have two independent pieces of information to
> capture here, (1) the bindings in which the bug manifests, and (2) the
> software component that is ultimately determined to be at fault. The
> potential drawback of using components to represent both of these is that
> for every engine bug we're likely to get a duplicate report from every
> binding.
>
> I'm just thinking out loud here, and I'm not a JIRA expert, so I don't know
> exactly what's possible, but it might be worth considering adding a custom
> field for the binding(s) under which the issue occurs and reserving
> component for whatever is actually at fault. This way it's possible we
> might get people to augment the first occurence of the bug with additional
> bindings and actually provide a valuable clue that it's a problem with the
> engine because it occurs in multiple bindings.
>
> --Rafael
>
> On Thu, Jan 31, 2013 at 9:16 AM, Darryl L. Pierce wrote:
>
>> Should we have separate components in Jira to represent each of the
>> language bindings? If someone were to report a bug against any of them,
>> it seems a little incorrect to report it against proton-c unless it were
>> specifically something under the language covers.
>>
>> I'd like to have the following components if everybody agrees:
>>  * proton-ruby
>>  * proton-perl
>>  * proton-php
>>  * proton-c++
>>  * proton-python
>>
>> --
>> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
>> Delivering value year after year.
>> Red Hat ranks #1 in value among software vendors.
>> http://www.redhat.com/promo/vendor/
>>
>>

Re: Reducing the visibility of proton-j constructors

[Rajith Attapattu]

This is also my impression about Hirams work.
So I'm not sure why there is resistance to the changes being proposed
as it's only going to benefit in the long run should the impl changes.
(I do understand that there will be some initial work in changing the
code to use the factories and interfaces).

I also thought part of Phil's (and Rob's) work was to come up with a
set of interfaces for the API.
If I'm not mistaken this work is already sitting in a branch ? (I
stand to be corrected though).

Rajith

On Fri, Jan 25, 2013 at 3:44 PM, Rafael Schloming  wrote:
> FWIW, I don't think Hiram's usage fundamentally needs to depend on the pure
> Java impl. I may be out of date on this one, but I believe his access of
> the implementation was done for expediency. I know at least in a number of
> cases we discussed how the C implementation could accommodate his
> requirements, and we plan do to so at some point. Perhaps our efforts here
> would be better spent in building out the interface so that he doesn't need
> direct access to the implementation.
>
> --Rafael
>
> On Fri, Jan 25, 2013 at 10:31 AM, Phil Harvey 
> wrote:
>
>> This has implications for API versioning,
>>
>> I think developers of third party code can assume that the API of public
>> methods, including constructors, will remain stable across minor revisions
>> of a library.  If a constructor is public then we, the Proton developers,
>> have an obligation to respect this assumption (or at least to add "for
>> internal use only" to its Javadoc).
>>
>> On reflection, I'm coming round to the idea that keeping these constructors
>> public will place unreasonable restrictions on our ability to evolve the
>> implementation whilst keeping the API stable.
>>
>> Phil
>>
>>
>> On 25 January 2013 14:51, Rob Godfrey  wrote:
>>
>> > What's your need for the direct constructor?
>> >
>> >
>> > -- Rob
>> >
>> > On 25 January 2013 15:49, Hiram Chirino  wrote:
>> > > Let me clarify, I have no problem with adding Factories and using them
>> > > everywhere possible.  Just don't take my access away from direct
>> > > constructors.
>> > >
>> > >
>> > > On Fri, Jan 25, 2013 at 9:39 AM, Rob Godfrey > > >wrote:
>> > >
>> > >> In general dependency on implementation is bad, dependency on
>> interface
>> > is
>> > >> better.  One of the problems we've had with Qpid historically is it
>> > become
>> > >> hard to test as we have dependencies on implementation everywhere.
>> > >>
>> > >> From an end user perspective the difference is only replacing
>> > >>
>> > >> X x  = new X();
>> > >>
>> > >> with
>> > >>
>> > >> X x = XFactory.newInstance();
>> > >>
>> > >> But it makes it a lot easier to properly describe the functionality
>> > being
>> > >> returned. For implementation reasons, the concrete class may have
>> public
>> > >> methods that are not intended to be exposed to the application
>> > programmer
>> > >> but only to other cooperating classes.  To avoid confusion, and to
>> make
>> > it
>> > >> clear to us the delta between the "shared" API and the API extensions
>> > that
>> > >> we are supporting for the pure Java implementation the interfaces seem
>> > like
>> > >> the right way to go for me.
>> > >>
>> > >> The idea is that those who need the extension features will still
>> have a
>> > >> very clear way to get the implementation that provides these, without
>> > ever
>> > >> having to cast.
>> > >>
>> > >> -- Rob
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On 24 January 2013 20:03, Rafael Schloming  wrote:
>> > >>
>> > >> > On Thu, Jan 24, 2013 at 5:06 AM, Rob Godfrey <
>> rob.j.godf...@gmail.com
>> > >> > >wrote:
>> > >> >
>> > >> > > On 23 January 2013 17:36, Phil Harvey 
>> > >> wrote:
>> > >> > > >
>> > >> > > > As part of the Proton JNI work, I would like to remove all calls
>> > to
>> > >> > > > proton-j implementation constructors from "client code".  I
>> intend
>> > >> that
>> > >> > > > factories will be used instead [1], thereby abstracting away
>> > whether
>> > >> > the
>> > >> > > > implementation is pure Java or proton-c-via-JNI.
>> > >> > > >
>> > >> > > > I'd like to check that folks are happy with me making this
>> change,
>> > >> and
>> > >> > to
>> > >> > > > mention a couple of problems I've had.
>> > >> > > >
>> > >> > > > In this context, "client code" is anything outside the current
>> > >> > > > sub-component, where our sub-components are Engine, Codec,
>> Driver,
>> > >> > > Message
>> > >> > > > and Messenger, plus each of the contrib modules, and of course
>> > third
>> > >> > > party
>> > >> > > > code.
>> > >> > > >
>> > >> > > > To enforce this abstraction, I am planning to make the
>> > constructors
>> > >> of
>> > >> > > the
>> > >> > > > affected classes package-private where possible.  I believe
>> that,
>> > >> > > although
>> > >> > > > third party projects might already be calling these
>> constructors,
>> > it
>> > >> is
>> > >> > > > acceptable for us to change its public API in this manner while
>> > >> Proton
>> > >> > is

Re: Reducing the visibility of proton-j constructors

[Rajith Attapattu]

I agree with Rob on this point. Given the trouble we had with our old
client it will be fool hardy not to correct those mistake in the new
implementation.
We've also had instances where users (and sometimes our own test
cases) directly creating implementation classes using the public
constructors making it difficult to make any changes.

I also think Phil has a great point about API versioning.
Allowing access via Factories will afford us a lot of leeway going forward.

Rajith

On Fri, Jan 25, 2013 at 9:39 AM, Rob Godfrey  wrote:
> In general dependency on implementation is bad, dependency on interface is
> better.  One of the problems we've had with Qpid historically is it become
> hard to test as we have dependencies on implementation everywhere.
>
> From an end user perspective the difference is only replacing
>
> X x  = new X();
>
> with
>
> X x = XFactory.newInstance();
>
> But it makes it a lot easier to properly describe the functionality being
> returned. For implementation reasons, the concrete class may have public
> methods that are not intended to be exposed to the application programmer
> but only to other cooperating classes.  To avoid confusion, and to make it
> clear to us the delta between the "shared" API and the API extensions that
> we are supporting for the pure Java implementation the interfaces seem like
> the right way to go for me.
>
> The idea is that those who need the extension features will still have a
> very clear way to get the implementation that provides these, without ever
> having to cast.
>
> -- Rob
>
>
>
>
>
> On 24 January 2013 20:03, Rafael Schloming  wrote:
>
>> On Thu, Jan 24, 2013 at 5:06 AM, Rob Godfrey > >wrote:
>>
>> > On 23 January 2013 17:36, Phil Harvey  wrote:
>> > >
>> > > As part of the Proton JNI work, I would like to remove all calls to
>> > > proton-j implementation constructors from "client code".  I intend that
>> > > factories will be used instead [1], thereby abstracting away whether
>> the
>> > > implementation is pure Java or proton-c-via-JNI.
>> > >
>> > > I'd like to check that folks are happy with me making this change, and
>> to
>> > > mention a couple of problems I've had.
>> > >
>> > > In this context, "client code" is anything outside the current
>> > > sub-component, where our sub-components are Engine, Codec, Driver,
>> > Message
>> > > and Messenger, plus each of the contrib modules, and of course third
>> > party
>> > > code.
>> > >
>> > > To enforce this abstraction, I am planning to make the constructors of
>> > the
>> > > affected classes package-private where possible.  I believe that,
>> > although
>> > > third party projects might already be calling these constructors, it is
>> > > acceptable for us to change its public API in this manner while Proton
>> is
>> > > such a young project.
>> > >
>> >
>> > +1 to all of the above
>> >
>> > >
>> > > Please shout if you disagree with any of the above.
>> > >
>> > > Now, onto my problem.  I started off with the
>> > org.apache.qpid.proton.engine.
>> > > impl package, and found that  o.a.q.p.hawtdispatch.impl.AmqpTransport
>> > calls
>> > > various methods on ConnectionImpl and TransportImpl, so simply using a
>> > > Connection and Transport will not work.  I don't know what to do about
>> > > this, and would welcome people's opinions.
>> > >
>> >
>> > So, the simplest change would be to change the factories to use
>> > covariant return types
>> >
>> > e.g. EngingFactoryImpl becomes:
>> >
>> > @Override
>> > public ConnectionImpl createConnection()
>> > {
>> > return new ConnectionImpl();
>> > }
>> >
>> > @Override
>> > public TransportImpl createTransport()
>> > {
>> > return new TransportImpl();
>> > }
>> >
>> >
>> > ... etc
>> >
>> > Code that requires the extended functionality offered by the pure Java
>> > implementation can thus instantiate the desired Factory directly.
>> >
>>
>> What's the point of going through the factory in this scenario rather than
>> directly instantiating the classes as Hiram suggests? Is there some class
>> of thing the factory would/could do that the constructor can't/shouldn't?
>>
>> A second refinement might be to actually separate out the interface
>> > and implementation within the pure Java implementation so that we have
>> > a well defined "extended" Java API.  This interface could then be the
>> > return type of the factory.
>> >
>>
>> Maybe I'm misunderstanding, but what's the point of using an interface here
>> if you're still locked into the pure Java impl? Are you expecting to swap
>> out that impl under some circumstances?
>>
>> --Rafael
>>

Re: Changing the Proton build system to accommodate jni bindings

[Rajith Attapattu]

> On Fri, Jan 18, 2013 at 2:29 PM, Rafael Schloming  wrote:
>> The nub of the problem is the sharing of the Java Proton-API between
>> both proton-c and proton-j trees. Solutions based on svn-external and
>> a simple tree copy have been considered and discussed at length on
>> conference calls.  We have identified drawbacks in both solutions.

It would be great if things are discussed on the mailing lists as
opposed to conference calls.
However I applaud Keith for posting a detailed summary of the proposal
on the list.

Rajith

Re: Proton Messenger and the Request/Response pattern

[Rajith Attapattu]

I'd agree with Gordon.

1. We should keep the Message as a pure value object without any sort
of coupling to Messenger or other objects.

2. I'm in favor of layering features on top of a generic flexible core
component rather than putting them all in the same layer.
This allows us the freedom to add "extra features" without
sacrificing the "core features" that makes messenger attractive in the
first place.

So things like Request/Response, A Synchronous version of
Messenger, Message Listener/Callback features etc.. can be implemented
as separate layers on top of the Messenger instead of passing flags
for blocking etc.. The code is going to be more complicated and error
prone and less maintainable in the long run.

Rajith

On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim  wrote:
> On 01/02/2013 07:14 PM, Ted Ross wrote:
>>
>> I'd like to start a discussion on how, from an API perspective,
>> applications can use the request/response pattern.  If we get this
>> right, we will remove a significant barrier to adoption of AMQP.
>>
>> Middleware messaging systems typically do a poor job of supporting this
>> pattern.  The Qpid APIs are quite lacking in this regard (requester
>> creates and subscribes to a temporary queue with a _unique_ name and
>> places this name in the reply-to field).
>>
>> Proton Messenger supports request/reply (see
>> examples/messenger/$LANG/{client,server}) as follows:
>>
>> The requester (client) has to put _something_ into the request message's
>> reply_to field.  It also sets the correlation_id field if it needs to
>> dispatch multiple responses.  The responder (server) must copy the
>> request message's reply_to field to the response message's address field
>> and also copy the correlation_id.
>>
>> This API is good for the case where the client wants the response to go
>> to a third party.  In this case the reply_to is well understood to be
>> the address of the third party receiver.  However in the more common
>> case where the response is intended to come back to the client, it's not
>> clear at all what to put in the reply_to field.
>>
>> I propose that we allow the client to simply say
>>
>>  request_msg.reply_expected(cid)
>
>
> A message has no association with a messenger (until it is put onto the
> messengers outgoing queue at least). It is not clear how a method on message
> would therefore be in any better position to determine the reply-address to
> use.
>
> At present, if a message on the outgoing queue has no reply-to set, then the
> messenger will automatically set the reply-to to be an address corresponding
> to itself.
>
> You can also use the (in my view slightly clumsy) syntax ~/xyz, which will
> be 'munged' into a form that prepends the messengers name to form a full
> address. This form I presume would be used if you want to distinguish
> different categories of replies.
>
> I'm not exactly enamoured with this, but it does seem at least to address
> the key concern expressed here, namely what to put in the reply-to for the
> simple case. (The answer being, nothing!)
>
>
>> (I added the correlation_id argument because it's almost always going to
>> be needed).  Further, the server could use
>>
>>  reply_msg.in_reply_to(request_msg)
>>
>> which would take care of the addresses and the correlation_id.  It also
>> provides a place to report an error if a request cannot be replied to
>> (absent or invalid reply_to address) rather than force each server
>> implementer to code this check.
>
>
> This is really just a short hand for:
>
>   response.address = request.reply_to
>   response.correlation_id = request.correlation_id
>
> plus some code or exception indicating whether the address has been set. I
> have no objection to such a utility, though I'm not entirely convinced that
> a method on message is the best place. It also seems of fairly marginal
> benefit rather than removing a 'significant barrier to adoption'.
>
> Have you seen the JMS solution to simplifying the request-response pattern?
> There it is done by specifying a distinct class that uses a supplied session
> and destination and provides a request() method that takes a request message
> as input and blocks until it can return the response message. (See
> http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html).
> Details of creating temporary response queues etc are hidden behind this. I
> like that general approach as it layers the functionality cleanly on top of
> the more flexible base, and would also allow different configurations or
> conventions to be used.
>
> --Gordon.
>

Re: [VOTE] 0.3 RC3

[Rajith Attapattu]

On Mon, Jan 14, 2013 at 2:48 PM, Darryl L. Pierce  wrote:
> On Mon, Jan 14, 2013 at 02:35:08PM -0500, Rajith Attapattu wrote:
>> Rafi,
>>
>> We should create tags for the releases.
>> Unless I have missed (in which case I apologize), I don't see any for
>> 0.1 and 0.2 releases (I do see branches for them though).
>
> There are branches for the releases, if not tags.

I do see the branches, but no tags!
The Apache process requires us to tag each release to do repeatable builds.

(In the case of 0.1 and 0.2 the stuff on the branch may or may not
correspond to the exact released versions)

Rajith

>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
>

Re: [VOTE] 0.3 RC3

[Rajith Attapattu]

Rafi,

We should create tags for the releases.
Unless I have missed (in which case I apologize), I don't see any for
0.1 and 0.2 releases (I do see branches for them though).

Regards,

Rajith

On Mon, Jan 14, 2013 at 2:14 PM, Rafael Schloming  wrote:
> +1 from me as well. I think we have enough votes/time to call this one.
> I'll post the RC3 as 0.3 shortly.
>
> --Rafael
>
> On Fri, Jan 11, 2013 at 3:29 PM, Ted Ross  wrote:
>
>> +1
>>
>> Tested against a couple of applications I'm developing.  It looks good.
>>
>> -Ted
>>
>>
>> On 01/08/2013 09:53 PM, Rafael Schloming wrote:
>>
>>> Source is here: 
>>> http://people.apache.org/~rhs/**qpid-proton-0.3rc3/
>>> Java binaries are here:
>>> https://repository.apache.org/**content/repositories/**orgapacheqpid-118/
>>>
>>> Fixes since RC2 include:
>>>
>>>- messenger now reports aborted connections
>>>- tarball for ruby gem generation
>>>- ssl fix (PROTON-171)
>>>
>>> --Rafael
>>>
>>>
>>

Re: 0.3 RC1

[Rajith Attapattu]

Rafi,

If you are spinning another RC please include [1] ?
It's done to ensure we close tcp connections.
Please see [2] for details.

[1] http://svn.apache.org/viewvc?rev=1425124&view=rev
[2] https://reviews.apache.org/r/7934/diff/3/?file=236677#file236677line108


Rajith

On Fri, Dec 21, 2012 at 2:28 PM, Ken Giusti  wrote:
> I hit a python problem on Centos-5 - which uses python 2.4:
>
>
> [kgiusti@localhost tests]$ ./proton-test
> Traceback (most recent call last):
>   File "./proton-test", line 553, in ?
> m = __import__(name, None, None, ["dummy"])
>   File 
> "/home/kgiusti/work/proton/RC3/qpid-proton-c-0.3/tests/proton_tests/__init__.py",
>  line 20, in ?
> import proton_tests.codec
>   File 
> "/home/kgiusti/work/proton/RC3/qpid-proton-c-0.3/tests/proton_tests/codec.py",
>  line 21, in ?
> from proton import *
>   File 
> "/home/kgiusti/work/proton/RC3/install/usr/local/lib/python2.4/site-packages/proton.py",
>  line 857, in ?
> class Data:
>   File 
> "/home/kgiusti/work/proton/RC3/install/usr/local/lib/python2.4/site-packages/proton.py",
>  line 1615, in Data
> put_mappings = {
> NameError: name 'bytes' is not defined
>
> The bytes type was added in 2.6
>
> -K
>
> - Original Message -
>> On Fri, Dec 21, 2012 at 9:32 AM, Ken Giusti 
>> wrote:
>>
>> > Ok, thanks Darryl - I'll give it a try.
>> >
>> > In any case, the README file in the release candidate needs to be
>> > updated
>> > - I imagine most folks just starting out with proton want to be
>> > able to
>> > evaluate it without needing root privs.
>> >
>>
>> I'll update the README for RC2.
>>
>> FWIW, I think there are two pretty distinct scenarios for make
>> install as
>> non root. The one proton developers keep running into is doing a non
>> root
>> make install as a way to test what would have gotten installed had a
>> normal
>> make install been done. For this purpose 'make install
>> DESTDIR=test-root'
>> is a much more accurate and robust way to do this because it gives
>> you
>> sub-tree with *exactly* what would have been installed based on the
>> particular cmake configuration. Doing a cmake
>> -DCMAKE_INSTALL_PREFIX=$HOME/blah is actually testing how a non-root
>> configuration would get installed which isn't necessarily the same at
>> all.
>>
>> The second scenario is as Ken points out when a user that wants to
>> evaluate
>> proton without doing a full system install. Right now we don't really
>> cater
>> to this case as an install scenario per/se, however it is quite easy
>> to do
>> from an svn checkout via the config.sh that is in the proton root.
>> Going
>> forward (e.g. for 0.4) we could probably have make install set up
>> something
>> similar for non root installs if we want to cater to that scenario
>> more
>> directly.
>>
>> --Rafael
>>

Re: Contrib under proton-c?

[Rajith Attapattu]

I also think qpid is a better place to host this than proton for the
reasons Rafi as mentioned.

However Qpid is fast becoming a collection of tools that are poorly
organized both on a source code level as well as on the product side.
I don't want to side track this conversation, but wanted to remind
that at some point we need to make an effort to work things at Qpid
level as well.

Rajith

On Tue, Dec 18, 2012 at 2:12 PM, Ted Ross  wrote:
>
> On 12/18/2012 02:06 PM, Rafael Schloming wrote:
>>
>> The proton-j/contrib directory was created to hold glue/integration with
>> established third-party libraries/apis (specifically jms and
>> hawtdispatch).
>> The fact that hawtdispatch and jms don't want a proton dependency, and
>> proton doesn't want a jms or hawtdispatch dependency kind of forces the
>> glue/integration to sit somewhere separate from each, and since it is kind
>> of silly to have an entirely separate project just for that
>> glue/integration code, creating a proton-j/contrib made sense. I wouldn't
>> be opposed to a proton-c/contrib for similar purposes, however what you're
>> describing sounds different. I'm assuming these libraries have no issue
>> depending on proton, which makes this use of contrib a kind of incubation,
>> which begs the question, what is the intent for this stuff once it's
>> hatched? Is the idea to provide another layer to the proton library
>> itself,
>> or to provide something that is really an independent library with a
>> different focus?
>>
>> If the former is the case then I think we should probably be taking a
>> careful look at the proposed APIs and understanding in detail what they
>> are
>> and how they build on what is already there (i.e. not just shoehorn it
>> into
>> contrib), and in the latter case I would say we should keep incubation to
>> qpid proper.
>
>
> I expect that in both cases, the proposed APIs will be looked at in detail
> so that their place in the world can be well understood.
>
> Thanks for all the input.  I think I'll incubate them in qpid/extras.  If we
> decide they should live in Proton, we can always move them.  They will
> either move somewhere or die from neglect.
>
> -Ted
>

[jira] [Commented] (PROTON-179) proton-j - replace system.out and system.err calls with java.util.logging

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510623#comment-13510623
 ] 

Rajith Attapattu commented on PROTON-179:
-

I started out with a logging callback when I started driver code, but since 
then have used java.util.logging as the rest of the engine code was using that.

I think a good long term solution is to provide a callback so the host 
application can decide what framework to use.

> proton-j - replace system.out and system.err calls with java.util.logging
> -
>
> Key: PROTON-179
> URL: https://issues.apache.org/jira/browse/PROTON-179
> Project: Qpid Proton
>  Issue Type: Improvement
>  Components: proton-j
>Affects Versions: 0.2
>Reporter: Philip Harvey
>Assignee: Rob Godfrey
> Attachments: 
> 0001-PROTON-179-use-java.util.logging-instead-of-system.o.patch
>
>
> We do not yet know what the final solution is for how proton-j should perform 
> logging.  We think we might want to allow Proton's host application to 
> provide logging callbacks, allowing Proton's output to be combined with the 
> host application's output.
> Pending the final logging design, we should use java.util.logging in the mean 
> time.  This is at least preferable to the existing system.out.println calls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] 0.2 RC4

[Rajith Attapattu]

[x] Ship it. Reviewed the C code change with Rafi.

Rajith

On Mon, Nov 5, 2012 at 7:29 AM, Rob Godfrey  wrote:
> [X] Ship it! (Release RC4 as 0.2)
>
> Tested java and reviewed C change.
>
> -- Rob
>
> On 5 November 2012 13:19, Rafael Schloming  wrote:
>
>> Posted here: http://people.apache.org/~rhs/qpid-proton-0.2rc4/
>>
>> [ ] Ship it! (Release RC4 as 0.2)
>> [ ] No (We need another RC because...)
>>
>> The only difference between RC3 and RC4 for proton-c is a one line change
>> in engine.c that ensures that pn_transport_output will not fail to produce
>> output under particular conditions despite there being output to produce.
>> There was also a public accessor added in proton-j. Both of these are very
>> small and sufficiently isolated changes that they shouldn't impact any
>> testing that has been done on RC3. I've attached the total diff in case
>> anyone wants to check it out. I'm hoping we can still get enough votes to
>> do a release tonight so please have a look and give it your +1 if it checks
>> out.
>>
>> --Rafael
>>

Re: [VOTE] RC8

[Rajith Attapattu]

[x] Ship it!

Rajith

On Mon, Oct 29, 2012 at 9:32 PM, Rafael Schloming  wrote:
> I'm optimistically starting an official release vote for RC8 here. I'm
> hoping if there is enough support and no procedural objections we can fudge
> the 72 hour formal vote process down to 24 hours. I personally think this
> is reasonable because (a) it's a 0.1 release and (b) we've been at the RC
> thing for well over 72 hours now and the changes at this point are pretty
> small and isolated. The motivation for this fudging is to get something
> official posted for conference attendees to download. Please place your
> votes ASAP. Thanks!
>
> [ ] Ship it! (http://people.apache.org/~rhs/qpid-proton-0.1rc8/ as 0.1)
> [ ] Don't ship it!
>
> --Rafael

Re: acks for messenger

[Rajith Attapattu]

I like Option 2 with the addendum.

1. Keeping "Message" purely as a value object without any coupling to
any state is very desirable.

2. As Justin mentioned, option 2 allows a clear and more importantly a
common approach for handling reliability for both sender and receiver.

3. Messenger.ack() to ack all messages received upto that point is a
good compromise for people looking for the more familiar JMS like
approach.

Regards,

Rajith

On Sun, Oct 28, 2012 at 2:58 AM, David Ingham
 wrote:
> I like option 2 too for the concept separation reasons that have already been 
> stated.
>
> The next question is how would an app select which receive mode it's 
> interested in (auto ack vs. explicit ack). Would this be an option on 
> Messenger.subscribe()?
>
> Dave.
>
> -Original Message-
> From: William Henry [mailto:whe...@redhat.com]
> Sent: Friday, October 26, 2012 12:27 PM
> To: proton@qpid.apache.org
> Subject: Re: acks for messenger
>
> +1  I like the separation.  And the analogy to tracking is not only palatable 
> but also kinda proves out the idea.
>
> William
>
> - Original Message -
>> I like option 2) for two reasons.  One, it produces very
>> straightforward semantics for the various levels of delivery
>> guarantee.  Two, it's easy to ignore if you'd like: if you just want
>> fire-and-forget messaging, you don't have to engage delivery as an api
>> concept; if you later change your mind, it's not hard to level up.
>>
>> Justin
>>
>> On Fri, 26 Oct 2012, Rafael Schloming wrote:
>>
>> > I'm taking a look at expanding the messenger API to support
>> > reliability and so far there seem to be two directions to explore
>> > which I'll attempt to describe below:
>> >
>> > Option 1)
>> >
>> >  Messenger.ack(Message) or possibly Message.ack()
>> >
>> >  I'll describe this as the simple/expected/conservative option, and
>> > those really are its strong points. Some less desirable points are
>> > that it takes the Message concept in a bit of a different  direction
>> > from where it is now. Message is no longer simply a holder of
>> > content, but it is also now (at least internally) tracking some
>> > kind  of delivery state. This is undesirable from a design
>> > perspective  since you're really merging two separate concepts here,
>> > delivery  state and message content, e.g. you now end up holding
>> > onto the  message content to track the delivery state when arguablly
>> > the  common case is that an app will be done processing the content
>> > quickly but may care about the delivery state for longer. Another
>> > implication of this merging is that it makes messages harder to
>> > reuse, e.g. imagine if you want to receive a message, mutate it a
>> > bit, and then resend it or send a number of similar messages.
>> >
>> >  It's also potentially more work from an implementation perspective
>> > as the underlying model treats Message as pure content and has
>> > delivery factored out as a separate concept, so this would be the
>> > start of a bit of an impedence missmatch between layers. It's
>> > certainly doable, but might result in more overall code since we'd
>> > be expressing similar concepts in two different ways.
>> >
>> > Option 2)
>> >
>> >  Introduce/surface the notion of a delivery/tracking number through
>> > the Messenger API, e.g.:
>> >
>> >  Messenger.put(Message) -> Delivery/Tracking-Number/Whatever
>> >  Messenger.get(Message) -> Delivery/Tracking-Number/Whatever
>> >
>> >  (I'll abbreviate Delivery/Tracking-Number/Whatever as DTW for
>> >  now.)
>> >
>> >  There are a couple of choices with this option, DTW could be a
>> > value  object (i.e. similar to a handle), with actions on the
>> > messenger  itself, e.g. Messenger.ack(DTW), Messenger.status(DTW).
>> > This kind  of  takes the tracking number analogy and runs with it,
>> > you ask your  messenger about the status of a given delivery via its
>> > tracking  number. Alternatively, DTW itself could be a more action
>> > oriented  interface with its own methods for acking/status/etc. This
>> > isn't  necessarily an either/or as there are reasons the C API might
>> > want  to use a handle approach even if we wish to
>> > conceptualize/surface  DTW as more of an independent thing in the
>> > object oriented  interfaces.
>> >
>> >  On the negative side this is less traditional/expected relative to
>> > Option (1), and it does in total add more surface area to the API.
>> >  On the positive side however it does provide a lot more capability
>> > since the same concept extends quite easily to track the state of
>> > outgoing deliveries.
>> >
>> >  From a design perspective this is nicer for a couple of reasons,
>> > unlike Option (1) by keeping Message as a pure holder of content
>> > you  have more flexibility with how you use the API, e.g. you can
>> > discard/reuse the Message but still track the status of your
>> > deliveries. Also, it seems likely that once the Option 1) path
>> > includes the ability to track the statu

Re: RC7

[Rajith Attapattu]

Opps I meant f17 :)

Rajith

On Sat, Oct 27, 2012 at 9:56 AM, Rajith Attapattu  wrote:
> On friday I ran the java and C builds on f14 (64bit) and passed
> without any issues.
>
> Rajith
>
> On Fri, Oct 26, 2012 at 11:51 AM, Ken Giusti  wrote:
>> +1 RC7 proton-c {mainly Debian6 i686 testing}
>>
>> -K
>>
>> - Original Message -
>>> Lucky number 7 posted here:
>>>
>>>   http://people.apache.org/~rhs/qpid-proton-0.1rc7/
>>>
>>> Only changes from RC6 are added LICENSE headers and two minor fixes
>>> to the
>>> Java SASL implementation (PROTON-103).
>>>
>>> --Rafael
>>>

Re: RC7

[Rajith Attapattu]

On friday I ran the java and C builds on f14 (64bit) and passed
without any issues.

Rajith

On Fri, Oct 26, 2012 at 11:51 AM, Ken Giusti  wrote:
> +1 RC7 proton-c {mainly Debian6 i686 testing}
>
> -K
>
> - Original Message -
>> Lucky number 7 posted here:
>>
>>   http://people.apache.org/~rhs/qpid-proton-0.1rc7/
>>
>> Only changes from RC6 are added LICENSE headers and two minor fixes
>> to the
>> Java SASL implementation (PROTON-103).
>>
>> --Rafael
>>

Re: svn commit: r1402510 - in /qpid/proton/trunk: examples/broker/ examples/mailbox/ proton-c/ proton-c/bindings/php/ proton-c/bindings/php/examples/ proton-c/bindings/python/ proton-c/bindings/ruby/

[Rajith Attapattu]

I used RAT to automatically add the headers.
I will correct it on monday.

Rajith

On Fri, Oct 26, 2012 at 8:16 PM, Robbie Gemmell
 wrote:
> It doesnt really affect the release since the files are now licenced, but
> the diff below suggests the header wasn't added to the Java files where we
> normally put it (at the top of the file), it would be nice if they were
> consistent with the others.
>
> Robbie
>
> On 26 October 2012 14:42,  wrote:
>
>> Author: rajith
>> Date: Fri Oct 26 13:42:54 2012
>> New Revision: 1402510
>>
>> URL: http://svn.apache.org/viewvc?rev=1402510&view=rev
>> Log:
>> NO_JIRA Added missing license headers based on the RAT output
>>
>> Modified:
>> qpid/proton/trunk/examples/broker/example-nodes
>> qpid/proton/trunk/examples/mailbox/ssl-setup.sh
>> qpid/proton/trunk/proton-c/CMakeLists.txt
>> qpid/proton/trunk/proton-c/bindings/php/cproton.ini
>> qpid/proton/trunk/proton-c/bindings/php/examples/client.php
>> qpid/proton/trunk/proton-c/bindings/php/examples/server.php
>> qpid/proton/trunk/proton-c/bindings/php/get_include_dir.php
>> qpid/proton/trunk/proton-c/bindings/php/php.i
>> qpid/proton/trunk/proton-c/bindings/php/tests.php
>> qpid/proton/trunk/proton-c/bindings/python/CMakeLists.txt
>> qpid/proton/trunk/proton-c/bindings/python/python.i
>> qpid/proton/trunk/proton-c/bindings/ruby/ruby.i
>> qpid/proton/trunk/proton-c/docs/api/user.doxygen.in
>> qpid/proton/trunk/proton-c/include/proton/cproton.i
>> qpid/proton/trunk/proton-c/src/libqpid-proton.pc.in
>> qpid/proton/trunk/proton-c/src/sasl/sasl-internal.h
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/codec/CompositeWritableBuffer.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/EndpointError.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/EndpointState.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/Sasl.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/SaslClient.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/SaslServer.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/EndpointImplQuery.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/LinkNode.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/SaslClientImpl.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/SaslImpl.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/engine/impl/SaslServerImpl.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/message/Message.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/message/MessageFormat.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/type/messaging/Outcome.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/type/transport/FrameBody.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/type/transport/Source.java
>>
>> qpid/proton/trunk/proton-j/src/main/java/org/apache/qpid/proton/type/transport/Target.java
>>
>> Modified: qpid/proton/trunk/examples/broker/example-nodes
>> URL:
>> http://svn.apache.org/viewvc/qpid/proton/trunk/examples/broker/example-nodes?rev=1402510&r1=1402509&r2=1402510&view=diff
>>
>> ==
>> --- qpid/proton/trunk/examples/broker/example-nodes (original)
>> +++ qpid/proton/trunk/examples/broker/example-nodes Fri Oct 26 13:42:54
>> 2012
>> @@ -1,3 +1,22 @@
>> +#
>> +# Licensed to the Apache Software Foundation (ASF) under one
>> +# or more contributor license agreements.  See the NOTICE file
>> +# distributed with this work for additional information
>> +# regarding copyright ownership.  The ASF licenses this file
>> +# to you under the Apache License, Version 2.0 (the
>> +# "License"); you may not use this file except in compliance
>> +# with the License.  You may obtain a copy of the License at
>> +#
>> +#   http://www.apache.org/licenses/LICENSE-2.0
>> +#
>> +# Unless required by applicable law or agreed to in writing,
>> +# software distributed under the License is distributed on an
>> +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>> +# KIND, either express or implied.  See the License for the
>> +# specific language governing permissions and limitations
>> +# under the License.
>> +#
>> +
>>  from queue import Queue
>>  from hole import Hole
>>
>>
>> Modified: qpid/proton/trunk/examples/mailbox/ssl-setup.sh
>> URL:
>> http://svn.apache.org/viewvc/qpid/proton/trunk/examples/mailbox/ssl-setup.sh?rev=1402510&r1=1402509&r2=1402510&view=diff
>>
>> ==
>> --- qpid/proton/trunk/examples/mailbox/ssl-setup.sh (original)
>> +++ qpi

Re: RAT output for Proton

[Rajith Attapattu]

I have now added the missing license headers.
Updated RAT output here [1]

I used the following options in case someone wants to try it in the future.
-e *gitignore -e *xml -e TODO -e README* -e *html -e EXAMPLES  -d


Regards,

Rajith

[1] http://creadur.apache.org/rat/

On Thu, Oct 25, 2012 at 9:01 PM, Rajith Attapattu  wrote:
> The following RAT output shows that we are missing a few license
> headers, mostly java files.
>
> http://people.apache.org/~rajith/proton/proton-rat-output.txt
>
> We need to fix them before releasing.
> Unfortunately I need to step out now.
> If somebody hasn't gotten to it by tomorrow morning, I can take care of it.
>
> Regards,
>
> Rajith
>
> P.S There is an option in RAT "-a" that will automatically add the
> license header.
>   This could potentially mess up cmake files. So be careful :)
>
> [1] http://creadur.apache.org/rat/
> [2] http://people.apache.org/~rajith/proton/proton-rat-output.txt

Re: RC6

[Rajith Attapattu]

On Fri, Oct 26, 2012 at 5:30 AM, Robbie Gemmell
 wrote:
> I dont think KEYS should be proton specific, its just the Qpid committers
> and should be the same file as a result. Commons-* seem to use the same
> KEYS file for example.
>
> (Aside: we should probably be transitioning away from the KEYS file anyway
> and using the new system they put in place for distribution via
> https://people.apache.org/keys/).
>

Sorry I haven't looked at this stuff for a while now, so wasn't aware
about the new mechanism.
The existing KEYS file was inside the qpid tree, hence my suggestion.
Actually a better suggestion would have been to move that file to the top level.

But anyways, given that we have a new system in place it's irrelevant now.
We should make an attempt to use the new approach.

Rajith

> Robbie
>
> On 26 October 2012 02:05, Rajith Attapattu  wrote:
>
>> Sorry I should have done this before you posted RC6 :(
>> We have a few files (mostly java files) that are missing license headers.
>> Please look at the RAT output I posted.
>>
>> Once the final RC is voted, the release manager (in this case Rafi)
>> will have to sign the final artefacts.
>> You already have your key listed here [1]
>> We should move [1] under the proton main dir.
>>
>> Regards,
>>
>> Rajith
>>
>> [1] https://svn.apache.org/repos/asf/qpid/trunk/qpid/KEYS
>>
>> On Thu, Oct 25, 2012 at 8:38 PM, Rafael Schloming 
>> wrote:
>> > I put up an RC6 here[1] with the java shim fixed so the SSL tests skip
>> > properly. That's the only change from RC5.
>> >
>> > [1] http://people.apache.org/~rhs/qpid-proton-0.1rc6/
>> >
>> > --Rafael
>>

Re: RC6

[Rajith Attapattu]

Sorry I should have done this before you posted RC6 :(
We have a few files (mostly java files) that are missing license headers.
Please look at the RAT output I posted.

Once the final RC is voted, the release manager (in this case Rafi)
will have to sign the final artefacts.
You already have your key listed here [1]
We should move [1] under the proton main dir.

Regards,

Rajith

[1] https://svn.apache.org/repos/asf/qpid/trunk/qpid/KEYS

On Thu, Oct 25, 2012 at 8:38 PM, Rafael Schloming  wrote:
> I put up an RC6 here[1] with the java shim fixed so the SSL tests skip
> properly. That's the only change from RC5.
>
> [1] http://people.apache.org/~rhs/qpid-proton-0.1rc6/
>
> --Rafael

RAT output for Proton

[Rajith Attapattu]

The following RAT output shows that we are missing a few license
headers, mostly java files.

http://people.apache.org/~rajith/proton/proton-rat-output.txt

We need to fix them before releasing.
Unfortunately I need to step out now.
If somebody hasn't gotten to it by tomorrow morning, I can take care of it.

Regards,

Rajith

P.S There is an option in RAT "-a" that will automatically add the
license header.
  This could potentially mess up cmake files. So be careful :)

[1] http://creadur.apache.org/rat/
[2] http://people.apache.org/~rajith/proton/proton-rat-output.txt

Re: RC5

[Rajith Attapattu]

We have a build failure on the java side.
It appears the SSL tests added in Kens fix is failing.
proton_tests.ssl.SslTest.test_client_authentication . fail

We should exclude this test before we spin the final release.

Rajith

On Thu, Oct 25, 2012 at 5:25 PM, Ken Giusti  wrote:
> mea culpa - apologies.
>
> However, RC5 looks pretty good on these platforms:
>
> Fedora 17 (64bit)
> Debian 6 (32bit, vm)
> Ubuntu 12.04 (32bit, vm)
> Centos 6 (64bit, vm)
>
>
> I simply did a make & install, then ran the proton test suite.   A few 
> warnings during build, but all tests pass.
>
> The only problem platform was my Centos5 VM (32bit).  The "make all/make 
> install" failed due to the php.swg file not being present.  It appears that 
> the version of swig on Centos5 (1.3.29) does not include that php.swg file:
>
> [ 95%] Swig source
> /home/kgiusti/work/proton/RC1/qpid-proton-c-0.1/bindings/php/php.i:4: Error: 
> Unable to find 'php.swg'
> make[2]: *** [bindings/php/phpPHP_wrap.c] Error 1
> make[1]: *** [bindings/php/CMakeFiles/cproton.dir/all] Error 2
> make: *** [all] Error 2
>
>
> IMHO: +1 RC5, fix the Centos 5 issue in the next release.
>
> -K
>
> - Original Message -
>> Please have a look. This one includes ken's ssl fix in addition to
>> everything that was in RC4:
>>
>> http://people.apache.org/~rhs/qpid-proton-0.1rc5/
>>

[jira] [Commented] (PROTON-87) Support plugging in a protocol tracer into the transport

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-87?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479023#comment-13479023
 ] 

Rajith Attapattu commented on PROTON-87:


All though not a big deal, I also think it's best to move the above code block 
inside,

if( _protocolTracer!=null ) {

}

After all we only want to duplicate the buffer if we are running the trace.

> Support plugging in a protocol tracer into the transport
> 
>
> Key: PROTON-87
> URL: https://issues.apache.org/jira/browse/PROTON-87
> Project: Qpid Proton
>  Issue Type: Bug
>  Components: proton-j
>Reporter: Hiram Chirino
> Attachments: PROTON-87.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-87) Support plugging in a protocol tracer into the transport

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-87?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479021#comment-13479021
 ] 

Rajith Attapattu commented on PROTON-87:


Also if I'm not mistaken, the originalPayload is never set, its always null.

+ByteBuffer originalPayload = null;
+if( payload!=null ) {
+payload = payload.duplicate();
+}
+

So it should be

+ByteBuffer originalPayload = null;
+if( payload!=null ) {
+originalPayload = payload.duplicate();
+}
+

> Support plugging in a protocol tracer into the transport
> 
>
> Key: PROTON-87
> URL: https://issues.apache.org/jira/browse/PROTON-87
> Project: Qpid Proton
>  Issue Type: Bug
>  Components: proton-j
>Reporter: Hiram Chirino
> Attachments: PROTON-87.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-71) Update .gitignore to include IDE and build artifacts

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474354#comment-13474354
 ] 

Rajith Attapattu commented on PROTON-71:


Looks Rafi beat me to it :)

> Update .gitignore to include IDE and build artifacts
> 
>
> Key: PROTON-71
> URL: https://issues.apache.org/jira/browse/PROTON-71
> Project: Qpid Proton
>  Issue Type: Improvement
>Reporter: Hiram Chirino
>Priority: Minor
> Attachments: PROTON-71.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-71) Update .gitignore to include IDE and build artifacts

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474318#comment-13474318
 ] 

Rajith Attapattu commented on PROTON-71:


+1 
I was about to do this. Will apply the patch soon after lunch.

> Update .gitignore to include IDE and build artifacts
> 
>
> Key: PROTON-71
> URL: https://issues.apache.org/jira/browse/PROTON-71
> Project: Qpid Proton
>  Issue Type: Improvement
>Reporter: Hiram Chirino
>Priority: Minor
> Attachments: PROTON-71.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PROTON-66) Driver implementation for proton-j

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-66:
--

 Summary: Driver implementation for proton-j
 Key: PROTON-66
 URL: https://issues.apache.org/jira/browse/PROTON-66
 Project: Qpid Proton
  Issue Type: New Feature
Reporter: Rajith Attapattu


Provide an implementation for the Driver API.
This consists of 3 interfaces. Connector, Listener and Driver.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-65) Provide a AMQP 1.0 Message to JMS Message mapping logic/module

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472540#comment-13472540
 ] 

Rajith Attapattu commented on PROTON-65:


Hiram,

First of all, thx for confirming that it's indeed a separate jar. I figured it 
was the case, but wasn't too sure given that I haven't messed with mvn for long.

I certainly see the value of having a mapping btw the AMQP message <---> JMS 
Message and make it available for other folks to use it.
But I'm not sure if proton is the right place to have it.

Now to be fair to you, this is a discussion that goes beyond this particular 
issue.
We have to make a clear statement about what proton is and what proton is not.

Based on the discussions we had so far about the goals of proton, this doesn't 
seem to fit in well.
IMO this is something that is best housed at Qpid (or even ActiveMQ for the 
time being).

Again this is only my opinion. I would talk to other folks as well to see what 
they think about this.

Regards,

Rajith

> Provide a AMQP 1.0 Message to JMS Message mapping logic/module
> --
>
> Key: PROTON-65
> URL: https://issues.apache.org/jira/browse/PROTON-65
> Project: Qpid Proton
>  Issue Type: New Feature
>  Components: proton-j
>Reporter: Hiram Chirino
> Attachments: PROTON-65.patch
>
>
> Having a easy/consistent way to transform a JMS message to an AMQP message 
> and back would be super handy.  Having that logic live in proton would help 
> make that mapping more standardized as different client and sever 
> implementations can just re-use the logic.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-65) Provide a AMQP 1.0 Message to JMS Message mapping logic/module

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472449#comment-13472449
 ] 

Rajith Attapattu commented on PROTON-65:


Hiram,

One of the goals of proton is to be as as dependency free as possible, so it 
can be used in as many environments as possible.
>From that pov I don't think this is a good idea at all.

For example, having the jms library as a dependency will be a big issue in 
using proton-j in Android (or similar environments). 

On the other hand if we can make this as a separate optional jar (maybe thats 
what you are doing in this patch, and I haven't messed with maven for ages) 
then that maybe acceptable.
But still I'm not too comfortable with the idea. I'm not too sure if proton is 
the place to do so.

I would like to see comments from other interested parties about this.

Regards,

Rajith

> Provide a AMQP 1.0 Message to JMS Message mapping logic/module
> --
>
> Key: PROTON-65
> URL: https://issues.apache.org/jira/browse/PROTON-65
> Project: Qpid Proton
>  Issue Type: New Feature
>  Components: proton-j
>Reporter: Hiram Chirino
> Attachments: PROTON-65.patch
>
>
> Having a easy/consistent way to transform a JMS message to an AMQP message 
> and back would be super handy.  Having that logic live in proton would help 
> make that mapping more standardized as different client and sever 
> implementations can just re-use the logic.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: What is Messenger API

[Rajith Attapattu]

One goal behind this thread was to get ideas on how to answer these
questions coming from a new user.
So thanks a lot for your answers.
I've had the privilege of speaking to Rafi several times, so I had a
fair bit of knowledge on each of those questions.

These are the sort of questions a prospective user would have.
>From past experience pointing people at examples or a header file to
figure out the API hasn't been great.
I'm not insisting we should do things one way or the other, but wanted
to have a discussion to see what peoples thoughts are on the $subject.
My opinion is that some sort of a document that describes the API (and
perhaps and FAQ) could be quite useful.

Otherwise we will be getting a lot of queries from end users and
testing folks about what the correct behaviour is.

Rajith

On Thu, Sep 20, 2012 at 5:18 PM, Darryl L. Pierce  wrote:
> On Thu, Sep 20, 2012 at 03:37:10PM -0400, Rajith Attapattu wrote:
>> Given some of the recent discussions, it appears there isn't much
>> consensus as to what the "Messenger API" is.
>> For my own sanity, could someone with more knowledge on the $subject
>> please explain the following?
>>
>> 1. What is Messenger API ?
>>  i.e Do we have a doc or a wiki page that documents what the API
>> is and more importantly what the expected behaviour is.
>
> From my perspective, a Messenger is a high-level end-point for sending
> and receiving messages. It keeps you from having to worry about the
> underlying components of a session, a context, encoding and decoding
> messages. Instead, it gives you very simple APIs to start and stop
> communications, queue up and send messages and receive and then pull out
> individual messages for processing.
>
>> 2. How is it different from the "Messaging API aka Qpid API"
>> I'm looking for something more than the matrix given by Rafi, all
>> though it provides a good start to understanding it.
>
> Again, from my perspective, it reduces the complexity by, again, keeping
> you from having to worry about the details of connecting with a remote
> messaging endpoint.
>
> Something else is that it's designed from the ground up with efficiency.
> One specific one that comes to mind is the pn_messenger_get() API, which
> can reuse existing instances of pn_message_t to avoid spending time
> allocating memory.
>
>> 3. How are we planning to position these various API's in the future?
>> I agree that a lot of things are up in the air, but it's also good
>> to share some early ideas all though they might change in the future.
>
> My understanding is that we'll have a layer on top of Proton that will
> provide the existing Qpid APIs while, underneath, Proton is doing the
> heavy lifting.
>
> I may be wrong on some (or all) points, in which case someone please set
> me straight.
>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
>

Re: What is Messenger API

[Rajith Attapattu]

On Thu, Sep 20, 2012 at 4:37 PM, William Henry  wrote:
> - Original Message -
>> On Thu, Sep 20, 2012 at 4:02 PM, William Henry 
>> wrote:
>> > Best to look at proton's examples/messenger send.py and recv.py
>> >
>> > That's the only documentation besides messenger.h
>>
>> That's precisely my point.
>> We can't continue to point people at examples to figure out what the
>> API is.
>> Given that we are going to do a bunch of releases, we should spend
>> some time trying to document this API and it's intended behaviour.
>>
>> William, given that you've used it extensively, would you like to
>> take
>> a stab at it?
>
> I wouldn't say I've used it extensively.  I am getting to know it better 
> though.
>
> Can we get some suggestions on the format that we'd like to see this 
> documentation in?

William, sorry for the late reply.
You raised a very good question. We will eventually have the API in
several languages from procedural to OO to functional.

While the core of the API remains the same, the look and feel would be
different for each paradigm and/or language.
For example the API in "C" will look different from the API for Java
or Erlang. But they both should provide the same set of features and
behaviour.

So to answer your question, I was thinking about doc that describes
the features and behaviour of the API in plain english.

My concern is that without an authoritative source we will start
having subtle differences between the API's.
A case in point is the current set of Qpid API's. At one stage we
weren't sure which implementation had the correct behaviour.

In the case of the engines, we have only two major implementations
that are tested using the python tests that acts as a functional spec.
But for the client API's, this quickly becomes an issue. I don't think
we could always use a common python test suite.
In such a case a spec could guide the testing for that respective
language which does not have support for python integration.

Also for somebody who wants to write a new language client, such a doc
would be beneficial.

I can buy the argument that we can designate the "messenger.h" as that source.
But then we have to make sure that any changes to the header file is
reflected across all language clients and the documentation in all
those clients are also updated.

Regards,

Rajith

> William
>
>>
>> Rajith
>>

Re: What is Messenger API

[Rajith Attapattu]

On Thu, Sep 20, 2012 at 4:02 PM, William Henry  wrote:
> Best to look at proton's examples/messenger send.py and recv.py
>
> That's the only documentation besides messenger.h

That's precisely my point.
We can't continue to point people at examples to figure out what the API is.
Given that we are going to do a bunch of releases, we should spend
some time trying to document this API and it's intended behaviour.

William, given that you've used it extensively, would you like to take
a stab at it?

Rajith

Engine API

[Rajith Attapattu]

We should seriously consider documenting the Engine API and it's
expected behaviour.
This has several advantages.

1. Use that as a basis for a functional spec, which we can then write
tests to verify the implementations.

2. Helps developers to navigate and understand the code more easily.

3. Finally we need this documentation sooner than later to accompany
our releases to for the end users

4. Helps to keep both C and Java impls in sync. We should probably
modify the doc before we change the code to ensure the doc is kept
current.

Any takers ? (Looking at Rafi :D)

Regards,

Rajith

What is Messenger API

[Rajith Attapattu]

Given some of the recent discussions, it appears there isn't much
consensus as to what the "Messenger API" is.
For my own sanity, could someone with more knowledge on the $subject
please explain the following?

1. What is Messenger API ?
 i.e Do we have a doc or a wiki page that documents what the API
is and more importantly what the expected behaviour is.

2. How is it different from the "Messaging API aka Qpid API"
I'm looking for something more than the matrix given by Rafi, all
though it provides a good start to understanding it.

3. How are we planning to position these various API's in the future?
I agree that a lot of things are up in the air, but it's also good
to share some early ideas all though they might change in the future.

Regards,

Rajith

Re: AMQP 1.0 support for JMS client via Proton

[Rajith Attapattu]

On Thu, Sep 20, 2012 at 4:00 AM, Rob Godfrey  wrote:
> On 20 September 2012 00:32, Rajith Attapattu  wrote:
>
>> Hi All,
>>
>> There are a few folks who are keen to have AMQP 1.0 support for our JMS
>> client.
>> Given that the parent AMQP TC is starting a bindings and mappings TC
>> which will cover JMS, I thought it would be a good idea to get a
>> discussion going on here as well.
>> We could aim to build the client as we progress through the TC and
>> provide feedback if necessary.
>>
>> On the hand, we've had extensive discussions on building a new JMS
>> client from scratch when adding 1.0 support.
>> The primary motivation was to address some of the nagging issues
>> around the old client.
>>
>> So far there have been two schools of thought on how to get AMQP 1.0
>> support.
>>
>> 1. JMS Client --> Qpid API --> Proton
>>
>> 2. JMS Client --> Proton
>>
>> While option #1 seems like killing two birds with one stone, I think
>> we should seriously consider option #2 as well.
>>
>>
> So, regardless of which path we take I think we need to carefully define
> the functional requirements for our JMS client. Looking at the JIRAs, and
> based on the experience of supporting the current client, the two areas
> where it causes most problems are 1) failover and 2) addressing.
>
> For  both (but especially failover) I think we need a very clear
> understanding of the functionality we are looking for the client to provide
> before we can decide which layer it is best placed in. If the functionality
> is likely to be common across the JMS and Qpid APIs then it would seem
> sensible not to write this code directly into the JMS layer.
>
> Before we start cutting any code I think we need general agreement in the
> Qpid Java community about the functionality that is required and where it
> is going to be implemented (and ideally we should also be coordinating by
> whom it is going to be implemented also :-) ).

Indeed! My intention was to get this conversation happening and for us
to start formulating a plan.
As I mentioned in my original email, a plan is very important, so we
know who's doing what and more importantly some milestones as to when
we can expect things to happen.

I agree with you about the main problem areas.
I also think we need to pay very close attention to our threading and
synchronization strategies as we have quite a few issues there.

We should put up a page on wiki to document the requirements and have
a conversation happening on the lists to come to an agreement.
We should then define milestones and deliverables. Without that it's
going to be meaningless.
Once we do that we could start coordinating about the code
contributions and adjust the plan if we feel we have resourcing
issues.

Rajith

> -- Rob
>
>
>
>> I would love to hear everybody's thoughts on this.
>> More importantly it's good if we could also discuss on a plan and set
>> some milestones to ensure we stay focused.
>>
>> Personally I would love to see a reasonably working prototype by 0.22.
>> If we can get something going for 0.20 that would be a bonus, even if
>> it's just experimental (preferably on a branch) and Alpha quality.
>>
>> I mentioned the above milestones to kick start the discussion and get
>> things rolling.
>> We could start on a branch and then move it to trunk during 0.22 if
>> everybody is satisfied with the progress.
>>
>> Regards,
>>
>> Rajith
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
>> For additional commands, e-mail: dev-h...@qpid.apache.org
>>
>>

AMQP 1.0 support for JMS client via Proton

[Rajith Attapattu]

Hi All,

There are a few folks who are keen to have AMQP 1.0 support for our JMS client.
Given that the parent AMQP TC is starting a bindings and mappings TC
which will cover JMS, I thought it would be a good idea to get a
discussion going on here as well.
We could aim to build the client as we progress through the TC and
provide feedback if necessary.

On the hand, we've had extensive discussions on building a new JMS
client from scratch when adding 1.0 support.
The primary motivation was to address some of the nagging issues
around the old client.

So far there have been two schools of thought on how to get AMQP 1.0 support.

1. JMS Client --> Qpid API --> Proton

2. JMS Client --> Proton

While option #1 seems like killing two birds with one stone, I think
we should seriously consider option #2 as well.

I would love to hear everybody's thoughts on this.
More importantly it's good if we could also discuss on a plan and set
some milestones to ensure we stay focused.

Personally I would love to see a reasonably working prototype by 0.22.
If we can get something going for 0.20 that would be a bonus, even if
it's just experimental (preferably on a branch) and Alpha quality.

I mentioned the above milestones to kick start the discussion and get
things rolling.
We could start on a branch and then move it to trunk during 0.22 if
everybody is satisfied with the progress.

Regards,

Rajith

Unit tests for Proton-J

[Rajith Attapattu]

While we should stick to python for writing tests (that is used across
proton-j and proton-c) as much tests as possible, we still need to
write junit tests on the java side.
In doing so we will have to have a lib directory to carry the junit jar.

I know we are quite keen to keep proton dependency free, but I suppose
this is an exception we are willing to make :)
What do you folks think ? Is this a good idea?

Regards,

Rajith

Java examples

[Rajith Attapattu]

I would like to commit the java version of the Mailbox example along
with the driver code.
I'm trying to determine the best location to place the files.
Any suggestion ?

I would like to include the examples in our first release.
In addition to showing how to use the driver it also doubles as a
great way to demonstrate interoperability.

I have verified that the Java examples works well with the current
python examples.

Regards,

Rajith

Re: release plans

[Rajith Attapattu]

I've already committed the bug fixes and is working on getting the
driver code in.
For the driver I plan to get it in, once I incorporate Rob's feedback.

Once I get that in, for the second phase I would like to work with Rob
to adjust the driver code to mirror the changes he's planning on the
transport side.
Hiram also has a patch that he's keen to push in sooner than later.

Maybe we might not be able to get the second phase in by the first release.
We could try to do more frequent releases initially to ensure we don't
rush things into make a deadline.

We could think about doing a few milestone releases (M1...Mx) monthly
or bi-monthly.
The frequent releases can also help in working out release and process
related issues (Ex documentation, license headers, files, notices
..etc).
So by the time we are ready to do the 1.0 release we will only need to
focus on the code as the process is sorted out.

We should also think about documentation. I see that as a major issues
leading upto a 1.0 release.

Rajith

On Fri, Sep 14, 2012 at 10:55 AM, Rob Godfrey  wrote:
> On 11 September 2012 20:42, Rafael Schloming  wrote:
>
>> Hi Everyone,
>>
>> It's about time (probably well past time) to put out a first release. Ken
>> is working on getting his SSL patch updated against the recent trunk
>> changes, and that should be ready to land very shortly. With that done I
>> think what we have so far (for proton-c at least) is ready for some real
>> world use. This seems like a good time to stabilize a little and work
>> through all the packaging issues necessary to get something out there that
>> is easily consumable. I'd like to propose doing a draft release at the end
>> of next week after the SSL patch lands.
>>
>> What are people's thoughts? Are there any
>> bugs/features/documentation/packaging issues/etc that should be addressed
>> for this?
>>
>> One obvious question is on proton-j. It needs a little bit of work to catch
>> up with some of the recent development on proton-c. Is this something that
>> could happen in the time frame mentioned above? Do we want to release both
>> at the same time or is it ok if proton-j lags a bit?
>>
>>
> So - I would like to release proton-j too... I've not had much time to work
> on it recently for which I apologies.  I'm going to try to clear some space
> to get things a bit more up to date next week - in particular to get the
> changes to the transport interface incorporated.  There's also work on the
> driver that Rajith has been doing that we need to integrate.
>
> Overall I think we need to have a much clearer (public) roadmap/plan/list
> of tasks of what we are doing in which langugae so that we can try to keep
> things better in sync and that multiple people can be working on it.
>
> -- Rob

Re: Ruby stable API design

[Rajith Attapattu]

On Thu, Sep 13, 2012 at 5:05 PM, Rafael Schloming  wrote:
> On Thu, Sep 13, 2012 at 4:46 PM, Rajith Attapattu wrote:
>
>> On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming 
>> wrote:
>> > On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce > >wrote:
>> > methods on Messenger and Message objects and by tying pn_messenger_free
>> and
>> > pn_message_free into the respective destructors, we could make things a
>> > whole lot safer, e.g. avoid dangling pointers and the like that a user
>> > could use to segfault the interpreter.
>>
>> Darryl, you could use %typemap(freearg) to ensure the respective free
>> function is called to cleanup.
>> How it gets called is also automatically handled by swig based on the
>> host language.
>>
>
> Maybe I'm missing something, but I don't think %typemap(freearg) helps in
> this case. From what I understand it's used to clean up temporary memory
> allocated for the purpose of argument processing. The situation I'm talking
> about is when you allocate an object in C and return it to the interpreter,
> e.g.:

Sorry I misunderstood. I thought it was for methods.

It appears Ruby does have support for %newobject
http://www.swig.org/Doc1.3/Ruby.html#Ruby_nn59
But it maybe simpler to use the strategy Rafi has used for the python binding.

> msg = pn_message()
>
> What swig returns to the interpreter here is a wrapper around a pointer. If
> that variable goes out of scope and the interpreter garbage collects it,
> then only the wrapper is freed, and the memory pointed to by the internal
> pointer is leaked.
>
> You can fix this by doing pn_message_free(msg) before it goes out of scope,
> however for a short time you end up with the wrapping object having an
> internal pointer that is invalid since it now points to freed memory. If
> you the try to do things you can cause the interpreter to seg fault just
> like the C runtime would seg fault if you were to try to use a pointer
> after freeing it.
>
> You can use %newobject and %delobject to address this, however I'm given to
> understand that support for that varies across target languages. It's
> fairly easy to work around with a destructor in the wrapper layer as I've
> done in the idiomatic python API layer.
>
> --Rafael

Re: Ruby stable API design

[Rajith Attapattu]

On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming  wrote:
> On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce wrote:
> methods on Messenger and Message objects and by tying pn_messenger_free and
> pn_message_free into the respective destructors, we could make things a
> whole lot safer, e.g. avoid dangling pointers and the like that a user
> could use to segfault the interpreter.

Darryl, you could use %typemap(freearg) to ensure the respective free
function is called to cleanup.
How it gets called is also automatically handled by swig based on the
host language.

Regards,

Rajith

Re: idiomatic python API

[Rajith Attapattu]

I would strongly favour shared documentation as much as possible.
The concepts , examples etc.. can be handled via a common document and
we might have to have some language specific sections.

Rajith

On Wed, Sep 12, 2012 at 3:42 PM, Rafael Schloming  wrote:
> Hey Everyone,
>
> I've put together a draft of an idiomatic python API that sits on top of
> the swig bindings. It's a very thin layer, all it does is translate from a
> procedural interface to an object oriented one, handling memory management
> and adding checked exceptions along the way. I've posted it here:
>
> https://issues.apache.org/jira/browse/PROTON-23
>
> Notably it does not have any docs yet, this is partly because I'm lazy and
> partly because it would be nice to figure out a way to share documentation
> between languages.
>
> Please check it out and let me know if you have any comments.
>
> --Rafael

Re: Proton engine api naming proposal

[Rajith Attapattu]

I must admit that when I looked at the code initially I did find it a
bit hard to navigate the code.
Once I figured out the pattern it was easy.

I understand the reasoning behind Rafi's choice of words for naming
the API methods.
On the java side some of the names were a bit odd, which looked ok on
the C side.
But I certainly understand the value of keeping the two
implementations as close as possible and naming plays a large part in
it.

I also agree with Justin that the API method names should be easily
understood without having to dig through the code.
A well named API is a huge asset in promoting Proton. We want people
to dive right into it without having to scratch heads.
I did have the advantage of asking questions from Rafi. Without his
help I may have struggled a bit.

We still haven't made a release yet and now is a good time to have a
discussion, rather than a few releases down the line.
(I'm not asking to hold onto the current release plans, but asking
that we rectify the situation sooner than later).

My honest appraisal is that the API is a bit difficult to understand.
Perhaps what we need is good documentation and not a name change.
Or perhaps we need both. I'm not quite sure yet.

I'm planning to write a small tutorial to figure this out myself. I
might be able to provide more detailed insight after that.

Regards,

Rajith

On Thu, Sep 13, 2012 at 1:29 PM, Rafael Schloming  wrote:
> On Thu, Sep 13, 2012 at 12:07 PM,  wrote:
>
>> On Thu, 13 Sep 2012, Rafael Schloming wrote:
>>
>>  On Thu, Sep 13, 2012 at 9:36 AM, Justin Ross  wrote:
>>>
>>>
 On Thu, 13 Sep 2012, Ted Ross wrote:

  I'm not crazy about the work-processing function names as they seem to

> disregard the grammar.  Should they not all be pn_connection_*
>> functions?
>>
>>
>  I agree about this.  I would definitely prefer to see pn_connection_*
 for
 the connection-scoped work interfaces.  I guess I thought I was pressing
 my
 luck, :).

>>>
>>>
>>> I think there actually already is a consistent rule here, it's just
>>> missing
>>> from your grammar. Wherever there is a linked list of things, the API uses
>>> the form:
>>>
>>> pn__head(pn_**_t)
>>> pn__next(pn_<**element_t>_t)
>>>
>>> I think this is better than trying to stick it all on the root or all on
>>> the element or splitting it up between the two. For example I think
>>> pn_work_head is better than pn_head_delivery as the latter gives you less
>>> information. The fact that it is a delivery is already contained in the
>>> type signature, and there are multiple lists of deliveries maintained by
>>> the engine, so just knowing that it is a list of deliveries isn't
>>> sufficient. Even scoping it to the connection is not terribly useful as
>>> there may well be multiple lists of deliveries on the connection. The
>>> relevant info here is that it is the head of the work queue, a concept
>>> that
>>> we actually do (or should) explain at length (somewhere). I would argue
>>> that the work queue is actually the relevant concept/noun here, it just
>>> doesn't have it's own lifecycle since it is a component of the connection.
>>>
>>
>> There's the connection-scoped work queue and then there's what appears to
>> be a link-scoped work queue, pn_current/pn_advance.  Should it also have
>> "work" in its name?
>>
>
> I would say no,  I was actually going to comment on your classification
> there. There are two linked lists of deliveries:
>
> 1) The work queue.
>
> This is scoped to the connection and provides efficient access to
> deliveries that are readable, writable, or have had their remote
> disposition updated, i.e. deliveries for which some work is expected to
> happen. The contents of this list are dynamically updated as conditions
> change, e.g. if the remote peer grants or revokes credit or sends more
> message data or updates the disposition of a delivery, the contents of the
> work queue will change automatically.
>
> 2) The unsettled deliveries on a link.
>
> This set of deliveries is scoped to the link, and unlike the work queue it
> is not dynamic. It simply contains every delivery for which the link is
> tracking state and it is ordered in the exact order that those deliveries
> were created, i.e. the send order for outgoing links, and the receive order
> for incoming links. Put another way this is the outgoing delivery list for
> senders and the incoming delivery list for receivers.
>
> The pn_current and pn_advance interface doesn't represent a list, but
> rather a pointer to a specific delivery within the list of unsettled
> deliveries. I would say accessing this and advancing it would fall under
> methods on the link, i.e. pn_link_current() and pn_link_advance() in your
> proposed scheme.
>
> Now where the notion of the current delivery does relate to the work queue
> is that you can only send and receive data from the current delivery on a
> link, so if a delivery is in the work queue because it is readable

[jira] [Commented] (PROTON-16) addTransportWork and addWork methods in ConnectionImpl.java could cause an infinite loop

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450062#comment-13450062
 ] 

Rajith Attapattu commented on PROTON-16:


I plan to cherry-pick the following commit to trunk to fix the above issue.
http://svn.apache.org/viewvc?view=revision&revision=1366219

> addTransportWork and addWork methods in ConnectionImpl.java could cause an 
> infinite loop
> 
>
> Key: PROTON-16
> URL: https://issues.apache.org/jira/browse/PROTON-16
> Project: Qpid Proton
>  Issue Type: Bug
>    Reporter: Rajith Attapattu
>
> When adding a head or tail with non null pointers for next and previous the 
> resulting link list will loop infinitely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PROTON-16) addTransportWork and addWork methods in ConnectionImpl.java could cause an infinite loop

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-16:
--

 Summary: addTransportWork and addWork methods in 
ConnectionImpl.java could cause an infinite loop
 Key: PROTON-16
 URL: https://issues.apache.org/jira/browse/PROTON-16
 Project: Qpid Proton
  Issue Type: Improvement
Reporter: Rajith Attapattu


When adding a head or tail with non null pointers for next and previous the 
resulting link list will loop infinitely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PROTON-16) addTransportWork and addWork methods in ConnectionImpl.java could cause an infinite loop

[Rajith Attapattu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PROTON-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajith Attapattu updated PROTON-16:
---

Issue Type: Bug  (was: Improvement)

> addTransportWork and addWork methods in ConnectionImpl.java could cause an 
> infinite loop
> 
>
> Key: PROTON-16
> URL: https://issues.apache.org/jira/browse/PROTON-16
> Project: Qpid Proton
>  Issue Type: Bug
>    Reporter: Rajith Attapattu
>
> When adding a head or tail with non null pointers for next and previous the 
> resulting link list will loop infinitely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PROTON-14) addModified method in ConnectionImpl can cause infinite loop

[Rajith Attapattu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PROTON-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajith Attapattu updated PROTON-14:
---

Summary: addModified method in ConnectionImpl can cause infinite loop  
(was: add modified method in ConnectionImpl can cause infinite loop)

> addModified method in ConnectionImpl can cause infinite loop
> 
>
> Key: PROTON-14
> URL: https://issues.apache.org/jira/browse/PROTON-14
> Project: Qpid Proton
>  Issue Type: Bug
>  Components: proton-j
>    Reporter: Rajith Attapattu
>
> When adding a new Head, you need to clear any pointers that it
> has to it's next and prev and also when adding an item to the tail it
> should clear it's next. The above bug resulted in infinite loops.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-15) Remove get/set Prev/Next methods from EndpointImpl.java as they seem redundant/unused

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-15?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450054#comment-13450054
 ] 

Rajith Attapattu commented on PROTON-15:


I'm planning to cherry-pick the following commit to trunk to address the above.
http://svn.apache.org/viewvc?view=revision&revision=1365737

> Remove get/set Prev/Next methods from EndpointImpl.java as they seem 
> redundant/unused
> -
>
> Key: PROTON-15
> URL: https://issues.apache.org/jira/browse/PROTON-15
> Project: Qpid Proton
>  Issue Type: Improvement
>Reporter: Rajith Attapattu
>Priority: Minor
>
> _next and _prev fields handled by the above methods does not seem to be 
> set/used properly causing errors in other parts of the code.
> After further investigation it appears the _transportPrev and _transportNext 
> fields are the correct fields to use.
> The issues that was caused by the above are fixed when switched to the 
> methods that use _transportPrev/Next

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PROTON-15) Remove get/set Prev/Next methods from EndpointImpl.java as they seem redundant/unused

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-15:
--

 Summary: Remove get/set Prev/Next methods from EndpointImpl.java 
as they seem redundant/unused
 Key: PROTON-15
 URL: https://issues.apache.org/jira/browse/PROTON-15
 Project: Qpid Proton
  Issue Type: Improvement
Reporter: Rajith Attapattu
Priority: Minor


_next and _prev fields handled by the above methods does not seem to be 
set/used properly causing errors in other parts of the code.

After further investigation it appears the _transportPrev and _transportNext 
fields are the correct fields to use.
The issues that was caused by the above are fixed when switched to the methods 
that use _transportPrev/Next

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PROTON-14) add modified method in ConnectionImpl can cause infinite loop

[Rajith Attapattu (JIRA)]

[ 
https://issues.apache.org/jira/browse/PROTON-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450048#comment-13450048
 ] 

Rajith Attapattu commented on PROTON-14:


A fix has been committed here 
http://svn.apache.org/viewvc?view=revision&revision=1365736
I'm planning to commit this fix to trunk (minus the dumpList() ) 

> add modified method in ConnectionImpl can cause infinite loop
> -
>
> Key: PROTON-14
> URL: https://issues.apache.org/jira/browse/PROTON-14
> Project: Qpid Proton
>  Issue Type: Bug
>  Components: proton-j
>Reporter: Rajith Attapattu
>
> When adding a new Head, you need to clear any pointers that it
> has to it's next and prev and also when adding an item to the tail it
> should clear it's next. The above bug resulted in infinite loops.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PROTON-14) add modified method in ConnectionImpl can cause infinite loop

[Rajith Attapattu (JIRA)]

Rajith Attapattu created PROTON-14:
--

 Summary: add modified method in ConnectionImpl can cause infinite 
loop
 Key: PROTON-14
 URL: https://issues.apache.org/jira/browse/PROTON-14
 Project: Qpid Proton
  Issue Type: Bug
  Components: proton-j
Reporter: Rajith Attapattu


When adding a new Head, you need to clear any pointers that it
has to it's next and prev and also when adding an item to the tail it
should clear it's next. The above bug resulted in infinite loops.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: driver/transport/sasl refactor

[Rajith Attapattu]

On Tue, Sep 4, 2012 at 9:16 PM, Rafael Schloming  wrote:
> On Tue, Sep 4, 2012 at 5:53 PM, Rajith Attapattu  wrote:
>
>> I'm trying to figure out what changes are needed on the Java side.
>> It seems the bind method will be of interest.
>>
>> Rafi, could you also explain how the refactor is going to help with SSL ?
>>
>
> Sure, the idea is to integrate the SSL implementation into the transport
> object similarly to the way SASL is integrated in the patch. This way it is
> separated from the driver, and porting of the driver won't conflict with
> the SSL implementation, i.e. cliff's windows port and ken's ssl branch
> won't tread on each other's toes.

Ok make sense now. Thanks for the explanation.

Rajith

>
> --Rafael

Re: driver/transport/sasl refactor

[Rajith Attapattu]

I'm trying to figure out what changes are needed on the Java side.
It seems the bind method will be of interest.

Rafi, could you also explain how the refactor is going to help with SSL ?

Rajith

On Tue, Sep 4, 2012 at 5:09 PM, Rafael Schloming  wrote:

> Hi Everyone,
>
> I've done some exploratory coding on a small refactor aimed at simplifying
> the driver in order to ease both portability and ssl integration. I've
> attached the patch. It's not huge, but it removes some of the trickiest
> code from driver.c. I believe there are further simplifications that could
> be made on top of this, but I think this represents the minimal coherent
> step. This involves the following API modifications:
>
> Changed:
>   pn_sasl_t *pn_sasl() -> pn_sasl_t *pn_sasl(pn_transport_t *transport)
>   pn_transport_t *pn_transport(pn_connection_t *connection) ->
> pn_transport_t *pn_transport(void)
>
> Added:
>   int pn_transport_bind(pn_transport_t *transport, pn_connection_t
> *connection)
>
> Note that a transport can now exist independently from a connection and
> must therefore also be explicitly freed.
> Note that the sasl object does *not* need to be explicitly freed as it
> will be automatically freed when the transport is freed.
>
> If you're using the messenger API or the driver and engine API together,
> then there will be no visible changes.
>
> Please have a look and give me any comments I'll be tying up some loose
> ends and looking to commit this soon.
>
>
> --Rafael
>

SSL support

[Rajith Attapattu]

Rafi,

>From what I understand there are two ways to use SSL/TLS with AMQP 1.0

a) A secure connection is established right off the bat.

b) A regular tcp connection is established and then based on the AMQP
header (with a protocol id of 2) you start encrypting the packets that
follow.

The first option seems fairly straightforward. For the second,

1.  Once we get the AMQP header indicating we need to use TLS, should
we expect the SASL header to follow ?
2.  If we have TLS connection, is SASL authentication optional ?

Additionally can the peers chose to use SASL encryption? If so how do
we figure that out?
I didn't find that information under "Book 5 : Security section "

Regards,

Rajith